GitLab Runner is perfectly capable of spawning multiple different runners from a single instance. It just need to register itself into GitLab few times. Unfortunately the GitLab Runner Helm chart supports only single registration call. On the other hand - registration scripts are executed from ConfigMap and there is even a pre-entrypoint-script hook ready. In the end it’s a matter of a few changes in ConfigMap to make GitLab Runner register per-architecture runners with different tags.

Enter the Qbec

The qbec project page describes it as a tool to configure and create Kubernetes objects on multiple environments. Sure, but to me its real strength comes from the fact that it’s based on jsonnet. Let’s see how it will help with the necessary changes to the GitLab Runner Helm chart.

Rendering the Helm chart

There are two ways now to render Helm chart in qbec:

• deprecated native function expandHelmTemplate
• custom data importer

This article and the underlying project still uses the older expandHelmTemplate function.

Working with the chart objects

Rendering Helm chart with expandHelmTemplate function returns an array of Kubernetes objects. It’s not a particularly handy format if you want to apply some changes to them. Two helper functions from k8slab-kube-libsonnet project address this problem:

• arrayByKindAndName changes array of K8S objects into object.<Kind>.<Name> tree-like structure
• arrayFromKindAndName reverses the above process

The intermediate structure can be easily navigated by the getByPath function.

local k8slab = import 'k8slab-kube-libsonnet/k8slab.libsonnet';
local expandHelmTemplate = std.native('expandHelmTemplate');

local chartPath = ...;
local chartValues = ...;
local chartOpts = {
  nameTemplate: ...,
  namespace: ...,
  thisFile: std.thisFile,
  verbose: true,
};

local renderedChart = expandHelmTemplate(chartPath, chartValues, chartOpts);
local renderedChartTree = k8slab.arrayByKindAndName(renderedChart);

local configMapName = 'xxx';

local changes = {
  ConfigMap+: {
    [configMapName]+: {
      data+: {
        'some-added-data-field': k8slab.getByPath(renderedChartTree, 'ConfigMap.xxx.data.yyy', ''),
      }
    }
  }
};

local output = k8slab.arrayFromKindAndName(renderedChartTree + changes);
output

You can see the described functions in action:

• Helm chart rendered and converted to intermediate structure
• reverse operation just before returning objects from the component

Customizations

The Helm chart values parameter holds a dummy runners.config entry. Real entries are generated from the template string and parameters. This template is a multi-line string with %(name)s placeholders as you can see. There are two sets of parameters in the arm64 and the amd64 fields. They are made of some common parameters possibly overwritten by per-environment values and the final local customizations.

Rendered templates are put into the GitLab Runner ConfigMap object.

With the multiple runners.config entries prepared now it’s time for the registration hook.

The original code is replaced with exit 0. Slightly patched version is kept under different name in the ConfigMap - it’s modified to use /configmaps/${CONFIG_TEMPLATE_K8SLAB} configuration file instead of hard coded /configmaps/config.template.toml path. This changed registration script is called multiple times with different CONFIG_TEMPLATE_K8SLAB and RUNNER_TAG_LIST variables in the pre-entrypoint-script.

That’s it - now a single manager Pod registers two GitLab runners.

Final notes

Later k8slab-infra-gitlab-runners versions are slightly more complicated as the code now is better suited for more than two registrations and have some security improvements. The links in this article points to a simpler version which should be easier to understand if you don’t have much experience with jsonnet, especially with features like object and array comprehension.