Skip to content
Stribog

CI/CD

All writing

GitHub Self-Hosted Runners on Kubernetes: Owning CI Compute with ARC

Run GitHub self-hosted runners on Kubernetes with actions-runner-controller (ARC): ephemeral, autoscaling, network-isolated CI compute you own — without leaving GitHub.

Stribog17 min read
sovereigntyoptionalityaudit

One component in your delivery chain quietly holds more trust than almost anything else you operate, and most teams have never looked at where it runs. It is the CI runner — the machine that checks out your source, mounts your deployment credentials, pulls your dependencies, and executes whatever a workflow file tells it to. On GitHub's hosted runners that machine is a fresh VM in GitHub's infrastructure, in a region GitHub chooses, running third-party actions you mostly did not write. For many teams that is a reasonable trade; for teams in regulated environments, or simply teams that have priced the dependency honestly, it is the part of the pipeline most worth owning first.

This is deliberately narrower than leaving GitHub altogether. We have written the full version of that — Forgejo, Woodpecker, and Zot as a complete GitHub Actions exit — and for some organizations it is the right call, but it is a larger project: migrating your source of truth, code review, and issue history all at once. There is a middle posture that captures most of the security and cost benefit for a fraction of the disruption — keep GitHub as the forge, move only the runner — and the tool that makes it practical at scale is actions-runner-controller, ARC, which GitHub itself now maintains.

This is the concrete version of that middle path — why the runner is the part to own first, what ARC is, why ephemeral runners are a security property and not just autoscaling, the config to install and isolate them, the awkward problem of building containers inside a containerized runner, and how to keep ARC from becoming its own lock-in. The throughline is the practice's thesis: sovereignty is not the absence of dependencies, but dependencies you can describe, constrain, and exit on your own schedule.

Why the Runner Is the Part You Should Own First

Of all the boxes in a CI/CD diagram, the runner holds the most concentrated trust. A single workflow run can legitimately access a full checkout of your source, the secrets that workflow declares, often a token with write access back to the repository, and — for any job that deploys — credentials to a registry and a cluster. A hosted runner holds all of that, briefly, inside infrastructure you do not operate, while executing a workflow that almost certainly calls third-party actions pinned, if you are lucky, to a tag. It is where your most sensitive material and the largest volume of code you did not write meet, with network access, at the same instant.

That concentration is what made the March 2025 supply-chain compromise so instructive. The widely used tj-actions/changed-files action was backdoored (CVE-2025-30066): an attacker rewrote its version tags to point at malicious code, so even workflows pinned to a tag suddenly ran it, dumping runner memory — secrets included — into build logs across more than 23,000 repositories. We covered the mechanics in the forge-exit piece; the point to carry here is structural. A shared runner that executes mutable third-party code with your credentials in scope has a blast radius equal to every secret it can see — and you do not control it. Owning the runner does not make supply-chain risk vanish, but it converts an open-ended exposure into one you scoped on purpose: you decide which images exist, you pin them, and you decide what a job's network can reach.

The second reason is economics, and it is less dramatic but more constant. Hosted minutes are billed per minute of wall-clock time, and the meter runs hardest on exactly the teams doing the most engineering — large matrices, frequent merges, big monorepos — while self-hosted runners carry no per-minute charge from GitHub at all. We treat the full cost argument in its own section below; the point here is only that the most expensive runners are the busiest ones, and busy is what serious engineering looks like.

The third reason is residency and auditability, and it turns a preference into a requirement. If your compliance regime cares where regulated data is processed — and the convergence of NIS2, DORA, and the EU AI Act means more teams now must — then a build that decrypts secrets and touches production data on a hosted runner in an unspecified region is a control you cannot fully describe. A runner that is a pod in your cluster, in your jurisdiction, behind your network policy, is one you can point an auditor at — part of the boundary you already document, not a third party you must trust and footnote.

What ARC Actually Is

actions-runner-controller is a Kubernetes operator that runs self-hosted GitHub Actions runners as pods and scales them with the job queue. It began as a community project (the summerwind controller) and is now maintained by GitHub itself under `actions/actions-runner-controller`, which matters for a decade-horizon dependency: the operator that runs your CI is governed by the same vendor that defines the runner protocol it speaks — GitHub even lists ARC on Kubernetes as a recommended pattern in its own Well-Architected library. The current, recommended architecture is called Autoscaling Runner Scale Sets, and it is worth understanding precisely because it is a meaningful improvement over the older webhook-driven model.

A scale set is a named pool of identical runners registered against a repository, organization, or enterprise. You install one controller per cluster, then one gha-runner-scale-set Helm release per pool. Each scale set gets a listener — a small pod that authenticates to the GitHub Actions Service and long-polls it for jobs assigned to that pool; when one is assigned, the controller creates an ephemeral runner pod to take exactly that job. Crucially, the listener reaches *out* to GitHub; GitHub never reaches *in*. There is no inbound webhook to expose, no ingress to secure, no public endpoint — the only connections crossing your cluster boundary are outbound HTTPS from components you run.

GitHub keeps the forge and the job queue; the runner — the only component that touches your source, your secrets, and your network — runs inside your cluster. The listener reaches out to GitHub over outbound HTTPS; nothing reaches in, so there is no webhook endpoint to expose or secure.

This polling model is the quiet headline. Legacy ARC scaled by receiving GitHub webhooks into a HorizontalRunnerAutoscaler — an endpoint to expose and delivery reliability to reason about. The scale-set listener inverts that, pulling work from GitHub's own assignment API so scaling follows the authoritative queue rather than webhook events you hope arrive. Nothing changes for the workflow author — a job still targets the pool with runs-on: <scale-set-name> — but the operational surface shrinks to outbound calls and ordinary pods.

The controller authenticates with either a GitHub App or a PAT, and an App is the right answer beyond a quick trial: it scopes precisely to the org or repositories you choose, issues short-lived installation tokens rather than a long-lived secret, and is auditable as its own identity. Its credentials live in a Kubernetes secret the controller reads; the runners themselves register with just-in-time credentials minted per pod, so no durable registration token sits on disk. That per-job registration is what lets the security model that follows actually hold.

Ephemeral by Default: Why It Is a Security Property

The most important word in the ARC scale-set model is *ephemeral*. Each runner pod runs exactly one job and is then destroyed; the next job gets a brand-new pod with a fresh filesystem. This is usually sold as autoscaling, but the deeper value is what it removes. A persistent, reused runner accumulates state — a poisoned dependency cache, a leftover credential file, a modified PATH, a container from a previous job — any of which can bleed from one run into the next, the very mechanism by which a compromised build of one project attacks the build of another sharing the runner. Ephemeral runners make that cross-job contamination structurally impossible: there is no next job on this machine, because this machine ceases to exist.

Each job gets a fresh pod that registers just in time, runs once behind a default-deny egress policy, and is destroyed — no reused state between jobs, and the pool scales back to zero when the queue is empty.

This matters most for the hardest case in CI: pull requests from forks, which run code you have not reviewed by definition. On a shared, persistent runner that is a standing invitation to pivot — read another job's secrets, tamper with a cache, persist a foothold. On an ephemeral pod with no secrets mounted for untrusted events and no neighbours to attack, the worst a malicious PR can do is misuse its own short-lived sandbox, which is then deleted. ARC does not solve fork-PR risk — you still gate which events get secrets and scope the runner's network — but it removes the shared-state amplifier that turns one bad job into many.

Installing ARC: Controller, Scale Set, and a GitHub App

The install is two Helm releases and a secret, and it is worth reading the values rather than copying a quickstart, because the values are where the sovereignty decisions live. First the controller, once per cluster:

bash
# 1. Install the ARC controller (once per cluster). Charts are published as OCI
#    artifacts under the GitHub actions org — mirror them into your own registry
#    for an air-gapped or pinned install rather than pulling at deploy time.
helm install arc \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller \
  --version 0.12.1 \
  --namespace arc-systems --create-namespace

# 2. Store the GitHub App credentials the controller uses to manage runners.
#    Prefer a GitHub App over a PAT: scoped, short-lived tokens, its own audit identity.
kubectl create secret generic arc-github-app \
  --namespace arc-runners \
  --from-literal=github_app_id=123456 \
  --from-literal=github_app_installation_id=7890123 \
  --from-file=github_app_private_key=./arc-app.private-key.pem
The controller and its GitHub App credentials. Pin the chart version and, for a serious deployment, mirror the OCI chart into your own registry so an install does not depend on pulling from a registry you do not control at the moment you deploy.

Then a runner scale set — one Helm release per pool. The values.yaml names the GitHub scope, binds the credentials, sets the autoscaling bounds, and — importantly — lets you replace the default runner image and pod spec with your own. That pod-template override is the hook through which every other control here attaches: your hardened image, node selectors, security context, volumes.

yaml
# values.yaml for one gha-runner-scale-set release.
# githubConfigUrl can point at a repo, an org, or an enterprise.
githubConfigUrl: "https://github.com/your-org"
githubConfigSecret: arc-github-app

# Autoscaling bounds. minRunners: 0 = scale to zero when the queue is empty;
# maxRunners caps how much hardware CI can ever consume.
minRunners: 0
maxRunners: 24

# The runner pod is yours to define. Replace the default image with a pinned,
# hardened one mirrored into your own registry, and lock the security context.
template:
  spec:
    securityContext:
      runAsNonRoot: true
      seccompProfile:
        type: RuntimeDefault
    containers:
      - name: runner
        image: registry.internal.example.com/ci/arc-runner:2.319.1
        command: ["/home/runner/run.sh"]
        resources:
          requests: { cpu: "1", memory: "2Gi" }
          limits:   { cpu: "4", memory: "8Gi" }
A runner scale set pinned to your own image and a non-root, seccomp-confined security context. Because the pod spec is fully yours, the runner inherits every cluster control you already run — node isolation, resource limits, admission policy — instead of being an opaque managed VM.

Workflows then target the pool by its release name: a repository that said runs-on: ubuntu-latest says runs-on: arc-linux-amd64, and nothing else changes. That single-line switch makes the migration incremental — move one repository, one workflow, even one job at a time onto owned compute, and leave the rest on hosted runners until you are satisfied. There is no big-bang cutover, because GitHub still orchestrates the workflow; only the execution moved.

The Container-in-a-Container Problem

Here is the wrinkle every team hits and most quickstarts gloss over: CI frequently needs to *build containers*, and your runner is now itself a container in a pod. Building an image, running a service container, or using a container: job all assume a working container runtime — but there is no Docker daemon in a vanilla pod, and handing one over carelessly is how a CI cluster becomes a privilege-escalation playground. ARC gives two supported answers, and the choice has real security consequences.

The first is Docker-in-Docker (containerMode: dind): a Docker daemon runs as a sidecar and jobs talk to it as a local daemon. It is the most compatible option — almost any image-building workflow works unchanged — but it requires a privileged container, widening the pod's attack surface in exactly the way a careful platform team is reluctant to allow. The second is Kubernetes mode (containerMode: kubernetes), where the runner uses the Actions container hooks to launch each job's containers as their own pods rather than nested Docker containers. It avoids the privileged daemon but needs a ReadWriteMany volume to share the workspace between runner and job pods, and a few patterns that assume a local Docker socket need adjusting.

yaml
# Kubernetes container mode: job containers run as their own pods, no privileged DinD.
# Requires a ReadWriteMany storage class for the shared work volume.
containerMode:
  type: "kubernetes"
  kubernetesModeWorkVolumeClaim:
    accessModes: ["ReadWriteMany"]
    storageClassName: "nfs-csi"
    resources:
      requests:
        storage: 10Gi
Kubernetes container mode avoids the privileged Docker-in-Docker daemon by running each job container as its own pod, at the cost of needing shared ReadWriteMany storage. It is the more defensible posture for a multi-tenant CI cluster.

For the common case of building images there is a better answer than either: a daemonless builder. Kaniko, Buildah, or rootless BuildKit build OCI images in an unprivileged pod with no Docker daemon at all, sidestepping the privilege question entirely. It is the same hygiene we apply on the deploy side of the supply chain — build without privilege, pin every base image, sign the result — pushed one stage earlier into the runner. The honest guidance: default to Kubernetes mode or a rootless builder, and accept Docker-in-Docker only where a workflow genuinely needs the daemon and you have isolated those runners on their own nodes.

Network Isolation and Controlled Egress

Owning the runner only pays off if you also own its network. A hosted runner can talk to the entire internet — convenient, and for a process executing third-party code with your secrets, alarming. Because an ARC runner is an ordinary pod, it is subject to the same NetworkPolicy enforcement as everything else in your cluster, and a runner is a near-perfect candidate for default-deny egress: a build job needs a small, knowable set of destinations — the GitHub Actions API to fetch the job and report logs, your registry and package mirrors for dependencies, little else. Everything beyond that list is, for a CI job, either a mistake or an exfiltration channel.

yaml
# Default-deny egress for runner pods; allow only DNS, the GitHub Actions API,
# and your internal registry/mirror. Pair with a CIDR/FQDN policy (e.g. Cilium)
# for egress to GitHub's published IP ranges.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: arc-runner-egress
  namespace: arc-runners
spec:
  podSelector:
    matchLabels:
      actions.github.com/scale-set-name: arc-linux-amd64
  policyTypes: ["Egress"]
  egress:
    - to:                                   # cluster DNS
        - namespaceSelector: {}
          podSelector:
            matchLabels: { k8s-app: kube-dns }
      ports: [{ protocol: UDP, port: 53 }]
    - to:                                   # internal registry + package mirror
        - ipBlock: { cidr: 10.42.0.0/16 }
      ports: [{ protocol: TCP, port: 443 }]
A default-deny egress baseline: the runner may reach cluster DNS and your internal registry, and nothing else by default. Egress to GitHub's own API is best expressed with an FQDN- or CIDR-aware policy engine rather than hand-maintained IP blocks.

Plain Kubernetes NetworkPolicy works on CIDR blocks, awkward for a destination like GitHub that publishes a changing set of IP ranges. This is where an eBPF dataplane earns its place: with Cilium's identity-aware policies you write egress rules against DNS names and let the dataplane resolve and enforce them, so "this runner may reach *.actions.githubusercontent.com and the internal registry, full stop" becomes a maintainable policy. Combined with per-scale-set namespaces and node isolation for the riskier Docker-in-Docker pools, each job's reachable surface becomes something you declared, not something you inherited.

Cost Control: Pay for Hardware, Not Minutes

The economic case for owned runners is not that compute is free — it plainly is not — but that you change what you are buying and who controls the price. Hosted minutes are a variable cost metered on every second of every job, growing with engineering throughput: more contributors, larger matrices, fatter monorepos. Owned runners convert that into a capacity cost you provision and amortize. With ARC's scale-to-zero the marginal cost of an idle pipeline is nothing, and the marginal cost of a busy one is hardware you already sized — hardware that, between CI bursts, runs other workloads rather than sitting as an idle runner fleet.

The structural advantage is that CI becomes one more tenant of a cluster you already operate, sharing capacity and subject to the same right-sizing discipline — the question shifts from "how many minutes did we burn" to "how well is our cluster utilized," which a platform team can actually act on. A late-2025 episode makes the governance point budgetary: GitHub floated charging a per-minute fee even for self-hosted runners, then reversed it within days. The fee never landed, but the lesson did — a platform whose pricing can change under you is a planning risk, and the antidote is the standing ability to absorb your own CI load on infrastructure whose costs you control.

None of this argues for moving every workload off hosted runners reflexively. The honest analysis is per-workload: low-volume repositories, occasional jobs, and workflows that need a clean public IP or a platform you have no appetite to operate may well belong on hosted runners, and keeping them there is a sovereign choice when made deliberately. ARC benefits most the teams whose CI is both high-volume and high-trust — where the meter is large and the material the runner touches is sensitive. For them owned runners are usually cheaper and always more controllable; the cloud-repatriation discipline of pricing the whole operating model, not just the sticker, applies here in miniature.

Keeping ARC From Becoming Its Own Lock-In

The recurring failure of self-hosting projects is to escape one dependency and quietly build another, so it is worth being clear-eyed about where ARC sits. ARC runs the standard, open-source GitHub Actions runner agent inside ordinary pods: no proprietary build format, no bespoke pipeline DSL — your workflows are still GitHub Actions YAML, your runners the same actions/runner binary GitHub ships, and the operator is open source under GitHub's own repository. Removing ARC means deleting two Helm releases; the workflows it executed do not change.

The dependency you retain is GitHub Actions itself — the workflow syntax, the marketplace, the orchestration. That is the deliberate trade of this middle path: sovereignty over the *compute* without sovereignty over the *forge*. For many teams that is exactly the right scope, and the honest way to hold it is as a known, bounded dependency rather than pretending it is gone. If GitHub itself ever becomes the dependency you must exit, the work is already partly done — the hardware, network policy, registry, and container-build tooling are all yours and portable. Only the orchestration remains to move, which is precisely the Forgejo, Woodpecker, and Zot exit we wrote up separately, and far easier to reach from owned runners than from hosted ones.

  • Workflows stay portable YAML. GitHub Actions files today, but the work each step does — checkout, build, test, push — is engine-agnostic, so re-expressing them on Woodpecker or another runner later is mechanical, not a rewrite.
  • The runner is the upstream actions/runner binary. ARC orchestrates it; it does not replace it. You can run that same runner outside Kubernetes if you ever need to.
  • The compute, network, and registry are already yours. The durable assets of a sovereign pipeline — owned hardware, default-deny egress, a pinned image mirror — are built once and survive any later change of orchestrator.
  • Exit is removing two Helm releases. No data is trapped in ARC; uninstalling returns you to hosted runners with a one-line runs-on change, so the experiment is cheap to reverse.

The Long Game: CI Compute as Owned Infrastructure

The runner is load-bearing infrastructure most organizations have never treated as such: it executes with the highest privileges in the delivery chain, runs the most third-party code, and on hosted infrastructure is the layer you can describe least precisely. Bringing it into a cluster you operate — ephemeral pods, behind a network policy, on hardware you size — is one of the highest-leverage sovereignty moves available, because it captures most of the security and cost benefit of a full forge exit at a fraction of the disruption. It is the platform-engineering golden-path discipline applied to the build tier: make the secure, owned way the default way.

As with everything this practice argues, the move should be proportional to your ability to operate it. A team that cannot reliably run a Kubernetes cluster has no business betting its CI on one, and a deliberate decision to keep using hosted runners — with a tested plan to move when the time comes — is itself a sovereign posture. The anti-pattern is not staying on hosted runners; it is drifting deeper into a dependency you have never priced, never tested an exit from, and could not describe to an auditor. ARC's value is that it makes the owned alternative ordinary: standard runners, standard pods, standard policy.

We didn't move our runners in-house because we were angry at our CI bill. We moved them because we couldn't answer a simple audit question — what can your build process reach on the network — and a hosted runner made that unanswerable. Now the answer is a NetworkPolicy in a pull request. The cost savings were real, but the control was the point.
Platform lead, regulated fintech — anonymized

That is the sovereign posture for CI compute in one idea: keep the forge that serves you well, but own the machine that touches your code. Self-hosted GitHub Actions runners on Kubernetes, run with ARC, give you ephemeral execution you can isolate, autoscale you can cap, and a runner you can describe — without the upheaval of leaving GitHub. Build it that way and the next change in someone else's pricing, terms, or region policy becomes a memo you read, not an emergency you absorb. The dependency that remains is one you chose on purpose, scoped on purpose, and can exit on your own schedule — which is the only kind worth keeping.

§FAQ/Common questions

Frequently asked

What is the difference between GitHub-hosted and self-hosted runners on Kubernetes?

GitHub-hosted runners are ephemeral virtual machines provisioned and billed per minute inside GitHub's own infrastructure, in a region GitHub controls, with internet egress GitHub defines. Self-hosted runners on Kubernetes — managed by actions-runner-controller (ARC) — are pods in a cluster you operate: you choose the hardware and region, you pin the runner image, you enforce network policy and security context, and there is no per-minute charge from GitHub. The workflow itself is unchanged; only the execution environment moves. The trade is that you now operate the runners, so the choice comes down to whether your CI volume, security requirements, and operating capability justify owning that compute.

Is actions-runner-controller (ARC) officially supported by GitHub?

Yes. ARC started as a community project (the summerwind controller) and is now maintained by GitHub under the actions/actions-runner-controller repository. The current recommended architecture is Autoscaling Runner Scale Sets, installed via the gha-runner-scale-set-controller and gha-runner-scale-set Helm charts. The legacy webhook-driven mode still exists, but new deployments should use scale sets, which scale by having a listener long-poll the GitHub Actions Service for assigned jobs rather than by receiving inbound webhooks.

Why are ephemeral runners more secure than persistent self-hosted runners?

An ephemeral runner runs exactly one job and is then destroyed, so the next job gets a brand-new pod with a clean filesystem. A persistent, reused runner accumulates state between jobs — caches, leftover credentials, modified environment, running containers — and that residual state is the mechanism by which a compromised build of one project can attack a later build that shares the runner. Ephemeral pods make cross-job contamination structurally impossible because there is no next job on the same machine. This matters most for pull requests from forks, which run unreviewed external code; on an ephemeral pod with no neighbours and no mounted secrets for untrusted events, the blast radius is a single throwaway sandbox.

How do I build container images inside an ARC runner without a privileged Docker daemon?

There are two supported container modes and one preferred pattern. Docker-in-Docker mode (containerMode: dind) runs a Docker daemon as a sidecar and is the most compatible, but it requires a privileged container, which widens the attack surface. Kubernetes mode (containerMode: kubernetes) runs each job container as its own pod via the Actions container hooks, avoiding the privileged daemon but requiring a ReadWriteMany shared work volume. For the common case of building images, a daemonless builder — Kaniko, Buildah, or rootless BuildKit — builds OCI images in an unprivileged pod with no daemon at all, which is the most defensible option. Default to Kubernetes mode or a rootless builder, and isolate any Docker-in-Docker runners on their own nodes.

Does moving GitHub Actions runners in-house lock me into ARC or Kubernetes?

Very little. ARC runs the standard open-source actions/runner agent inside ordinary pods, executing unchanged GitHub Actions YAML; there is no proprietary build format or pipeline DSL, and the operator is open source under GitHub's own repository. Removing ARC is deleting two Helm releases and changing one runs-on line back to a hosted label. The dependency you keep is GitHub Actions itself — the workflow syntax and orchestration — which is a deliberate, bounded trade. If you later want to leave GitHub entirely, owned runners make that easier, not harder: the hardware, network policy, registry, and build tooling are already yours, and only the orchestration remains to move to a self-hosted forge and CI engine.

github self-hosted runners kubernetesactions-runner-controller ARC setupephemeral self-hosted runners autoscalinggha-runner-scale-set helmGitHub Actions runner cost control self-hostsecure CI runners network isolation egress

Executive Briefing

Thirty minutes to clarify your infrastructure risk

Walk us through your vendor footprint and regulatory constraints. We will tell you honestly where sovereignty creates leverage — and where it does not. No pitch deck. No obligation.