SOC 2 Compliance on Self-Hosted Kubernetes: Own the Evidence

SOC 2 is the most common gate in enterprise procurement and one of the least understood pieces of machinery a platform team is handed. There is no regulator and nobody issues a certificate: it is a voluntary attestation performed by a CPA firm because a customer's security questionnaire demanded one.

Search the term and you get readiness checklists, published by compliance-automation vendors, all assuming a hyperscaler underneath and all sharing one premise: that a large provider makes SOC 2 easier because you inherit controls from its report. That misreads how subservice organizations work.

What a SOC 2 Report Actually Attests — And What It Does Not

SOC — System and Organization Controls — is described by the AICPA as "a suite of service offerings CPAs may provide in connection with system-level controls of a service organization or entity-level controls of other organizations." The deliverable is a practitioner's report, not a badge. Google Cloud states the standard: the report "is based on the Auditing Standards Board of the American Institute of Certified Public Accountants (AICPA) SSAE 18, which evaluates the service organization's controls relevant to the Trust Services Criteria of security, availability, processing integrity, confidentiality, or privacy."

The Type distinction is what changes engineering work. A Type 1 "covers the design of the service organization's controls at a specific point in time"; a Type 2 "covers the design and operating effectiveness of the service organization's controls over a period of time." That period is the unit of work, and enabling a control the week before fieldwork produces no evidence for the months behind it.

The criteria are the 2017 Trust Services Criteria, reissued with revised points of focus in 2022 without renumbering, so CC-series references stay stable. Security's Common Criteria run CC1 through CC9 — CC6 logical and physical access, CC7 system operations, CC8 change management, CC9 risk mitigation. The criterion text sits behind an AICPA account wall, so this article describes families by scope rather than quoting it.

The Inheritance Myth: Carve-Out, Inclusive, and What Stays Yours

When your system depends on another service organization, that provider is a subservice organization and your report must say how you treat it. Schellman, a CPA firm that performs these examinations, lists the candidates directly — "Data center hosting organizations; or Managed services organizations (e.g., network monitoring services), among others" — and there are two ways to treat them.

The carve-out method is the usual choice, and its name is literal: "You would not include any controls from your subservice organizations — i.e., you'd carve those controls out of the description of your system and controls testing." Carved out does not mean covered by someone else's report; it means absent from yours. Then the part that surprises people — "you would need to have a method of monitoring the subservice organization in place." You did not inherit a control. You acquired a vendor-monitoring obligation.

The inclusive method instead has the provider tested alongside you. In Schellman's stated experience most clients choose carve-out because "their subservice organizations have their own SOC report that customers can review," and the inclusive method is one organizations "rarely choose" — that firm's experience, not an industry statistic. Then come the complementary user entity controls, where your auditor asks whether "these CUECs [are] in place at your organization, and is your organization performing these CUECs effectively?" The provider's report hands you homework; discharging it is your control.

The logistics are rarely priced in. AWS SOC 1 and SOC 2 reports "are available to customers by using AWS Artifact," and "an NDA is required to review" them — only the summary SOC 3 is public, so the report you supposedly inherit cannot be forwarded to the customer who asked. Periods rarely align either: the AWS SOC 2 covers "12 months: ending 3/31, 9/30." That gap is closed by a bridge letter, which Google Cloud describes as an attestation "made by the management of the service provider." Management's word, not a CPA's opinion.

Click to zoom

What Self-Hosting Hands Back: Facility Controls and Control-Plane Operations

Here is the counterargument. AWS describes its side of the shared responsibility model as operating, managing and controlling "the components from the host operating system and virtualization layer down to the physical security of the facilities in which the service operates." That is a real control set, and self-hosting hands it back.

It lands hardest on CC6, which covers logical and physical access. On a hyperscaler you carve the physical half out and monitor the provider. Off it, two other cases. In colocation the colo provider is itself a data-centre hosting organization — a subservice organization in exactly the sense above — so you carve it out, read its report, and discharge its CUECs; the mechanics are re-pointed, not escaped. In your own building nobody is left to carve out: badge records, visitor logs and environmental monitoring become first-party evidence. The control plane returns too: node hardening, etcd encryption, patching cadence, certificate lifecycle.

CC6: Logical Access as a Cluster Control, Not a Policy PDF

CC6 is where most self-hosted engagements are won or lost, because logical access draws the most sampling. The questions are ordinary — who can reach production, how was it granted, how is it revoked — and so are the answers, if the cluster was built to give them: OIDC federation to a provider you operate, scoped RBAC, no long-lived kubeconfig certificates.

Click to zoom

Two RBAC constructs make CC6 unevidenceable, both called out in the Kubernetes project's RBAC good practices. The first is the system:masters group: "Any user who is a member of this group bypasses all RBAC rights checks and will always have unrestricted superuser access, which cannot be revoked by removing RoleBindings or ClusterRoleBindings." The same page adds that where a cluster uses an authorization webhook, membership bypasses that too — a control your external authorizer never sees is one you cannot evidence.

The second is wildcards: "providing wildcard access gives rights not just to all object types that currently exist in the cluster, but also to all object types which are created in the future." To an auditor sampling a point in time, that is a permission set which silently grows after the sample date. Kyverno turns both into standing monitors — start in Audit mode, which its docs describe as allowing the resource "but record it as a fail result in a policy report." Both rules exclude platform-shipped roles and bindings, so the report shows only what your team created.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: cc6-logical-access-guardrails
spec:
  background: true
  rules:
    - name: no-wildcard-clusterroles
      match:
        any:
          - resources:
              kinds:
                - ClusterRole
      exclude:
        any:
          - resources:
              names:                 # platform-shipped
                - cluster-admin
                - "system:*"
      validate:
        failureAction: Audit        # Enforce once coverage is measured
        message: >-
          Wildcard rules grant access to resource types that do not exist yet.
        deny:
          conditions:
            any:
              - key: "{{ request.object.rules[].resources[] || `[]` | [?@ == '*'] | length(@) }}"
                operator: GreaterThan
                value: 0
              - key: "{{ request.object.rules[].verbs[] || `[]` | [?@ == '*'] | length(@) }}"
                operator: GreaterThan
                value: 0
    - name: no-cluster-admin-bindings
      match:
        any:
          - resources:
              kinds:
                - ClusterRoleBinding
      exclude:
        any:
          - resources:
              names:
                - cluster-admin      # shipped by Kubernetes
                - "kubeadm:*"        # shipped by kubeadm
                - break-glass-*      # documented exception
      validate:
        failureAction: Audit
        message: >-
          Binding cluster-admin outside the break-glass path is a CC6 finding.
        pattern:
          roleRef:
            name: "!cluster-admin"

CC7: System Monitoring and the Audit Trail You Own

CC7 covers system operations, and the primary artifact is the API server audit log — described in the Kubernetes documentation in the terms an auditor uses: "a security-relevant, chronological set of records documenting the sequence of actions in a cluster," covering users, applications, and the control plane itself.

One sentence there is worth an entire readiness project: "If the flag is omitted, no events are logged." API auditing is off by default. If your period opened in January and --audit-policy-file was set in April, the first quarter has no evidence and no tooling will manufacture it.

The policy is a list of rules where "the first matching rule sets the audit level of the event," choosing between None, Metadata, Request and RequestResponse. That ordering makes a compliant trail affordable — full bodies for sampled writes, metadata for the read path, nothing for probes — and keeps secret payloads out of a log retained across the period.

# /etc/kubernetes/audit/policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
  - RequestReceived
rules:
  # Probe noise buries the sample.
  - level: None
    nonResourceURLs:
      - "/healthz*"
      - "/livez*"
      - "/readyz*"
      - "/metrics"

  # Secrets: record the access, never the payload.
  - level: Metadata
    resources:
      - group: ""
        resources: ["secrets", "configmaps"]

  # CC6: access changes, in full.
  - level: RequestResponse
    verbs: ["create", "update", "patch", "delete"]
    resources:
      - group: "rbac.authorization.k8s.io"
        resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]

  # CC8: workload mutations.
  - level: Request
    verbs: ["create", "update", "patch", "delete"]
    resources:
      - group: "apps"
      - group: ""
        resources: ["pods", "services", "serviceaccounts"]

  # Everything else: metadata, so the trail has no holes.
  - level: Metadata

Wiring it in is a static pod edit, and the flag reference's own retention defaults mislead: --audit-log-maxage at 366 days is already past a twelve-month period, while --audit-log-maxbackup (100) times --audit-log-maxsize (100 MB) caps the file backend near 10 GB regardless. Treat that backend as a spool; the webhook backend, which "sends audit events to a remote web API," carries retention off the node.

spec:
  containers:
    - name: kube-apiserver
      command:
        - kube-apiserver
        - --audit-policy-file=/etc/kubernetes/audit/policy.yaml
        - --audit-log-path=/var/log/kubernetes/audit/audit.log
        - --audit-log-maxage=400      # not the binding limit
        - --audit-log-maxbackup=500   # x maxsize = ~100 GB; give it its own volume
        - --audit-log-maxsize=200     # MB before rotation
        - --audit-webhook-config-file=/etc/kubernetes/audit/webhook.yaml
      volumeMounts:
        - name: audit-policy
          mountPath: /etc/kubernetes/audit
          readOnly: true
        - name: audit-log
          mountPath: /var/log/kubernetes/audit
  volumes:
    - name: audit-policy
      hostPath:
        path: /etc/kubernetes/audit
        type: DirectoryOrCreate
    - name: audit-log
      hostPath:
        path: /var/log/kubernetes/audit
        type: DirectoryOrCreate
# ... rest of the static pod spec unchanged

CC8: Change Management Is GitOps, If You Wire It That Way

CC8 asks whether changes are authorized, tested and approved before reaching production, and whether you can show it for a sampled change. Most answer with a ticketing system, then reconstruct whether the ticket described what shipped. GitOps collapses that gap: Argo CD "follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state," so the change record and the deployment mechanism are one object.

The control set falls out naturally. Authorization is the pull-request approval, made non-optional by branch protection; tested-before-approved is CI on the merge commit; and the operating-effectiveness half a Type 2 tests is Git's commit history plus sync events exported to the evidence store. Argo CD's in-cluster revisionHistoryLimit is a rollback buffer — default 10, and Argo's docs advise against raising it — not the archive. With automated sync, the deployed state is whichever commit was HEAD at sync time; the exported record says which.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: payments-platform
  namespace: argocd
spec:
  project: regulated
  source:
    repoURL: https://git.internal.example.com/platform/payments.git
    targetRevision: release      # protected branch; merges require review
    path: overlays/production
  destination:
    server: https://kubernetes.default.svc
    namespace: payments
  syncPolicy:
    automated:
      prune: true                # drift is corrected, not accumulated
      selfHeal: true             # out-of-band edits are reverted
  revisionHistoryLimit: 10       # rollback buffer, not the evidence archive

A1: Availability Evidence Is a Tested Restore, Not a Promise

If Availability is in scope, the A1 criteria reach recovery. A green backup dashboard evidences that a job exited zero, not that the data comes back; and because a Type 2 tests operating effectiveness across a period, you want a series of dated drill results.

The mechanics are covered elsewhere: proving RPO and RTO with Velero and etcd and chaos experiments as audit-grade evidence go deeper. What matters here is the output's shape. Velero's v1.16 documentation notes it "gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes" and that you "can run Velero with a cloud provider or on-premises." Write the drill to emit a file whether it passes or fails, and gate on data integrity, not pod readiness.

#!/usr/bin/env bash
# Quarterly A1 restore drill. One dated result file per run, pass or fail.
set -euo pipefail

BACKUP="${1:?usage: restore-drill.sh <velero-backup-name>}"
DRILL_ID="a1-restore-$(date -u +%Y%m%d)"
NS="drill-${DRILL_ID}"
OUT="evidence/a1/${DRILL_ID}.json"
mkdir -p "$(dirname "$OUT")"
START=$(date -u +%s)
VERDICT="fail"          # an aborted drill is a failed drill

# Runs on every exit path, so a hung restore still files evidence.
record() {
  printf '{"drill":"%s","backup":"%s","rto_seconds":%d,"integrity":"%s"}\n' \
    "$DRILL_ID" "$BACKUP" "$(($(date -u +%s) - START))" "$VERDICT" | tee "$OUT"
  mc cp "$OUT" "worm/soc2-evidence/a1/" || true
  kubectl delete namespace "$NS" --wait=false || true
}
trap record EXIT

velero restore create "$DRILL_ID" \
  --from-backup "$BACKUP" \
  --namespace-mappings "payments:${NS}" \
  --wait

# Gate on integrity, not readiness — readiness only proves scheduling.
kubectl -n "$NS" wait --for=condition=Ready pod -l app=ledger --timeout=15m
./scripts/verify-ledger-integrity.sh --namespace "$NS"
VERDICT="pass"

Building the Evidence Pipeline: Collect, Retain, Prove

Four streams now exist — audit events, policy reports, sync history, drill results — sharing one requirement: they must survive intact until an auditor asks, months later. Retention and immutability turn a log into evidence, and both fail silently.

Retention first. If audit events land in a self-hosted observability stack built on Loki, two constraints fail quietly. Grafana's documentation is explicit that retention_enabled "should be set to true. Without this, the Compactor will only compact tables," and that delete_request_store "should be set to configure the store for delete requests." Then the one that catches people: "Retention is only available if the index period is 24h." A compactor block pasted onto any other schema silently does nothing.

schema_config:
  configs:
    - from: "2026-01-01"
      store: tsdb
      object_store: s3          # MinIO via the S3 API
      schema: v13
      index:
        prefix: index_
        period: 24h             # retention applies at 24h and only at 24h

storage_config:
  tsdb_shipper:
    active_index_directory: /loki/tsdb-index
    cache_location: /loki/tsdb-cache
  aws:
    s3: https://minio.internal.example.com
    bucketnames: soc2-evidence
    s3forcepathstyle: true

limits_config:
  retention_period: 9600h       # 400 days: period + fieldwork + tail

compactor:
  working_directory: /loki/compactor
  retention_enabled: true       # without this the compactor only compacts
  delete_request_store: s3      # must name the store actually in use
  retention_delete_delay: 2h

Immutability second. An auditor asking whether records could have been altered wants a structural answer. MinIO's object locking "enforces Write-Once Read-Many (WORM) immutability to protect versioned objects from deletion" — note versioned: bucket versioning is the prerequisite, and locking an unversioned bucket protects nothing.

Failure Modes That Sink a Self-Hosted SOC 2

None are exotic. Each looks fine until someone samples across the period.

• Audit logging never enabled. No --audit-policy-file, no events, no evidence for the months already elapsed. Verify on every control-plane node.
• Retention shorter than the observation window. The audit log's volume cap, a Loki compactor with retention_enabled unset, or an index period that is not 24h.
• Treating policy reports as history. Kyverno's documentation states that reports "always represent the current state of the cluster and do not record historical information." Excellent monitor, poor archive — export on a schedule if you want a series.
• Unmonitored subservice providers. Carve-out obliges you to monitor. A report pulled once at onboarding, with no record of anyone reading the next, is the finding.
• CUECs nobody was assigned. Every complementary user entity control needs a named owner and an artifact. Unassigned means unperformed.
• No facility evidence for racks you now own. The physical half of CC6 moved to you, or to a colo provider you must carve out and monitor.

Exit Ramps and the Long Game: Controls That Outlive the Audit

Auditors change. Compliance platforms get acquired. Hosting changes. The evidence layer should outlive all three, so build it from formats that belong to nobody — JSON audit events, Git history, S3-API storage, dated drill files — and every vendor swap becomes a copy operation.

The same logic applies to runtime dependencies. Flux's Gitless GitOps model, where "the Flux controllers are fully decoupled from Git, relying solely on container registries as the source of truth," removes a hosted forge from the production path while Git remains the interface engineers use — so the change record CC8 depends on is unaffected.

SOC 2 recurs, so the decisive question is not what the first readiness project costs but what the third one costs. Carve-out costs stay administrative: chasing a report under NDA, checking a bridge letter, maintaining a CUEC matrix nobody owns. Make evidence a by-product of running the system and the annual engagement becomes mostly an export — the same structural argument that applies to DORA and operational resilience.

Which is the real reason to own the substrate. Not that SOC 2 is easier self-hosted — the facility controls alone say otherwise — but that the evidence is first-party, inspectable, and yours. A carved-out control you can describe but never examine; a control you operate you can improve. Across a decade of observation periods that difference compounds into the only compliance worth having: where the report describes how the system already works.