Skip to content
Stribog

Security

All writing

Sealed Secrets: Encrypted Secrets in Git, and the Key You Now Own

Sealed Secrets keeps ciphertext in Git and keys in-cluster. Renewal is not rotation; a pre-renewal backup cannot open secrets sealed with the new key.

Stribog15 min read

Most comparisons treat Sealed Secrets as a row in a feature table next to SOPS and External Secrets Operator. That misses where the operating risk sits. The interesting questions are about the key: what renewal actually does, what a backup can and cannot decrypt, and what happens on the day the cluster is gone. This article works through that lifecycle as of 24 October 2026, against the latest GitHub release, v0.41.1, and the controller the Helm chart currently installs, then draws the line between ciphertext-in-Git and the reference model ESO uses.

The problem GitOps leaves you with

GitOps works because the repository is the whole truth: every manifest reviewed, versioned and reconciled. Secrets break that. A Kubernetes Secret manifest is base64, which is encoding, not encryption, so committing one publishes the value to everyone who can clone the repository, forever. The cluster side is not much better by default. The Kubernetes documentation states that Secrets are stored unencrypted in etcd unless you configure otherwise, and that anyone authorised to create a Pod in a namespace can use that access to read any Secret in it.

So the secret has to reach the cluster by some route other than plaintext in Git. Argo CD's secret management guidance frames the choice: populate secrets on the destination cluster, or inject them during manifest generation. It strongly recommends the former, naming both Sealed Secrets and External Secrets Operator among the tools. The reason is concrete. Argo CD caches plugin-generated manifests, injected secrets included, in its Redis instance, and serves them through the repo-server API, so anyone with access to either can read them.

That leaves two families of answer that keep the decryption step inside the destination cluster. Either Git holds ciphertext and an in-cluster component decrypts it, or Git holds a reference and an in-cluster component fetches the value from a store you run. Sealed Secrets is the first. It is worth understanding exactly, because the price of ciphertext-in-Git is a key that has to outlive every cluster you will ever run.

How Sealed Secrets works: asymmetric, one key set, one controller

The project README describes two parts: a cluster-side controller and a client utility, kubeseal. The client uses asymmetric cryptography, so anyone holding the public certificate can encrypt and only the controller can decrypt. You turn a Secret into a SealedSecret custom resource, commit it, let Argo CD or Flux apply it, and the controller writes the ordinary Secret into the target namespace. Workloads never know the difference.

The crypto design note is short and worth reading. Each secret is encrypted with AES-256-GCM under a randomly generated, single-use 32-byte session key; that session key is encrypted with RSA-OAEP using SHA-256. The controller's certificates are renewed every 30 days and carry a 10-year validity span. In the controller source, the default RSA key size is 4096 bits and --key-renew-period defaults to 30 days, with 0 disabling automatic renewal.

The private keys are ordinary Kubernetes Secrets in the controller's namespace, labelled sealedsecrets.bitnami.com/sealed-secrets-key, and the controller loads the ones whose label value is active. On startup it generates a key in three cases: no active key was loaded; the newest key is older than --key-cutoff-time; or, with renewal enabled, the newest key is already at least one renewal period old, in which case the renewal job's initial delay clamps to zero and it mints a key immediately. That third case matters during restores, as the backup section shows.

Git holds only ciphertext. Everything that can decrypt it is the key set in kube-system — and whatever backup of it you keep.

Installation is a Helm chart. As of this writing, chart 2.20.0 in the v0.41.1 tree installs controller image 0.40.0, not 0.41.1, so pin the chart and describe what you run accurately. Setting fullnameOverride keeps kubeseal's defaults — controller name sealed-secrets-controller in kube-system — working without extra flags:

bash
helm repo add sealed-secrets https://bitnami.github.io/sealed-secrets
helm repo update
helm install sealed-secrets sealed-secrets/sealed-secrets \
  --namespace kube-system \
  --version 2.20.0 \
  --set-string fullnameOverride=sealed-secrets-controller
kubectl -n kube-system rollout status deployment/sealed-secrets-controller

# Public certificate for offline sealing. Safe to share; it can only encrypt.
kubeseal --fetch-cert > pub-cert.pem
Chart 2.20.0 runs controller 0.40.0. The kubeseal client binaries ship with the v0.41.1 GitHub release.

The controller images are signed with cosign, with signatures published in GitHub Container Registry; images up to and including v0.20.2 used cosign v1 and newer ones cosign v2. That location matters to anyone running a signature-verifying admission policy: the chart pulls docker.io/bitnami/sealed-secrets-controller:0.40.0, and its signatures are not stored beside that tag. Verifying either the ghcr.io or the docker.io image requires COSIGN_REPOSITORY=ghcr.io/bitnami/sealed-secrets-controller/signs, as the project README's cosign example sets it, so a policy that only looks for signatures next to the image will refuse the controller until that repository is configured. The chart's defaults also bind its service-proxier Role to system:authenticated; that Role grants access to the controller Service and its proxy (certificate, verify and re-encrypt endpoints), not to Secrets, but narrow the subjects through the rbac.serviceProxier values if every authenticated identity is broader than you intend.

Scopes: what the ciphertext is bound to

A sealed secret is not just encrypted; it is bound to a destination. The README defines three scopes. strict is the default: the secret must be sealed with exactly the name and namespace it will have, because those attributes become part of the encrypted data, and changing either produces a decryption error. namespace-wide lets you rename the sealed secret freely within one namespace. cluster-wide lets it be unsealed in any namespace under any name.

The scope can be chosen with kubeseal --scope or with annotations: sealedsecrets.bitnami.com/namespace-wide: "true" or sealedsecrets.bitnami.com/cluster-wide: "true". No annotation means strict, and if both are set, cluster-wide wins.

Pick deliberately, because each scope is a security statement. Strict is the right default for most application secrets: a SealedSecret copied into another team's namespace simply fails to decrypt, so the binding acts as an authorisation boundary. The cost is that renames and namespace moves are re-seal operations, not edits — a refactor that moves a service between namespaces touches every secret it owns. Namespace-wide suits charts that template resource names. Cluster-wide is the one to question in review: because the ciphertext is bound to no name or namespace, anyone holding a copy of the manifest who can create a SealedSecret anywhere can materialise that value in a namespace they control, which collapses the boundary strict gave you.

Sealing in practice: kubeseal, offline certs and CI

Nobody who seals needs cluster credentials. The README's offline pattern is to store the certificate with kubeseal --fetch-cert >mycert.pem and seal against it with --cert; kubeseal also accepts a URL, so a platform team can publish the current certificate on an internal endpoint and point developers and CI at it. Generate the Secret client-side and pipe it straight through:

bash
# Strict scope (default): bound to name db-credentials in namespace payments.
kubectl create secret generic db-credentials \
  --namespace payments \
  --from-file=password=./db-password.txt \
  --dry-run=client -o yaml \
  | kubeseal --cert pub-cert.pem --format yaml > db-credentials-sealed.yaml

# Namespace-wide: may be renamed inside payments, nowhere else.
kubectl create secret generic db-credentials \
  --namespace payments \
  --from-file=password=./db-password.txt \
  --dry-run=client -o yaml \
  | kubeseal --cert pub-cert.pem --scope namespace-wide --format yaml > db-credentials-sealed.yaml

shred -u ./db-password.txt
Reading the value from a file keeps it out of shell history; --dry-run=client means the plaintext Secret never reaches the API server.

The output is a manifest you can review and commit. The ciphertext below is a placeholder — real output is long and specific to the key that sealed it — but the shape is what lands in Git:

yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: payments
  annotations:
    sealedsecrets.bitnami.com/namespace-wide: "true"
spec:
  encryptedData:
    password: PLACEHOLDER-CIPHERTEXT-REPLACE-WITH-KUBESEAL-OUTPUT
  template:
    metadata:
      name: db-credentials
      namespace: payments
      annotations:
        sealedsecrets.bitnami.com/namespace-wide: "true"
    type: Opaque
A namespace-wide SealedSecret. The encryptedData value is a placeholder; kubeseal emits a long base64 string in its place.

Refresh the offline certificate on a schedule. The README notes that certificates renew every 30 days and recommends teams update their offline copy periodically. A stale certificate still seals correctly while its private key stays active, so nothing fails loudly; the cost is that new secrets keep landing on an older key, which matters the moment that key is suspect. Fetching from a URL in CI removes the drift.

Renewal is not rotation

This is the misunderstanding with the largest consequences. Every 30 days the controller creates a new sealing key and appends it to the set of active keys; the newest key seals new secrets and is the one --fetch-cert returns. Old keys are not deleted, so old SealedSecrets still decrypt. The README concedes that the feature having been historically called "key rotation" adds to the confusion, and it is explicit that key renewal and re-encryption are not a substitute for periodically rotating the actual secret values.

Follow that through. A value sealed two years ago is still decryptable by the controller today, and by anyone who obtains the key set. The README's warning is the one to internalise: anything stored in version control must be assumed impossible to delete, and if a sealing key leaks, every SealedSecret encrypted with it is compromised — no amount of key renewal or re-encryption changes that. kubeseal --re-encrypt produces a file freshly encrypted with the latest key without the plaintext leaving the cluster, but it does not update the in-cluster object and it cannot reach the old ciphertext sitting in Git history.

So the response to a suspected key leak is value rotation, done in an order that never leaves the controller without a usable key. The README says to renew early before sealing any rotated value, via --key-cutoff-time or SEALED_SECRETS_KEY_CUTOFF_TIME, in RFC1123 format. The controller parses it as RFC1123 with a numeric timezone, so capture the timestamp once and reuse it:

bash
#!/usr/bin/env bash
set -euo pipefail
NS=kube-system
DEPLOY=sealed-secrets-controller

# 1. Renew early while the suspect key is still active. Capture the cutoff ONCE.
#    GNU date -R prints a numeric zone (+0000). On macOS use:
#    date -u '+%a, %d %b %Y %H:%M:%S +0000'
CUTOFF="$(date -R)"
echo "key cutoff: ${CUTOFF}"          # record in the incident log
kubectl -n "${NS}" set env "deployment/${DEPLOY}" SEALED_SECRETS_KEY_CUTOFF_TIME="${CUTOFF}"
kubectl -n "${NS}" rollout status "deployment/${DEPLOY}"

# 2. Rotate the real values at their source (database, API provider). Not shown.

# 3. Re-fetch the certificate FIRST, then re-seal every rotated value.
#    A stored pub-cert.pem still encrypts to the suspect key.
kubeseal --fetch-cert > pub-cert.pem
kubectl create secret generic db-credentials --namespace payments \
  --from-file=password=./db-password-new.txt --dry-run=client -o yaml \
  | kubeseal --cert pub-cert.pem --format yaml > db-credentials-sealed.yaml
# Commit, let Argo CD or Flux apply, confirm workloads use the new values.

# 4. Retire the suspect key. It stays in kube-system for manual decryption.
kubectl -n "${NS}" get secret -l sealedsecrets.bitnami.com/sealed-secrets-key \
  -L sealedsecrets.bitnami.com/sealed-secrets-key
SUSPECT_KEY=sealed-secrets-keyabcde   # the compromised key's Secret name
kubectl -n "${NS}" label secret "${SUSPECT_KEY}" \
  sealedsecrets.bitnami.com/sealed-secrets-key=compromised --overwrite

# 5. Restart: relabels are not picked up until the controller restarts.
kubectl -n "${NS}" delete pod -l app.kubernetes.io/name=sealed-secrets
Order matters: never label the only active key compromised before step 1 has produced a new one.

Three details are easy to get wrong. The restart in step 5 is required because the controller does not pick up manually created, deleted or relabelled keys until it restarts, and --watch-for-secrets, the beta flag that would change that, defaults to false. Leaving the cutoff variable set across that restart is harmless: the key minted in step 1 is newer than the cutoff, so no further key is generated. Relabelling in step 4 changes nothing on its own; only after the step 5 restart does the controller load just the keys still labelled active and stop unsealing anything sealed with the suspect key, so step 3 has to re-seal every such secret before that restart. If GitOps manages the Deployment, make the cutoff change in the chart values it applies, or it will be reverted.

The controller key is your new single point of failure

The FAQ is blunt. The private keys live only in the Secret the controller manages, unless you back them up; there are no backdoors. Without the key that sealed a given SealedSecret, it cannot be decrypted, and if you can reach neither the key Secrets nor the live decrypted Secrets, every password has to be regenerated and re-sealed. Each SealedSecret is recoverable only with the key that sealed it. Losing the whole key set, with no backup and no live Secrets, is what turns the repository into noise.

The backup is a label-selector export. Keep the selector as the bare label key, not =active: keys relabelled compromised are still needed to decrypt history. The README's note decides the cadence: after every renewal, re-take the backup, because a backup taken before a renewal cannot decrypt secrets sealed with the key that renewal created. Secrets sealed with keys the backup already holds — including ones sealed after the backup, before the next renewal — still decrypt. Running weekly keeps the exposure window well inside the 30-day period:

bash
#!/usr/bin/env bash
# backup-sealing-keys.sh: run weekly from a timer, and after any manual key change.
set -euo pipefail
: "${AGE_RECIPIENT:?set AGE_RECIPIENT to the offline age public key}"
BACKUP_DIR="${BACKUP_DIR:-/srv/backup/sealed-secrets}"
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
OUT="${BACKUP_DIR}/sealed-secrets-keys-${STAMP}.json.age"

# Every key Secret, active AND compromised. Server-assigned fields are stripped
# so the List applies cleanly to a cluster that has never seen these objects.
kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o json \
  | jq 'del(.metadata.resourceVersion,
            .items[].metadata.resourceVersion,
            .items[].metadata.uid,
            .items[].metadata.creationTimestamp,
            .items[].metadata.managedFields)' \
  | age --encrypt --recipient "${AGE_RECIPIENT}" --output "${OUT}"

# Inventory for the custody log: names, ages, labels. Never key material.
kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key \
  -L sealedsecrets.bitnami.com/sealed-secrets-key
echo "wrote ${OUT}"
The age identity that can decrypt this file belongs offline, with a different custodian from cluster admin.

A backup you have not restored is a hypothesis. The README's restore is to put the key Secrets back before the controller starts — or, if it already started, replace the secrets it created and restart it. Prefer the first path. On the second, the controller has already minted a fresh key that is newest, active and absent from your backup: find it with the listing command above, delete it with kubectl -n kube-system delete secret <name> only if nothing has been sealed against it, apply the backup, then restart the pod. The drill, which disaster recovery testing should include, looks like this:

bash
#!/usr/bin/env bash
set -euo pipefail
# Restore drill into a fresh cluster: keys first, controller second.
age --decrypt --identity ./offline-identity.txt \
  --output main.key ./sealed-secrets-keys-20261024T060000Z.json.age
kubectl apply -f main.key

helm repo add sealed-secrets https://bitnami.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets --namespace kube-system \
  --version 2.20.0 --set-string fullnameOverride=sealed-secrets-controller
kubectl -n kube-system rollout status deployment/sealed-secrets-controller

# Proof: a SealedSecret straight from Git unseals.
kubectl create namespace payments
kubectl apply -f db-credentials-sealed.yaml
until kubectl -n payments get secret db-credentials >/dev/null 2>&1; do sleep 2; done

# If the newest restored key was already a full renewal period old, the
# controller just minted a catch-up key that is NOT in main.key. Re-export now.
kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key \
  -L sealedsecrets.bitnami.com/sealed-secrets-key
./backup-sealing-keys.sh

# Offline path, no cluster at all. Output is plaintext: send it somewhere controlled.
kubeseal --recovery-unseal --recovery-private-key main.key \
  < db-credentials-sealed.yaml > /dev/null && echo "offline unseal OK"
shred -u main.key
Record the date, the backup file used, and both results. That record is audit evidence.

The offline path is the one teams forget exists. With backed-up private keys, kubeseal --recovery-unseal --recovery-private-key decrypts a sealed file without any running controller, which is what you need when the cluster is gone and standing up a replacement is not the fastest route to a database password. Store the encrypted backups the way immutable backups are stored: off-cluster, versioned, and out of reach of the credentials that could delete the cluster.

Sealed Secrets or External Secrets Operator: where does the truth live?

The real decision is not between tools but between two places the source of truth can live. With Sealed Secrets, Git holds ciphertext and the truth is Git plus the controller key. With External Secrets Operator, Git holds a reference. ESO's overview describes the flow: it instantiates a client for an external API from a SecretStore, fetches what an ExternalSecret asks for, creates a Kubernetes Secret from the template, and keeps the values in sync with the external API.

yaml
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: payments
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: openbao
  target:
    name: db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: payments/db
        property: password
Under the reference model, this is all Git holds: where the value lives, not the value.
Ciphertext in Git versus a reference in Git. Each model concentrates risk in one place; the question is which place you can run best.

Compare on the axes an operator lives with. What breaks: under Sealed Secrets, losing the key set is the catastrophe; under ESO, the store being unreachable stops new and refreshed Secrets. Who is on call: Sealed Secrets adds one controller and a backup job; ESO puts a highly available secret store, such as OpenBao or Vault, on your critical path. Rotation: with Sealed Secrets a new value is a re-seal and a commit, so Git history records that a change happened; with ESO the value changes in the store and propagates according to refreshInterval — zero creates the Secret once and never updates it, a positive value updates on that interval or when the ExternalSecret changes. What an auditor sees: a Git log and a key custody record, or a store's access policy and audit trail.

Choose Sealed Secrets when you have no secret store and do not want to run one, when clusters are few, and when values change rarely. Choose ESO when a store already exists, when values rotate often or are issued dynamically, or when many clusters need the same value — the multi-cluster GitOps case, where per-cluster keys mean sealing every secret once per cluster. Both are current open-source projects: ESO's latest release as of this writing is v2.14.0, published 23 October 2026, the same day as Sealed Secrets v0.41.1.

SOPS is the other ciphertext-in-Git option. It encrypts with AWS KMS, GCP KMS, Azure Key Vault, HuaweiCloud KMS, age or PGP, and was donated to the CNCF as a Sandbox project in 2023. Flux decrypts it in-cluster through kustomize-controller with --decryption-provider=sops and a decryption secret, and Flux's guide notes the encrypted files are not meant to be applied with kubectl. The trade-off differs: you choose the key and where it lives, rather than inheriting a controller-generated one.

What an auditor is shown

Ciphertext in a repository proves nothing on its own. The evidence that the control works is about the key, and every item can be produced on demand:

  • Key access. The RBAC subjects that can read Secrets in kube-system, from a review of every RoleBinding and ClusterRoleBinding that grants get or list on secrets in that namespace, or from a tool that enumerates subjects rather than verbs. kubectl auth can-i --list is not that inventory: it prints the caller's own allowed actions, and with --as it checks one impersonated identity at a time. Anyone on that list can read every sealed value, past and present.
  • Backup custody. Where the encrypted key exports live, who holds the age identity, and the inventory output from each run showing every key name was captured after each renewal.
  • Restore drills. Dated records of the fresh-cluster restore and the offline --recovery-unseal check, with the backup file each one used.
  • Value rotation, separately logged. A record of real secret rotations, kept apart from key renewal, so nobody mistakes the controller's 30-day key creation for evidence that credentials changed.
  • Compromise runbook. The ordered procedure above, with the cutoff timestamp recorded each time it is used.

That last distinction is where audits go wrong. A finding that says "secrets rotate every 30 days" because the controller renews keys is a false statement. The project's own documentation says otherwise.

Exit ramps and the long game

Leaving is straightforward because the controller writes ordinary Kubernetes Secrets. To migrate, read the live Secrets, or recover values offline with --recovery-unseal, write them into an external store or re-encrypt them with SOPS, switch the manifests, and remove the SealedSecrets. Nothing about the workloads changes. The format is open, the client is open source, and the keys are yours. The one irreversible fact is history: every SealedSecret ever committed stays decryptable by any key that ever sealed it, so retire leaving values by rotating them, not by deleting files.

That is the long-game view. A repository can outlive many clusters, and its ciphertext is only as durable as the key set and its backups. Adopt Sealed Secrets for the simplicity it gives GitOps, and treat the key set as infrastructure that must last a decade: backed up weekly, restored on a schedule, held by more than one custodian, and never confused with rotation.

§FAQ/Common questions

Frequently asked

Is it safe to commit Sealed Secrets to a public Git repository?

The ciphertext can only be decrypted with the controller's private keys, so it is designed to be shareable. But the project warns that anything in version control must be assumed undeletable, and if a sealing key ever leaks, every SealedSecret encrypted with it is compromised. Treat public commits as a bet on the key set staying private forever.

Does Sealed Secrets key renewal rotate my secrets?

No. Every 30 days by default the controller appends a new sealing key and seals new secrets with it, but old keys are kept and old SealedSecrets still decrypt. The README states renewal and re-encryption are not a substitute for rotating the actual secret values.

How do I back up and restore the Sealed Secrets key?

Export every Secret labelled sealedsecrets.bitnami.com/sealed-secrets-key from the controller namespace, encrypt the file and store it off-cluster. Re-take it after each renewal. To restore, apply the key Secrets before the controller starts, or replace the keys it generated and restart it.

Can I decrypt a SealedSecret without a running cluster?

Yes, if you backed up the private keys. kubeseal --recovery-unseal --recovery-private-key with the key file decrypts a sealed secret file offline. Without the key that sealed it, there is no backdoor.

Should I use Sealed Secrets or External Secrets Operator?

Sealed Secrets keeps ciphertext in Git and needs only a controller and a key backup. External Secrets Operator keeps a reference in Git and syncs values from an external store you must run. Pick Sealed Secrets without a store and with few clusters; pick ESO when a store exists or values rotate often.

sealed secretskubesealsealed secrets key backup restoresealed secrets key renewal vs rotationsealed secrets vs external secrets operatorsealed secrets scope strict namespace-wide cluster-wide

Executive Briefing

Thirty minutes to clarify your infrastructure risk

Walk us through your vendor footprint and regulatory constraints. We will tell you honestly where sovereignty creates leverage — and where it does not. No pitch deck. No obligation.