After MinIO Goes Dark: Rook/Ceph, SeaweedFS or Garage for Sovereign S3

For most of the last decade, "self-hosted S3" had a one-word answer: MinIO. It was the default object store behind half the Kubernetes platforms ever built — the backup target for Velero, the WAL archive for Postgres, the chunk store for Loki and Thanos, the registry backend, the data lake. Then the default went dark. Through 2025 MinIO stripped the management console out of its community edition, stopped publishing Docker images, and in December 2025 moved the open-source server into maintenance mode. By February 2026 the project was marked *no longer maintained*, and on 25 April 2026 the `minio/minio` repository was archived read-only — no issues, no pull requests, no future. The thing under your backups stopped being a project and became an artifact.

This is not a post-mortem on MinIO. It is a decision framework for what comes next, because the wrong reaction to losing a default is to reach for the nearest drop-in replacement and reproduce the exact dependency you just got burned by. The object-storage tier is load-bearing infrastructure, and the teams that handle this well treat the replacement as an architecture decision, not a docker pull. The three open-source contenders worth serious evaluation in 2026 are Rook/Ceph's RADOS Gateway, SeaweedFS, and Garage, and they are not interchangeable. They encode three different bets about tenancy, object profile, jurisdiction, and how much operational weight you are willing to carry.

What follows is the honest comparison: what each system actually is, the axes that should drive the choice, config you can read, and the part most guides skip — how to leave whatever you pick. That last property is the whole point. The goal was never to run MinIO, or Ceph, or anything else. The goal is to own the storage your business depends on, and to keep the exit ramp paved.

What Actually Happened to MinIO — and the Sovereignty Lesson In It

The mechanics are worth getting right, because the lesson is in the sequence, not the outcome. MinIO did not get acquired or shut down. It made a series of rational commercial decisions that each narrowed the open-source surface: in mid-2025 the full admin console — user management, bucket policies, lifecycle, monitoring — was removed from the community build and reserved for the commercial AIStor tier, leaving the open binary with little more than an object browser. Docker Hub images stopped. Then the server itself entered maintenance mode, then unmaintained, then archived. None of this was a betrayal. It was a single vendor doing what single vendors eventually do with a project they fully control.

MinIO is licensed AGPLv3, and that license is irrevocable — the code that exists can be forked forever. So why is "just fork it" not the answer? Because a fork is a project, and a project needs maintainers to track CVEs and ship releases on a cadence regulated buyers can depend on. The community attempt to fork the console (OpenMaxIO) stalled almost immediately. Open *source* and open *project* are different guarantees, and only the second one protects you over a multi-year horizon. This is the same distinction that runs through all of our work: open source is a method, not a checkbox — what matters is governance, contributor diversity, and a license that keeps the exit ramp open, not the mere presence of a public repo.

Why the Object-Storage Tier Is Load-Bearing

It is tempting to treat object storage as a commodity. That framing is exactly why MinIO's exit caught so many teams flat-footed. The S3 endpoint is the substrate a sovereign platform quietly rests on. Your Velero disaster-recovery backups land in it. Your CloudNativePG WAL archive and point-in-time recovery stream to it continuously — lose the object store and your proven recovery point objective becomes a hope. Your self-hosted observability stack writes Loki log chunks, Tempo traces, and Thanos metric blocks to it. Container images, ML model artifacts, and the data lake all terminate there too.

Which means the object store is not a peer of those systems — it is underneath all of them. That is why this choice deserves the same rigor as the database tier. The question is not which is fastest, but which system's failure modes, operational burden, and jurisdiction controls match the platform you actually run. Speed matters, but it is the fourth question, not the first.

The Three Contenders, Honestly Described

Rook/Ceph RADOS Gateway (RGW). Ceph is a distributed storage system that provides block, file, and object from one cluster; Rook is the CNCF-graduated operator that runs it on Kubernetes. Object storage is exposed through the RADOS Gateway, a stateless S3/Swift front end you scale horizontally behind a load balancer. RGW has the most complete S3 API of the three — multipart, versioning, lifecycle, object lock, server-side encryption, bucket policies, STS-style tenancy — and Ceph is the only option here with true erasure coding and battle-tested active-active multi-site replication. The price is operational weight: CRUSH maps, placement groups, OSD lifecycle, a multi-daemon topology, a real RAM floor per OSD, and a sane minimum of roughly five storage nodes. Ceph is what you run when you need a multi-tenant object platform at scale and you have, or will build, Ceph expertise.

SeaweedFS. Apache-2.0 licensed and over a decade old, SeaweedFS began as a small-file distributed filesystem inspired by Facebook's Haystack paper, with an S3 gateway added on top. Its architecture is three tiers: a master tracking volume topology, volume servers that pack many small files into large volume files (giving O(1) disk seeks regardless of file count), and a filer that provides directory semantics and hosts the S3 API. Its superpower is billions of small objects — model artifacts, thumbnails, log fragments, CI caches — where it outclasses everything else, and its operations are far lighter than Ceph. The caveats are honest ones: S3 is a translation layer over the filer rather than a native design, and there are documented edge cases in multipart-plus-compression and high-concurrency filer consistency. It is replication-only; there is no erasure coding.

Garage. Written in Rust by the French non-profit Deuxfleurs and licensed AGPLv3, Garage is the youngest and most opinionated. It is a single statically linked binary with no external database — no etcd, no Postgres, no ZooKeeper — and it coordinates through CRDTs rather than a Raft/Paxos consensus on the data path, which is precisely what lets it stay responsive across high-latency links between distant nodes. It was designed from day one for heterogeneous, geo-distributed deployments: nodes in different cities, on different hardware, over the open internet. Its v2.0 release in mid-2025 added scoped admin tokens, key expiration, and website-redirect support. It deliberately forgoes erasure coding in favor of simple replication, and its S3 surface is narrower than RGW's. Garage is what you reach for when jurisdiction and low operational burden matter more than raw single-datacenter throughput.

A Decision Framework, Not a Default

Replacing one default with another is how you end up here again in three years. The discipline is to choose against the axes that actually distinguish these systems. Six matter, roughly in order of how decisively they should move you.

• Jurisdiction and data residency. Do you need to guarantee that every object physically resides in specific countries? This is Garage's home turf — zone-aware replication places copies by geography as a first-class control. Ceph multi-site can do active-active across zones but at real operational cost. For most teams this axis, when it binds, binds hardest.
• Object profile: many small vs. fewer large. Billions of small objects favor SeaweedFS decisively. Large objects and aggregate throughput at scale favor Ceph. Garage handles small objects competently but is not optimized for peak local throughput.
• Tenancy. Rich multi-tenancy — per-user identities, bucket policies, quotas, isolation between teams — is Ceph RGW's strength. SeaweedFS and Garage offer access keys and bucket-level permissions suited to a trusted operator team, not a self-service multi-tenant platform.
• Operational burden. Garage is the lightest (one binary, no dependencies); SeaweedFS is moderate (three component types, well documented); Ceph is heaviest and demands genuine expertise. Be honest about the team you have, not the one you wish you had.
• Storage efficiency: erasure coding vs. replication. At hundreds of terabytes, Ceph's erasure coding (e.g. 4+2 ≈ 1.5× overhead) is dramatically cheaper than the 3× of replication. Below that, replication's simplicity usually wins. Only Ceph offers erasure coding here.
• S3 API completeness. If you depend on advanced features — object lock for WORM compliance, lifecycle transitions, STS — verify them explicitly. RGW is closest to AWS parity; SeaweedFS and Garage cover the core well but have edges. Test your actual workload's calls, not the marketing matrix.

Click to zoom

Architecture in Depth, with Config You Can Read

The shape of each system's configuration reveals its philosophy more honestly than any feature table. Start with Ceph RGW under Rook. You declare a CephObjectStore and the operator provisions the gateway and the underlying RADOS pools. Note the deliberate split: the metadata pool is replicated (you never erasure-code metadata), while the bulk data pool is erasure-coded for efficiency.

apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: sovereign-store
  namespace: rook-ceph
spec:
  metadataPool:
    failureDomain: host
    replicated:
      size: 3                 # metadata: 3x replication — never erasure-coded
  dataPool:
    failureDomain: host
    erasureCoded:
      dataChunks: 4           # 4+2 erasure coding ≈ 1.5x overhead vs 3x replication
      codingChunks: 2
  preservePoolsOnDelete: true
  gateway:
    port: 80
    securePort: 443
    instances: 3              # RGW is stateless — scale horizontally behind a LB
    resources:
      requests: { cpu: "1", memory: "4Gi" }
  # A second zone here turns this into active-active multi-site replication.

SeaweedFS shows its filesystem heritage. There is no single binary that is "the object store"; you compose a master, volume servers, and a filer, and the S3 gateway is a thin process hung off the filer. The replication string (001, 010, 110) is SeaweedFS's compact encoding of how many copies to keep across servers, racks, and data centers.

# Master — tracks volume topology and the file → volume mapping.
weed master -mdir=/data/master -defaultReplication=010

# Volume servers — pack many small files into large volumes (O(1) seek).
weed volume -mserver=master:9333 -dir=/data/vol -max=200

# Filer holds directory semantics; the S3 gateway is layered on top of it.
weed filer -master=master:9333
weed s3 -filer=filer:8888 -config=/etc/seaweedfs/s3.json
# defaultReplication=010 → keep one extra copy on a different rack, same DC.

Garage is the inverse of Ceph's ceremony. Every node runs the same binary and config; there are no special roles. You set a replication factor, then assign each node to a zone — and the layout engine guarantees one replica per zone. Make each zone a jurisdiction and you have turned data residency into two lines of configuration.

# garage.toml (excerpt) — symmetric on every node, no external database.
replication_factor = 3
consistency_mode = "consistent"

# Then assign each node a zone = a jurisdiction. With replication_factor 3
# and three zones, every object gets exactly one replica in each:
#   garage layout assign <node-id> -z eu-central-de -c 2T
#   garage layout assign <node-id> -z eu-west-fr    -c 2T
#   garage layout assign <node-id> -z eu-north-se   -c 2T
#   garage layout apply --version 1
# Every object now physically resides in DE, FR, and SE. Nowhere else.

Click to zoom

Jurisdiction, GDPR, and the Sovereignty Boundary

Object storage is where data sovereignty stops being abstract. A bucket is a physical thing on physical disks in a physical country, and regulators increasingly care which one. The EU Data Act, applicable from September 2025, moves the market toward portability and bans most cloud egress fees within its phase-in window; the CLOUD Act means a US-headquartered provider can be compelled to produce data regardless of which region it sits in. "Our data is in Frankfurt" and "we control who can compel access to our data" remain different statements — and only the second is sovereignty. This is the same gap that NIS2, DORA, and the EU AI Act have made legally material for regulated workloads.

Self-hosting the object tier closes that gap, and the system you pick determines how precisely. Garage lets you assert residency as configuration: a replica in Germany, one in France, one in Sweden, and provably nowhere else. Ceph multi-site can pin zones to locations but expects you to operate the clusters that back them. SeaweedFS supports cross-data-center replication but is not built for fine-grained geographic placement. For an auditor, the difference is concrete: residency stops being a provider's contractual promise and becomes a property of your own layout — the same audit-grade posture we hold for backups and the node OS underneath, where an immutable platform like Talos Linux makes the storage hosts auditable down to the boot.

Migrating Off MinIO Without a Big Bang — and the Exit Ramp for Next Time

Here is the quiet good news inside the bad: the reason you can leave MinIO at all is that it speaks S3, and so does everything you might move to. The S3 API is the portability contract, and it is what turns a vendor's exit from a catastrophe into a migration. The mechanical move is unglamorous and safe — mirror, verify, cut over — and tools like rclone and mc make it a background job rather than a maintenance-window gamble.

# The S3 API is the portability contract. Mirror old → new, verify, then cut over.
rclone sync  minio:backups newstore:backups --checksum --transfers 32
rclone check minio:backups newstore:backups --checksum   # byte-for-byte audit

# Dual-write from the application during the bake period, then flip the
# endpoint in one config change once 'rclone check' is clean and stable.

Design the new deployment so the next exit is just as cheap. Keep your applications pointed at a stable internal endpoint, not a vendor-specific hostname. Use only the S3 features your workload actually needs, so portability is not quietly traded away for a proprietary extension. Keep credentials in your secrets layer rather than baked into manifests. None of this is exotic — it is the discipline of designing the exit ramp before you need it, applied to storage.

Choosing Well: A Recommendation Per Profile

The framework resolves, for most teams, into a few clear answers. If your binding constraint is jurisdiction — prove where every byte lives, across sites, with a small team — choose Garage, and accept replication-only storage as the price of that control. If you store enormous numbers of small objects and want lighter operations than Ceph, choose SeaweedFS, and budget testing for the S3 edge cases you touch. If you are building a genuinely multi-tenant object service, need erasure-coded efficiency at scale, or already run Ceph for block and file, choose Rook/Ceph RGW, and invest in the expertise it demands rather than pretending it runs itself.

And if the honest answer is that no one on your team will own any of these yet, the disciplined move is to stay on a managed S3 for now and build the capability deliberately — the same operating-model shift that determines whether any cloud repatriation succeeds. Sovereignty is a goal pursued in proportion to readiness, not a mandate to self-host before you can operate it. Choosing to wait, with a plan to build, is itself a sovereign decision; drifting onto the next single-vendor default because it was the path of least resistance is not.

That is the sovereign object tier in one idea: not the fastest store or the most fashionable one, but the one whose tenancy, jurisdiction, and operational shape match the platform you actually run — and that you can leave on your own terms when the landscape shifts again. It will shift again. MinIO going dark was a reminder that the things we call defaults are someone else's business decision waiting to happen. Own the tier, keep to the standard that makes it portable, and the next reminder costs you a weekend instead of a quarter.