Cloud Repatriation: The Kubernetes Engineering Playbook

The cloud repatriation narrative of 2026 has the quality of a gold rush. Barclays puts 83% of enterprises planning some degree of exit from major hyperscalers — the highest figure ever recorded. The 37signals case is the canonical reference: $3.2M in annual cloud spend reduced to $1.3M for 2024 (all of it AWS S3) after about $700K in Dell hardware, breaking even in under six months. These numbers are real. They are also the best-case scenario, and the coverage that quotes them almost universally omits the failure modes.

Roughly 70% of repatriation attempts stall in a permanent expensive hybrid state or revert within 18 months, having spent significant engineering capital and achieved nothing durable. The causes are well-understood by teams that have executed successful exits, and almost never discussed by vendors who profit from the attempt. This is the engineering playbook those teams wish had existed: dependency audit, stateful classification, egress-cost math, 36-month TCO, and the operating model changes that determine whether you stay off the hyperscaler. The same structure is available as free interactive tools — a TCO model and a sovereignty readiness assessment.

Repatriation is a systems engineering problem with a three-to-five year TCO horizon, not an infrastructure swap. Teams that treat it as a swap are the ones who go back.

Why 83% Are Planning Exit — and Why Only ~30% Will Succeed

The repatriation momentum is real and driven by two distinct pressures that have converged in 2026. The first is cost: cloud bills that were tolerable at early scale have become structural P&L problems at maturity. The hyperscaler model charges a significant premium for convenience — managed control planes, managed databases, managed everything — and that premium compounds. A workload that costs $40K/month at $2M ARR can cost the same $40K/month at $20M ARR if you have not re-architected it, except now it represents twice the gross margin pressure. That pressure does not vanish once you repatriate — it moves in-house, which is why measuring per-namespace cost on owned metal with OpenCost matters as much after the exit as the TCO model did before it.

The second pressure is sovereignty. This is not the abstract kind. Regulated industries — fintech, healthcare, defense-adjacent, critical infrastructure — are facing a new generation of compliance requirements — NIS2, DORA's exit-strategy mandate for ICT arrangements, and the EU AI Act — that create genuine legal friction with US-parent hyperscaler data models. The CLOUD Act means that data residency guarantees from AWS or Azure are not sovereignty guarantees. But sovereignty anxiety alone does not drive action; it is the combination of sovereignty pressure plus a large enough cloud bill that makes the engineering investment worthwhile.

So why does ~70% fail? The audit across failed repatriations consistently surfaces the same root causes. First: the dependency inventory was never completed — teams discovered hyperscaler-specific bindings mid-migration that were not visible in the initial planning. Second: stateful workloads were underestimated — the assumption that Kubernetes portability means workload portability is wrong, and stateful services require data migration that is slow, risky, and often not reversible quickly. Third: the operating model shift was not resourced — running bare metal requires platform engineering capabilities that most cloud-native teams genuinely do not have, and hiring or building them takes 12-18 months. Fourth: the TCO model was built on Year 1 optimism and did not account for the Year 0 capex trough or the operational learning curve.

The Dependency Audit: Mapping What Actually Binds You to the Hyperscaler

Before any migration sequence is planned, a structured dependency audit must produce a complete inventory of hyperscaler bindings. This is not a Terraform audit. It is an application-level inventory of every service, SDK call, IAM assumption, and managed endpoint that creates a migration dependency. The audit has four layers.

Layer 1 — Compute bindings: Which workloads use hyperscaler-specific compute primitives? Lambdas, Fargate tasks, Step Functions orchestration, Spot-instance pricing assumptions baked into autoscaling logic — these are not portable Kubernetes workloads, they are cloud-native functions that require architectural rewrites, not lift-and-shift. This is usually where repatriation stalls, because there has been no portable artifact for the function layer — a gap WebAssembly components are only now starting to close.

Layer 2 — Data plane bindings: Which services have data in managed databases (RDS, Aurora, DynamoDB, Spanner, Bigtable), managed caches (ElastiCache), or managed queues (SQS, Pub/Sub)? Data migration is the hardest part of any repatriation. Aurora Serverless is not PostgreSQL — the autoscaling assumptions are different, the connection pooling behavior differs, and applications that were tuned for Aurora will require re-tuning for on-prem Postgres at scale. DynamoDB with a deeply denormalized access pattern may require a data model redesign before it can move to Cassandra or PostgreSQL, which is months of work.

Layer 3 — Identity and access bindings: IAM roles, instance profiles, IRSA (IAM Roles for Service Accounts), Cognito user pools, AWS-managed KMS keys. Every service that authenticates by assuming an IAM role needs a new identity model on bare metal — typically OIDC with a self-hosted identity provider, Vault for secrets, and Kubernetes service account tokens for workload identity.

Layer 4 — Network topology bindings: VPC assumptions, PrivateLink endpoints, Transit Gateway routing, security groups used as an east-west policy model. These are often the most invisible bindings because they are infrastructure-level, not application-level — your application does not know it is reaching a database over PrivateLink, but your migration does.

Output a migration readiness score per service: Green (portable, no rewrites), Amber (portable with data migration effort estimated), Red (requires architectural change before migration). Only Green services belong in the initial migration wave. Never let schedule pressure push Amber services into the first wave — that is where the stalls happen.

Click to zoom

Stateful Workloads Are the Hard Problem — A Framework for Classification

The Kubernetes portability story is accurate for stateless workloads. A container that takes HTTP requests, processes them, and returns a response is genuinely portable. Its Deployment manifest moves with minor annotation changes. Its resource requests get retuned for the new node topology. This is the easy part, and it is the part that vendor case studies overwhelmingly feature.

The hard part is everything with persistent state. The classification framework has three axes. First: data volume and migration velocity. A 50GB PostgreSQL database can be migrated with a week of dual-write and a planned cutover. A 40TB Aurora cluster with 200 read replicas and applications that have never been tested against standard Postgres connection pooling behavior is a different problem — one that takes months to de-risk and may require query-plan audits because Aurora's optimizer has made different choices than stock PostgreSQL for years.

Second: consistency requirements during migration. Can the service tolerate a maintenance window, or must it be zero-downtime? Zero-downtime stateful migrations require dual-write periods, change data capture (CDC) pipelines, lag monitoring, and a rollback trigger that works under production load. This infrastructure — Debezium or similar for CDC, monitoring for replication lag, automated failback — must be built and tested before any production traffic is cut over. Teams that skip this step and discover a replication issue mid-cutover have a serious incident, not a migration.

Third: API surface compatibility. PostgreSQL is PostgreSQL, with caveats. MySQL is mostly MySQL. But DynamoDB, Bigtable, Spanner, and Cosmos DB do not have open-source equivalents with the same access model — they require application-layer rewrites to move to Cassandra, ScyllaDB, or CockroachDB. This is architectural investment, not migration, and it belongs in the Red category of your dependency audit.

For storage, the on-prem equivalent of cloud block storage is typically Ceph for distributed workloads and direct-attached NVMe for latency-sensitive databases. Ceph at production scale is operationally non-trivial — it requires dedicated storage nodes, careful CRUSH map design, and platform engineers who understand its failure modes. A single-rack colocation deployment that skips Ceph and uses local NVMe with pod anti-affinity for database high availability is often the better starting point. the logistics-platform worked scenario covers exactly this architecture: four physical nodes, local NVMe for PostgreSQL with streaming replication, and a Kubernetes operator (CloudNativePG) managing failover — no distributed storage layer, no new operational complexity.

Kubernetes Portability: What's Real vs. the Marketing Version

Kubernetes portability is real at the container and manifest layer. The marketing version extends this to claim that EKS workloads are trivially portable to bare metal. The engineering reality is more granular.

What is genuinely portable: container images (OCI-compliant, architecture-matched), Kubernetes resource definitions (Deployment, Service, ConfigMap, Secret, HorizontalPodAutoscaler), and custom resources managed by CNCF-ecosystem operators (Prometheus, cert-manager, Flux, ArgoCD). These move with configuration changes, not rewrites.

What is not portable without work: cluster-level managed services that EKS/GKE provides but you must now operate yourself — the load balancer controller and ingress path (you need MetalLB or kube-vip), the storage class provisioner (you need a CSI driver for your on-prem storage), the cluster autoscaler (you need Karpenter's on-prem equivalent or Cluster API with an on-prem provider), the certificate management (ACM certificates become Let's Encrypt via cert-manager), and the ingress controller (ALB Ingress Controller becomes Nginx or Envoy Gateway).

Each of these replacements is well-supported by the CNCF ecosystem — none requires building something from scratch. But each requires someone on your team to understand it, configure it, and operate it. That person is the operating model shift, which is covered in its own section. The honest accounting: a mature EKS cluster with 15 managed add-ons will require 15 equivalent on-prem configurations, and the first time you do it, each one will take longer than expected.

# Minimal on-prem equivalents for common EKS managed add-ons

# MetalLB — L2 or BGP load balancer for bare metal.
# IMPORTANT: IPAddressPool alone is insufficient. MetalLB will still assign an EXTERNAL-IP
# from the pool — the Service looks healthy — but nothing advertises it, so the address is
# unreachable. Always pair with L2Advertisement (or BGPPeer + BGPAdvertisement for BGP mode).
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.0.10.100-10.0.10.150
---
# L2Advertisement tells MetalLB to actually advertise the pool via ARP/NDP.
# For BGP mode, replace with BGPPeer + BGPAdvertisement pointing to your ToR switch.
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: first-pool-advert
  namespace: metallb-system
spec:
  ipAddressPools:
    - first-pool

---
# kube-vip — control plane VIP (alternative to EKS managed endpoint)
# Deployed as a DaemonSet on control-plane nodes
# https://kube-vip.io/docs/installation/daemonset/

---
# CloudNativePG — PostgreSQL operator (replaces RDS)
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: postgres-primary
spec:
  instances: 3
  storage:
    size: 500Gi
    storageClass: local-nvme

The 37signals Pattern and Its Limits: When Colocation Economics Work

The 37signals repatriation is the most carefully documented public case, and it deserves precise treatment rather than citation as a general-purpose argument for repatriation. Their numbers: $3.2M annual cloud spend in 2022, reduced to $1.3M for 2024 (all of it AWS S3, which they have since been moving onto their own Pure Storage array) after about $700K in Dell hardware, colocated in two Deft data centers in Chicago and Ashburn, Virginia. The payback period was under six months. DHH documented the outcome.

What made 37signals work: they had a predictable, relatively flat load profile (Basecamp and HEY are SaaS products with known traffic patterns and minimal burst requirements), their engineering team already had bare-metal operations experience (they ran their own hardware before moving to cloud in 2017-2018), and their workload mix was dominated by web application compute — the easiest category for repatriation. They also had organizational patience: DHH and Jason Fried own the company and can make a five-year infrastructure bet without a board demanding quarterly metrics.

Where the 37signals pattern does not apply: workloads with high traffic variability (10x burst requirements mean you provision for peak, wasting capacity at baseline — the exact opposite of the cloud value proposition), teams without bare-metal operations experience (the hidden operating cost is the engineering time to build that competency), businesses where time-to-market on new regions matters (cloud regions spin up in days; colocation contracts take months), and any workload where the primary managed service binding is to a hyperscaler-specific API (DynamoDB, BigQuery, SageMaker) rather than a portable open-source equivalent.

GEICO's repatriation onto OpenStack and Kubernetes is a different pattern at a different scale — hundreds of application teams, a platform function large enough to absorb operational complexity, and a financial-services regulatory environment that makes sovereignty non-optional. Neither case is wrong; they are different engineering decisions. The error is generalizing from either one without completing the TCO model.

Network Egress as the Hidden Tax: Calculating the True Hyperscaler Bill

Most cloud cost analyses focus on compute and storage. The number that surprises engineering teams doing a repatriation analysis for the first time is egress. AWS charges $0.09/GB for the first 10 TB/month of data transferred out of a region to the internet, stepping down to $0.085/GB for the next 40 TB, $0.07/GB for the next 100 TB and $0.05/GB above 150 TB, with the first 100 GB/month free and the tiers aggregated across EC2, S3, RDS and the rest. Azure's $0.087/GB and GCP's $0.08-0.12/GB are the equivalent first tiers, with GCP also varying by destination and volume. These rates appear small until you calculate them at production data volumes.

# Egress cost example: SaaS platform with moderate data throughput

Monthly data served to end users:        20 TB
Monthly data transferred between regions:  8 TB  (analytics pipeline)
Monthly data to third-party APIs:          3 TB  (integrations)
                                         ------
Total billable egress:                    31 TB

AWS egress (tiered; 100 GB free, then $0.09 / $0.085):
  100 free + 10,240 GB x $0.09 + 20,660 GB x $0.085
  = $2,678 / month = ~$32,100 / year

On-prem colocation equivalent:
  31 TB at 10Gbps sustained = well within a 20Gbps colo port
  Typical unmetered 10Gbps colo port: $500-$800/month
  Egress cost on-prem: $0 marginal

Annual egress savings:            ~$32,100 - $9,600 = ~$22,500 / year
Over 3 years (unadjusted):        ~$67,600

# At 100 TB/month egress (common for media, analytics, video):
  AWS: ~$93,700/year (100 free + 10 TB@$0.09 + 40 TB@$0.085 + 48.7 TB@$0.07)
  On-prem: $9,600/year (same port, same math)
  3-year savings: ~$252,300 from egress alone

Egress is a permanent recurring cost that does not diminish as infrastructure matures. Compute can be reduced through reserved instances and savings plans; egress is structural — you pay for the right to own your data at hyperscaler-managed bandwidth rates. On-prem, egress is essentially a colocation port: flat-rate bandwidth that does not scale with data volume.

The full hyperscaler bill for a repatriation analysis includes compute, managed services, storage, egress (frequently underestimated by 40-60% in initial estimates), and support plans. Analyze compute alone and you will underestimate the savings.

Click to zoom

Operating Model Shift: What Your Team Needs to Run Bare Metal

This is the section most repatriation coverage omits, and it determines whether you stay off the hyperscaler. Running bare-metal Kubernetes in colocation requires capabilities your cloud-native team likely lacks, and building or hiring them takes time. Teams that plan for the transition succeed; teams that assume Kubernetes knowledge transfers directly to bare metal are the ones who go back.

Hardware lifecycle management: Ordering, racking, cabling, firmware updates, disk replacement, failed NIC diagnosis, drive health monitoring. In cloud, this is invisible — the hyperscaler handles it and you pay for the abstraction. On bare metal, someone on your team is getting paged when a disk pre-fails, needs to know how to initiate a hot-swap, and needs to coordinate with your colocation provider's remote hands when the physical intervention cannot wait for your next site visit.

Network operations: On-prem networking is not VPCs and security groups. You are configuring switches, managing VLANs, handling BGP sessions with your colocation provider (or working with their NOC to do so), and diagnosing layer 1-3 issues that cloud abstracts away. Reaching between sites and back to your engineers needs its own coordination plane too — self-hosted Headscale or NetBird, not a Transit Gateway you no longer have. If your colocation provider offers managed networking, take it for the first year.

Storage operations: Whether you are running Ceph, local NVMe, or NFS, someone needs to understand the storage layer's failure modes, capacity planning, and performance characteristics. This is a distinct skill set from Kubernetes storage class configuration.

What to outsource in year one: Network management (managed switching from your colo provider), out-of-band management (IPMI/iDRAC remote access through the colo's KVM-over-IP), and physical security and power. These are not differentiating capabilities — outsource them and spend your engineering time on the Kubernetes platform.

The staffing model for a successful repatriation: one senior platform engineer with bare-metal networking experience (or the willingness to develop it quickly), one Kubernetes platform engineer who already understands the CNCF storage and networking ecosystem deeply, and budget for six months of parallel cloud spend while the on-prem environment is proven. If you cannot resource this, the repatriation will stall — not because bare metal is wrong for you, but because you will hit an operational issue that no one on the team can diagnose quickly under production pressure. CI compute is often the first workload to repatriate: running GitHub Actions runners on Kubernetes with ARC hands you owned build nodes that sit inside the cluster, with no new infrastructure paradigm — and the savings on hosted-runner minutes compound quickly at scale.

That is the ROI repatriation actually returns, and it precedes the bill: years in the cloud is enough time for an organization to get very good at paying for things nobody on the team can fully explain, because a managed platform lets the underlying cost drivers stay abstract. Repatriation reverses that by force — every dollar now has to trace to a server, a link, or an hour of someone's time, so understanding stops being optional. The lower bill that follows is a consequence of that understanding, not the point of the exercise.

The 36-Month TCO Model: How to Build the Business Case

A credible repatriation business case requires a 36-month TCO model that accounts for all costs on both sides of the comparison. The cloud side is usually underestimated (egress, support plans, managed service premiums). The on-prem side is usually underestimated on the operational cost side (salaries for the people running it, maintenance contracts, colocation cost including power and cooling at actual density).

Cloud baseline (annual, typical $1.6M/year profile): Compute at $720K (EC2/node instances), managed databases at $360K (RDS/Aurora), storage at $120K (S3, EBS), egress at $240K (underestimated in most analyses), managed Kubernetes control plane at $8.76K ($0.10/cluster/hour × 10 clusters × 8,760 hours), support plan at approximately $145K (Enterprise Support is the greater of $5K/month or 10% of monthly AWS charges below the $150K/month breakpoint — the percentage bands are monthly, not annual). Total: approximately $1.6M/year baseline.

On-prem Year 0 (one-time): Hardware at $480K (20 dual-socket servers, 1TB RAM each, 10GbE, NVMe — Dell PowerEdge R750 class), networking at $48K (managed switches, cabling, patch panels), colocation setup at $24K (cage buildout, cross-connects), IPMI/out-of-band infrastructure at $12K, initial software licenses at $36K (if applicable — prefer open-source to preserve optionality). Total Year 0 capex: approximately $600K.

On-prem annual run rate (Year 1+): Colocation (power, cooling, space) at $120K (2 cabinets at $5K/month each), bandwidth (unmetered 10Gbps) at $10K, hardware maintenance contracts at $48K (8% of capex/year), incremental engineering head count or re-allocation at $200K (1 FTE senior platform engineer), software and tooling at $24K. Total annual: approximately $402K. But in Year 1, add the learning curve overhead — typically 30-40% in unplanned engineering hours for the first 12 months, adding approximately $120K. Conservative Year 1 total: $522K.

Year 1 net: $1.6M cloud baseline − $522K Year 1 operations = approximately $1,078K against running cloud cost, before the $600K Year 0 capex. With a typical 12-month migration and 50% cloud residual, Year 1 net falls to ~$278K and the cumulative position after Year 0 capex stays negative (~−$322K) — normal. The model works over 36 months, not 12.

36-month total: Cloud baseline over 36 months = $4.8M. On-prem total (Year 0 capex + 3 years operations including Year 1 overhead) = $600K + $522K + $402K + $402K = $1.926M. Net savings over 36 months: $2.874M. This is in the range of published case studies and is achievable with a disciplined execution. The 37signals $3.2M-to-$1.3M trajectory is better than this model — their load profile was simpler and their team's prior bare-metal experience compressed Year 1 costs significantly.

# 36-Month TCO Comparison (simplified; adjust to your actuals)

                     Year 0      Year 1      Year 2      Year 3      Total
CLOUD BASELINE
  Compute            $720K       $720K       $720K       $720K
  Managed DBs        $360K       $360K       $360K       $360K
  Storage            $120K       $120K       $120K       $120K
  Egress             $240K       $240K       $240K       $240K
  Managed K8s          $9K         $9K         $9K         $9K   # $0.10/hr x 10 clusters x 8,760h = $8,760/yr
  Support            $145K       $145K       $145K       $145K
  Incidentals          $6K         $6K         $6K         $6K   # data transfer, misc
  CLOUD TOTAL      $1,600K     $1,600K     $1,600K     $1,600K    $4,800K†

ON-PREMISES
  Hardware capex     $480K           -           -           -
  Networking setup    $48K           -           -           -    # switches, cabling, patch panels
  Colo buildout       $24K           -           -           -    # cage, cross-connects
  IPMI/OOB infra      $12K           -           -           -    # remote management
  Software (initial)  $36K           -           -           -    # licenses (prefer OSS)
  Colocation (run)        -       $120K       $120K       $120K
  Bandwidth               -        $10K        $10K        $10K
  Maintenance             -        $48K        $48K        $48K
  Platform eng            -       $200K       $200K       $200K
  Software/tooling        -        $24K        $24K        $24K
  Year 1 overhead         -       $120K           -           -
  ON-PREM TOTAL      $600K*      $522K       $402K       $402K    $1,926K

NET SAVINGS                    $1,078K     $1,198K     $1,198K    $2,874K†
(Years 1-3: cloud minus on-prem running cost; Total nets Year 0 capex)

* Year 0 = hardware $480K + setup $120K = $600K. 36-month net = $3,474K of running-cost savings minus $600K Year 0 capex = $2,874K.
† Totals cover Years 1-3. Year 0 cloud spend is excluded because it is incurred under both scenarios and cancels out; Year 0 on-prem capex is included because it is incremental. Net savings = $3,474K of Years 1-3 running-cost savings minus $600K Year 0 capex.

Note: parallel cloud spend during 12-month migration reduces Year 1 savings.
Model at 50% cloud residual in Year 1: net savings drop to ~$278K in Year 1; the 36-month total falls to roughly $2.07M -- still a strong business case.

Click to zoom

Optionality Is the Real Asset: Designing the Exit Ramp Before You Need It

The strongest argument for repatriation is not the TCO model, though the model is real — it is optionality. A team that has successfully repatriated can negotiate with hyperscalers from a position of credible alternatives. A fully cloud-dependent team has no exit ramp: they pay what the hyperscaler charges because switching cost is too high to contemplate. That asymmetry compounds as pricing power increases and internal operating knowledge atrophies.

The regulated fintech cloud exit demonstrated this directly. Three vendor negotiations had called their cloud costs 'competitive for their profile.' After a completed dependency audit and a credible migration plan, renewal produced a 28% reduction — without moving a single workload. The optionality was the leverage, not the execution.

Sovereignty is the other asset. Teams that have executed repatriation understand their infrastructure in a way cloud-native teams often do not — not because bare metal is harder, but because the cloud abstracts away knowledge of what is actually running. When you own the hardware, you know the failure modes, the network topology, and the storage characteristics you chose. That knowledge compounds into an organizational capability no managed service sells.

The long-game argument is simple: investment in infrastructure knowledge, owned hardware, and platform engineering grows in value over time; your cloud bill does not. Teams who made this bet five years ago — before repatriation became the dominant narrative — have lower infrastructure costs, stronger negotiating positions, and platform organizations that are competitive advantages rather than cost centers. That is the outcome this playbook is designed to produce.