# Stribog > Engineering-sovereignty practice — infrastructure, Kubernetes, open-source tooling, sovereign AI, and audit-grade compliance for technical organizations. Operating since 2013. Stribog partners with CTOs, VPs of Engineering, and technology executives to build production infrastructure that teams own and operate after the engagement. The practice rests on five pillars: sovereignty (owning the systems you depend on), open source as method (a discipline of review and independence, not a license checkbox), audit-grade rigor (compliance built in as a byproduct of operations), optionality (anti-lock-in architecture with exit ramps designed in), and the long game (systems built to outlast the tools and vendors that will change around them). The practice has operated since 2013. Stribog does not sell SaaS. Every engagement ends with the client's team owning and operating what was built — runbooks, architecture decision records, and GitOps repositories are deliverables, not retention mechanisms. Contact: hello@stribog.com Below is the complete text of all 123 articles from https://stribog.com/blog, newest first. Index: https://stribog.com/llms.txt --- # Postgres vs SQL Server: Standard, Enterprise, or Neither URL: https://stribog.com/blog/postgres-vs-sql-server-licence-exit-operational-parity Published: 2026-10-05 Category: Data Keywords: postgres vs sql server, sql server vs postgres, sql server per core licence cost 2025, sql server standard readable secondary replica limitation, sql server to postgresql migration parity, babelfish for postgresql self-hosted t-sql, pg_tde transparent data encryption postgres, sql server 2016 end of extended support migration TDE, columnstore and In-Memory OLTP ship in SQL Server Standard. Readable secondaries and multi-database availability groups do not. That line decides it. > **The short answer** — Postgres vs SQL Server is not decided by the feature list: in SQL Server 2025, columnstore, In-Memory OLTP, transparent data encryption, Query Store, Resource Governor and SQL Server Agent are all in Standard, the first two capped by memory rather than withheld. Two things are Enterprise-only and they set your topology instead — readable secondary replicas, and an availability group that covers more than one database. The honest verdict is three-way, not a ranking: Standard, Enterprise, or Postgres, selected by properties of your workload. Treated as a feature comparison, this question gives a stale answer in 2026. The capabilities teams budget an Enterprise upgrade for mostly stopped being Enterprise-only years ago. What did not change is the line that forces the upgrade — and it is a topology line, not a feature one. The framing is backwards too. The useful question has three answers, not two: does this workload need Enterprise, does it fit Standard, or is the only thing Enterprise would buy something Postgres gives away? ## The arithmetic that starts the conversation Microsoft publishes the numbers, so no reseller restatement is needed. On the Per Core model the SQL Server 2025 price sheet lists Enterprise at $15,123 per two-core pack and Standard at $3,945, with subscription equivalents of $5,434 and $1,418 per year. The sheet is explicit: "Pricing represents open no-level (NL) estimated retail price. For your specific pricing, contact your Microsoft reseller." What follows is a list-price model, not a quote. Two counting rules turn a price into a bill. Core SKUs "are sold in packs of two-core licenses each", and a physical server needs "sufficient licenses based on the number of physical cores on the server, with a minimum of four Licenses per Physical Processor". A two-socket, sixteen-core server takes sixteen core licences — eight packs: $31,560 on Standard, $120,984 on Enterprise. Which core number goes in is where models go wrong. Per-VM licensing is a different rule with a different input: virtual cores, plus — "if any virtual core is mapped to multiple hardware threads" — every thread each maps to; a four-licence minimum per virtual OSE; and eligibility only with Software Assurance or a subscription. It is under that per-VM heading that Microsoft notes "the 32-core limit for the Standard edition means 32 hardware threads". Pass a logical-processor count while licensing a physical server and you roughly double the bill on a hyperthreaded host. ```sql -- Inventory for a SQL Server licensing decision. -- Prints the inputs the two published licensing models actually consume. -- (a) Edition, build and HADR state of the instance being priced. SELECT SERVERPROPERTY('Edition') AS edition, SERVERPROPERTY('ProductVersion') AS product_version, SERVERPROPERTY('ProductLevel') AS product_level, SERVERPROPERTY('IsHadrEnabled') AS hadr_enabled; -- (b) Sockets and cores AS SEEN BY THE SYSTEM THIS INSTANCE RUNS ON. -- REQUIRES SQL SERVER 2016 SP2 OR LATER: socket_count, cores_per_socket -- and numa_node_count are all documented "Applies to: SQL Server 2016 -- (13.x) SP2 and later versions". On 2016 RTM/SP1 or earlier this part -- fails with Invalid column name 'socket_count'. The fallback is -- cpu_count with hyperthread_ratio from the same DMV -- "the ratio of the -- number of logical or physical cores that are exposed by one physical -- processor package" -- or a hardware/hypervisor inventory. Neither is a -- substitute for a physical-core count; both are inputs to estimating it. -- sys.dm_os_sys_info documents socket_count as "the number of processor -- sockets available on the system" -- the system being the OS the instance -- runs in. On bare metal that product IS the figure the physical-server -- rule licenses, and the figure the four-licences-per-physical-processor -- floor applies to. Inside a guest VM it is the GUEST's virtual topology: -- the host's physical sockets and cores cannot be read from this DMV at -- all and must come from the hypervisor or a hardware inventory. The -- documentation says only "the system"; the guest reading is an inference -- from that word, and it is the caveat that most often breaks a model. -- Do NOT substitute the logical count below for either. SELECT socket_count AS sockets_visible_to_this_instance, cores_per_socket, socket_count * cores_per_socket AS cores_visible_to_this_instance, -- (c) LOGICAL processors / schedulers. Relevant only under per-VM licensing, -- where every hardware thread a virtual core maps to must also be -- licensed -- and where Standard's 32-core limit means 32 hardware -- threads. Per-VM licensing itself requires SA or a subscription. cpu_count AS logical_processors_per_vm_model, scheduler_count FROM sys.dm_os_sys_info; -- (d) Configured memory against Standard's 256 GB buffer-pool ceiling. The -- documented limit is "Maximum memory for buffer pool per instance", and -- max server memory bounds more than the buffer pool -- so this is the -- knob a DBA checks, not literally the same object as the limit. SELECT name, value_in_use AS configured_mb, 256 * 1024 AS standard_buffer_pool_ceiling_mb FROM sys.configurations WHERE name IN ('max server memory (MB)', 'min server memory (MB)'); -- (e) Memory-optimized data against Standard's 32 GB per-database cap. -- UNLIKE (a)-(d), THIS PART IS NOT INSTANCE-WIDE. -- sys.dm_db_xtp_memory_consumers reports only the database you are -- connected to, and the cap is per DATABASE -- so run part (e) in every -- user database, not once in master. A zero from master says nothing -- about a user database on the same instance that is over the cap. -- A database with no memory-optimized objects returns no rows here, so -- the bare SUM would be NULL, not 0 -- ISNULL makes that read as zero -- usage rather than as a broken query. SELECT DB_NAME() AS database_name, ISNULL(SUM(allocated_bytes) / 1024 / 1024, 0) AS in_memory_oltp_allocated_mb, 32 * 1024 AS standard_per_database_cap_mb FROM sys.dm_db_xtp_memory_consumers; -- Sweep every database instead of trusting one connection. sp_MSforeachdb is -- undocumented; a cursor over sys.databases where state = 0 is the supported -- equivalent if you would rather not depend on it. -- EXEC sp_MSforeachdb N'USE [?]; -- SELECT DB_NAME() AS database_name, -- ISNULL(SUM(allocated_bytes) / 1024 / 1024, 0) AS in_memory_oltp_allocated_mb -- FROM sys.dm_db_xtp_memory_consumers;'; ``` The second published path exists on only one side of that line. Standard has a Server + CAL model at $989 per server plus $230 per CAL; the editions table's scale-limits footnote records that Enterprise "with Server + Client Access License (CAL) based licensing (not available for new agreements) is limited to a maximum of 20 cores per SQL Server instance". CALs "are required even when the server is accessed indirectly", and "using hardware or software that reduces the number of devices or users that directly access or use the software (multiplexing/pooling) does not reduce the number of CALs required". A connection pool does not collapse ten thousand users into four application servers. ```bash #!/usr/bin/env bash # SQL Server 2025 list-price model. NOT A QUOTE. # # The Per Core and server prices here are Microsoft's published open no-level # estimated retail prices from the SQL Server 2025 pricing sheet, whose # footnote on those rows reads: "Pricing represents open no-level (NL) # estimated retail price. For your specific pricing, contact your Microsoft # reseller." The two subscription figures come from that sheet's separate # subscription and add-on pricing table. Negotiated EA/MPSA pricing is # normally lower. This models the SHAPE of the decision, not your invoice. set -euo pipefail # --- Inputs ----------------------------------------------------------------- # CORES: which number belongs here depends on which model you are pricing. # Licensing a PHYSICAL server -> pass PHYSICAL cores. The four-licence floor # below applies to this figure. # Licensing a VIRTUAL MACHINE -> pass virtual cores, PLUS every hardware # thread each virtual core is mapped to. Per-VM licensing requires # Software Assurance or a subscription and has its own four-licence # minimum per virtual OSE. The two models are not interchangeable. CORES=${CORES:-16} SOCKETS=${SOCKETS:-2} # USERS_DEVICES: EVERY user and device that accesses the instance, including # indirect access through an application tier, and including devices nobody # attends. Multiplexing or pooling does not reduce the count. This is not your # visible seat count, and the break-even below is only valid against the # honest total. USERS_DEVICES=${USERS_DEVICES:-200} # --- Published open no-level prices ----------------------------------------- ENT_PACK=15123 # Enterprise, 2-core pack, perpetual STD_PACK=3945 # Standard - per core, 2-core pack, perpetual # Subscription is a separate licensing model, not a per-year Per Core rate. ENT_PACK_YR=5434 # Enterprise, 2-core pack, subscription STD_PACK_YR=1418 # Standard, 2-core pack, subscription STD_SERVER=989 # Standard - server (Server + CAL model) STD_CAL=230 # Standard - CAL # --- Core licence count ----------------------------------------------------- # Minimum four licences per physical processor, then rounded up to whole # two-core packs. floor=$(( SOCKETS * 4 )) licences=$(( CORES > floor ? CORES : floor )) packs=$(( (licences + 1) / 2 )) std_percore=$(( packs * STD_PACK )) ent_percore=$(( packs * ENT_PACK )) std_year=$(( packs * STD_PACK_YR )) ent_year=$(( packs * ENT_PACK_YR )) std_servercal=$(( STD_SERVER + USERS_DEVICES * STD_CAL )) breakeven=$(( (std_percore - STD_SERVER) / STD_CAL )) printf '%d cores / %d sockets -> %d core licences -> %d two-core packs\n\n' \ "$CORES" "$SOCKETS" "$licences" "$packs" printf ' Standard Per Core, perpetual : $%d\n' "$std_percore" printf ' Enterprise Per Core, perpetual : $%d\n' "$ent_percore" printf ' Standard Subscription, 1 year: $%d\n' "$std_year" printf ' Enterprise Subscription, 1 year: $%d\n' "$ent_year" printf ' Standard Server + CAL (%d) : $%d\n' "$USERS_DEVICES" "$std_servercal" printf '\n Server + CAL is cheaper ONLY at or below %d users AND devices,\n' "$breakeven" printf ' counted with indirect access and unattended devices included.\n' printf ' Enterprise Server + CAL is not available for new agreements.\n' # --- The high-availability line, never folded into the single-node total ---- # Microsoft: "licenses without SA or non-subscription licenses do not include # disaster recovery or high availability rights, including use of fail-over # clusters", and passive fail-over replicas "may be implemented in a variety # of ways, including via availability groups and fail-over cluster instances." # The SA benefit is capped: one passive fail-over replica for high # availability per licensed OSE, and its licence count "may not exceed the # number of licenses needed to cover your primary" OSE. # READ THE SECOND LINE BELOW AS A MODEL, NOT A PRICE: Microsoft states that # those rights are not INCLUDED; it nowhere instructs you to license the # passive node in full. Two fully licensed OSEs simply need no passive # fail-over benefit, which is the standard reading -- but it is a reading. printf '\nTwo-node Standard failover cluster instance:\n' printf ' with SA or a subscription : $%d (ONE passive HA replica,\n' "$std_percore" printf ' no larger than the primary)\n' printf ' without : $%d (passive node licensed in full)\n' "$(( std_percore * 2 ))" printf '\n NOTE: the SA or subscription cost that unlocks the first line is\n' printf ' NOT in that figure. Microsoft publishes no SA price on the 2025\n' printf ' sheet; the published alternative is the per-year subscription\n' printf ' above. Passive-node coverage is conditional, not free.\n' ``` At the sixteen-core shape above, Server + CAL stops being cheaper near 132 users and devices, counted honestly — a low ceiling, and an easy one to overshoot. ## The edition line that actually decides it Start with the correction, because it changes the budget. In the SQL Server 2025 editions table, transparent data encryption is Yes for Enterprise and Yes for Standard. So are Resource Governor and SQL Server Agent. Columnstore, In-Memory OLTP and Query Store are Yes in all three editions, the first two footnoted as "limited to the amount of memory specified by edition in the Scale Limits section" — capped, not withheld. None of these is a reason to buy Enterprise. The caps still matter. Standard is limited to "lesser of 4 sockets or 32 cores" per instance — 32, raised from the 24 of SQL Server 2022 and earlier — plus 256 GB of buffer pool and 32 GB of memory-optimized data per database. One database needing 80 GB of memory-optimized data is an Enterprise workload even though In-Memory OLTP ships in Standard. Now the line that decides it. The editions table reads counter-intuitively, and it is not a transcription error: Always On availability groups are Enterprise Yes, Standard No; Basic availability groups are Enterprise No, Standard Yes. A basic availability group "supports two replicas, with one database" — forty databases on one instance means forty availability groups and forty failover units. That secondary is inert too: "No read access on secondary replica. No backups on secondary replica. No integrity checks on secondary replicas." The nightly CHECKDB is not offloadable at Standard's price either. > "Readable secondary replicas are only supported with SQL Server Enterprise edition." — Microsoft, "Availability Groups Overview - SQL Server on Linux" documentation That is the SQL Server on Linux availability-group overview, scoped to that platform. The same page is blunter about Standard: "one primary, and one secondary that can only be used for availability purposes. It can't be used for anything else, such as readable queries." A SQL Server DBA answers all of that in one sentence: "or run a failover cluster instance." They are right. Standard supports Always On failover cluster instances on two nodes, against Enterprise's sixteen, and an FCI "is redundant at the server-instance level": every database on the instance fails over as one unit, no Enterprise licence. The multiplication above belongs to basic availability groups, not to Standard edition. > **What the FCI branch costs, in full** — An FCI "must use shared storage among all nodes of the FCI for database and log storage", and "the shared storage has the potential of being the single point of failure" — one copy of the data, no readable secondary. Those sentences sit on the Windows-scoped FCI page; the Linux shared-disk cluster concepts page documents the same single shared volume on Linux. Nor is it licence-free: Microsoft scopes high-availability rights "including use of fail-over clusters" to licences with Software Assurance or a subscription, so without SA the passive node is licensed in its own right — another $31,560 beside the primary's, doubling the branch. With SA it is one passive fail-over replica for high availability per licensed OSE, no larger than the primary. [Figure: Three bands. The top band lists capabilities already present in SQL Server 2025 Standard — transparent data encryption, columnstore, In-Memory OLTP, Query Store, Resource Governor, SQL Server Agent and the vector data type — and the real Standard ceiling of four sockets or 32 cores, 256 GB of buffer pool and 32 GB of memory-optimized data per database. A hexagonal gate below it carries the Per Core list prices of $15,123 and $3,945 per two-core pack and the four-licence floor per physical processor. The middle band holds the Enterprise-only boundaries: Always On availability groups, the basic availability group substitute limited to two replicas and one database, readable secondary replicas attributed to the SQL Server on Linux availability-group overview, and Query Store on secondary replicas; alongside them a failover cluster instance node stating both what that branch buys and what it costs, including shared storage as a documented potential single point of failure and the Software Assurance condition with a $31,560 unlicensed-passive-node figure. The bottom band splits the Postgres column three ways: answered by community code, answered only by a patched PostgreSQL, and not a parity question at all.] ## The decision test: which of the three your workload selects A test over workload properties, not a ranking. Run it and you land on one branch. 1. Stay on Standard with basic availability groups when the instance fits inside 32 cores (physical cores if you license the server, 32 hardware threads per VM), 256 GB of buffer pool and 32 GB of memory-optimized data per database; when few enough databases make one availability group each tolerable; and when nothing reads from the secondary. 2. Stay on Standard with a two-node failover cluster instance when the workload needs one failover unit for the whole instance, shared storage is acceptable as the single copy of the data, nothing reads off a secondary, and you hold Software Assurance or a subscription — or have budgeted the second set of Standard core licences the passive node otherwise needs. Standard caps the FCI at two nodes, so patching the passive node is this branch's only rolling headroom. 3. Buy Enterprise when read-scale off a secondary is real, when you need one failover unit across many databases *and* a second independent copy of the data, or when the core and memory ceilings bind. That is what the extra $89,424 per sixteen-core server buys at list price. 4. Move to Postgres when those Enterprise-only lines are the only reason for the upgrade and nothing depends on Integration Services, Analysis Services, distributed transactions under an availability group on Linux, or a T-SQL procedural surface deep enough to be its own rewrite. One correction to branches one and two: on Linux each basic availability group needs an arbitrating third replica, and that replica carries no licence. ALTER AVAILABILITY GROUP's CONFIGURATION_ONLY replica "can be hosted on any edition of SQL Server, including Express Edition". What multiplies per database is the group, its Pacemaker resource set and its listener, not the server count: Microsoft's basic availability groups page states that "You might have multiple Basic availability groups connected to a single instance of SQL Server", and nothing requires a separate arbitrating instance per group. The multiplication costs operational surface, not licence spend. Same discipline as pricing a cloud exit as a number. ## The Postgres answers, feature by feature If branch four is where you landed, the mapping matters more than the comparison — and the counterparts do not all have equal standing. - Audit logging → pgaudit, which "provides detailed session and/or object audit logging via the standard PostgreSQL logging facility" and implements object-level auditing "via the roles system": the grants on the role named in pgaudit.role are your evidence scope. - SQL Server Agent → pg_cron, "a simple cron-based job scheduler for PostgreSQL ... that runs inside the database as an extension" — with one constraint: it "may only be installed to one database in a cluster", and jobs elsewhere go through cron.schedule_in_database(). - The 2025 vector data type → pgvector. SQL Server's native type stores vectors "in an optimized binary format" but exposes them "as JSON arrays", to 1,998 dimensions at float32 and 3,996 at float16. pgvector covers "exact and approximate nearest neighbor search" over "single-precision, half-precision, binary, and sparse vectors". - Transparent data encryption → pg_tde, Percona's "open source, community driven and futureproof PostgreSQL extension". This one has an asterisk; see below. - Availability groups → streaming replication with a quorum expression, which deserves its own section. The engine underneath has not stood still either. PostgreSQL 18 added an asynchronous I/O subsystem "that can improve performance of sequential scans, bitmap heap scans, vacuums, and other operations", B-tree skip-scan lookups, uuidv7() for timestamp-ordered identifiers, and OAuth authentication — changes that quietly retire the reasons a 2019 evaluation gave for staying put. ```sql -- Postgres parity layer for a SQL Server migration. -- -- THIS SCRIPT SPANS TWO CONNECTIONS, and the split is not cosmetic: pg_cron -- "may only be installed to one database in a cluster" -- by convention the -- maintenance database -- while the tables being audited live in the -- application database. Run Part 1 in one and Part 2 in the other; neither -- half completes from the other's connection. -- -- PREREQUISITE NEITHER PART CAN SATISFY ITSELF: pgaudit and pg_cron must be -- listed in shared_preload_libraries and the server RESTARTED before the -- pg_cron and pgaudit CREATE EXTENSION statements below succeed (see the -- postgresql.conf settings further down). A reload is not enough. -- =========================================================================== -- PART 1 -- connect to the MAINTENANCE database: psql -d postgres -- =========================================================================== CREATE EXTENSION IF NOT EXISTS pg_cron; -- one database per cluster -- Cluster-wide, so they belong on exactly one connection: roles are -- cluster-level objects, and ALTER SYSTEM writes postgresql.auto.conf for the -- whole instance. pgaudit implements object-level auditing through the roles -- system -- it logs statements touching objects the role named in -- pgaudit.role has been granted rights on, so that grant list IS the evidence -- scope you hand an assessor. The grants themselves are in Part 2, because -- they have to be made where the tables are. CREATE ROLE auditor NOLOGIN; ALTER SYSTEM SET pgaudit.role = 'auditor'; ALTER SYSTEM SET pgaudit.log = 'ddl, role, write'; SELECT pg_reload_conf(); -- The SQL Server Agent equivalent. schedule_in_database() is how a job reaches -- a database other than the one pg_cron is installed in. SELECT cron.schedule_in_database( job_name => 'nightly-vacuum-analyze', schedule => '0 2 * * *', command => 'VACUUM (ANALYZE) public.ledger_entries', database => 'ledger' ); -- The cron schema exists only in the database pg_cron was installed into, so -- this readback belongs here and nowhere else. SELECT jobid, jobname, schedule, database, active FROM cron.job ORDER BY jobid; -- =========================================================================== -- PART 2 -- reconnect to the APPLICATION database: psql -d ledger -- =========================================================================== -- Extensions are per-database objects, so these two are created again here, -- in the database holding the tables being audited and the vectors being -- searched. Do NOT add CREATE EXTENSION pg_cron to this part -- it is already -- installed in the one database per cluster that may have it. CREATE EXTENSION IF NOT EXISTS pgaudit; CREATE EXTENSION IF NOT EXISTS vector; -- the pgvector project -- The evidence scope, granted where the relations actually exist. Run these -- against the maintenance database instead and both fail with "relation does -- not exist". GRANT SELECT, INSERT, UPDATE, DELETE ON public.ledger_entries TO auditor; GRANT SELECT, INSERT, UPDATE, DELETE ON public.customer_tokens TO auditor; ``` ```ini # postgresql.conf -- EXCERPT. Only the settings this article argues about. # ... # Both pgaudit and pg_cron are preload libraries. Neither CREATE EXTENSION # succeeds until this line is present and the server has been restarted -- # a reload is not enough. shared_preload_libraries = 'pgaudit,pg_cron' # ... # The commit guarantee. ANY 1 of two listed standbys must acknowledge before # the primary commits -- the analogue of requiring one synchronised secondary # on an availability group. synchronous_commit = on synchronous_standby_names = 'ANY 1 (standby_a, standby_b)' # ... # WAL retention. wal_keep_size is the floor kept for any standby; # max_slot_wal_keep_size is the backstop that stops a dead replication slot # pinning WAL until the primary's volume fills. wal_keep_size = 4GB max_slot_wal_keep_size = 64GB # ... # Object-level audit logging. pgaudit.role names the role whose grants define # the evidence scope; the log classes are what an assessor reads. pgaudit.log = 'ddl, role, write' pgaudit.log_relation = on pgaudit.log_parameter = off # keep bound values out of the audit trail # ... ``` ## What does not port Three things turn this migration into two projects, and none is a gap in the engine. Microsoft ships Integration Services and Analysis Services as separate server features beside the database engine — the first "a set of graphical tools and programmable objects for moving, copying, and transforming data", the second the tools for OLAP and data mining. The question is not whether Postgres has an equivalent, but whether your ETL and cubes belonged in the database decision at all. A hundred SSIS packages running the nightly load is a separate procurement — the same way choosing a migration tool is a decision about rollback and evidence, not syntax. Distributed transactions are the second, and the scope matters. Microsoft's statement: "DTC isn't yet supported on Availability Groups on Linux. If your applications require the use of distributed transactions and need an AG, deploy SQL Server on Windows." That is about availability groups on Linux — not a claim that SQL Server on Linux has no distributed transactions, and silent about Windows. Third is the procedural T-SQL surface, where Babelfish for PostgreSQL is the credible answer: it "understands the SQL Server wire-protocol and T-SQL ... so you don't have to switch database drivers or rewrite all of your application queries". There is no honest coverage percentage to quote. The real constraint is structural, and it is three sections below. ## Replacing an availability group The translation that matters is not "availability group becomes streaming replication" but two narrower questions: how many replicas must acknowledge before the primary commits, and who arbitrates a failover. Both engines answer both, with different numbers of machines. On SQL Server the commit guarantee is REQUIRED_SYNCHRONIZED_SECONDARIES_TO_COMMIT, introduced in SQL Server 2017, which "sets a minimum number of synchronous secondary replicas required to commit before the primary replica commits a transaction" — default 0, maximum the replica count minus one. Arbitration is what costs hardware: "arbitration under Linux happens in SQL Server, where all the metadata is stored", and "having just a two-node configuration doesn't work for an AG." For quorum and automatic failover at cluster type External an availability group "either must: Have three synchronous replicas (SQL Server Enterprise edition only); or Have two replicas (primary and secondary) and a configuration-only replica." On Standard that is three replicas per group — primary, secondary, configuration-only — which for a single group means three instances. ```sql -- Basic availability group, SQL Server Standard, Linux, cluster type External. -- -- THREE instances, not two. The third holds the AG configuration in master and -- none of the user databases. Per CREATE AVAILABILITY GROUP's -- CONFIGURATION_ONLY option, that replica "Can be hosted on any edition of SQL -- Server, including Express Edition" -- an instance and an endpoint to -- operate, not a licence to buy. -- -- PREREQUISITES THIS STATEMENT CANNOT SATISFY ITSELF. Pasted on its own it -- errors instead of producing the group above. Per Microsoft's "Configure an -- availability group for high availability on Linux": -- 1. Always On enabled on EACH of the three instances, then a restart: -- sudo /opt/mssql/bin/mssql-conf set hadr.hadrenabled 1 -- sudo systemctl restart mssql-server -- 2. A certificate exchanged between the nodes and a DATABASE_MIRRORING -- endpoint on every instance, on the port used below: -- CREATE ENDPOINT [Hadr_endpoint] AS TCP (LISTENER_PORT = 5022) -- FOR DATABASE_MIRRORING (ROLE = ALL, -- AUTHENTICATION = CERTIFICATE dbm_certificate, -- ENCRYPTION = REQUIRED ALGORITHM AES); -- ALTER ENDPOINT [Hadr_endpoint] STATE = STARTED; -- On the node hosting the configuration-only replica the role is WITNESS -- -- on Express Edition it is the only valid value. -- 3. [ledger] in FULL recovery with a valid log backup, because this -- statement names it in FOR DATABASE: -- ALTER DATABASE [ledger] SET RECOVERY FULL; -- BACKUP DATABASE [ledger] TO DISK = N'/var/opt/mssql/data/ledger.bak'; -- A DIVERGENCE TO KNOW ABOUT: FOR DATABASE is valid grammar and the shape -- the Windows examples use, but Microsoft's Linux examples all create the -- group with FOR REPLICA ON and no database, then ALTER AVAILABILITY -- GROUP [ag_ledger] ADD DATABASE [ledger] once the secondaries have -- joined. Naming it here is what pulls this prerequisite forward. -- -- AND AFTERWARDS: SEEDING_MODE = AUTOMATIC seeds nothing until the group is -- granted the right to create databases -- on the primary immediately after -- CREATE, and on each secondary once it has joined (a configuration-only -- replica needs the join alone): -- ALTER AVAILABILITY GROUP [ag_ledger] GRANT CREATE ANY DATABASE; -- primary -- ALTER AVAILABILITY GROUP [ag_ledger] JOIN WITH (CLUSTER_TYPE = EXTERNAL); -- ALTER AVAILABILITY GROUP [ag_ledger] GRANT CREATE ANY DATABASE; -- secondary -- The Pacemaker resource is configured AFTER the group exists, and -- FAILOVER_MODE = EXTERNAL does nothing until it is. That ordering -- install, -- configure the AG, then configure the cluster resource manager -- is the -- roadmap on a DIFFERENT page, "Configure a Pacemaker cluster for SQL Server -- availability groups", and so is the fencing warning: that page's examples -- "don't use fencing agents. They are for testing and validation only." -- Production needs one. CREATE AVAILABILITY GROUP [ag_ledger] WITH ( BASIC, CLUSTER_TYPE = EXTERNAL ) FOR DATABASE [ledger] REPLICA ON N'sqlnode-a' WITH ( ENDPOINT_URL = N'tcp://sqlnode-a.db.example.com:5022', AVAILABILITY_MODE = SYNCHRONOUS_COMMIT, -- FAILOVER_MODE = EXTERNAL is documented in CREATE AVAILABILITY -- GROUP's add_replica_spec; ALTER AVAILABILITY GROUP's reference -- grammar lists FAILOVER_MODE = { AUTOMATIC | MANUAL } only. Set -- it here, at CREATE time -- do not rely on an ALTER to set it. FAILOVER_MODE = EXTERNAL, SEEDING_MODE = AUTOMATIC ), N'sqlnode-b' WITH ( ENDPOINT_URL = N'tcp://sqlnode-b.db.example.com:5022', AVAILABILITY_MODE = SYNCHRONOUS_COMMIT, FAILOVER_MODE = EXTERNAL, SEEDING_MODE = AUTOMATIC ), -- The arbitrating replica. No FAILOVER_MODE and no SEEDING_MODE -- clause: those options "aren't supported when availability_mode is -- set to configuration_only for a replica". Its mirroring endpoint -- must be of type WITNESS, and the availability mode can't be altered later. N'sqlnode-c' WITH ( ENDPOINT_URL = N'tcp://sqlnode-c.db.example.com:5022', AVAILABILITY_MODE = CONFIGURATION_ONLY ); GO -- The commit guarantee. Postgres equivalent: synchronous_standby_names with -- an ANY quorum count (see the next script). -- -- CAVEAT, and it is the whole point of this comment: the value you set here is -- not the value that persists. "For cluster type EXTERNAL the setting is -- changed when the availability group is added to a cluster resource", and on -- Linux "the value is configured automatically by the cluster on the AG -- resource itself." Pacemaker owns this number. Read it back; do not assume it. ALTER AVAILABILITY GROUP [ag_ledger] SET (REQUIRED_SYNCHRONIZED_SECONDARIES_TO_COMMIT = 1); GO -- MODIFY REPLICA carries no FAILOVER_MODE clause here on purpose: these -- replicas are EXTERNAL, and that value appears only in CREATE's documented -- grammar. Leave the failover mode where CREATE put it. -- Postgres equivalent of AVAILABILITY_MODE: synchronous_commit. ALTER AVAILABILITY GROUP [ag_ledger] MODIFY REPLICA ON N'sqlnode-b' WITH (AVAILABILITY_MODE = SYNCHRONOUS_COMMIT); GO SELECT ag.name AS availability_group, ag.basic_features, ag.cluster_type_desc, ag.required_synchronized_secondaries_to_commit AS required_sync_secondaries, ar.replica_server_name, ar.availability_mode_desc, ar.failover_mode_desc, ar.secondary_role_allow_connections_desc AS secondary_readable FROM sys.availability_groups AS ag JOIN sys.availability_replicas AS ar ON ar.group_id = ag.group_id ORDER BY ag.name, ar.replica_server_name; GO ``` Postgres expresses the same guarantee as a setting rather than a topology. The ANY keyword in synchronous_standby_names "specifies a quorum-based synchronous replication and makes transaction commits wait until their WAL records are replicated to at least num_sync listed standbys" — so ANY 1 (standby_a, standby_b) is the direct analogue of requiring one synchronised secondary. The arbitration role that needed a third SQL Server instance moves to the operator, alongside fencing and leader election; those mechanics are covered in running PostgreSQL on Kubernetes with CloudNativePG. ```sql -- Postgres half of the availability-group translation. Run as superuser. -- Maps to REQUIRED_SYNCHRONIZED_SECONDARIES_TO_COMMIT = 1: commits wait until -- the WAL is replicated to at least ONE of the two listed standbys. On a -- SELF-MANAGED cluster the value lands in postgresql.auto.conf, which you own -- and can diff -- and which overrides the same setting in the postgresql.conf -- file shown earlier, so this statement is the one that wins. UNDER AN OPERATOR IT -- DOES NOT: CloudNativePG's replication documentation states that "direct -- configuration of the synchronous_standby_names option is not permitted", -- and that the operator populates the option with the names of local pods, -- customisable through the .spec.postgresql.synchronous stanza. -- https://cloudnative-pg.io/docs/1.30/replication/ -- A separate default, documented on its PostgreSQL-configuration page and not -- tied to this setting, disables ALTER SYSTEM on new clusters altogether: -- every statement below needs .spec.postgresql.enableAlterSystem = true. -- https://cloudnative-pg.io/docs/1.30/postgresql_conf/ -- Same question as Pacemaker owning the SQL Server value: find out which -- layer writes the number, then read the number back from the engine. ALTER SYSTEM SET synchronous_standby_names = 'ANY 1 (standby_a, standby_b)'; -- Maps to AVAILABILITY_MODE = SYNCHRONOUS_COMMIT. ALTER SYSTEM SET synchronous_commit = 'on'; -- WAL retention for the standbys' replication slots. max_slot_wal_keep_size is -- the backstop: a slot is allowed to pin this much WAL and no more. ALTER SYSTEM SET wal_keep_size = '4GB'; ALTER SYSTEM SET max_slot_wal_keep_size = '64GB'; SELECT pg_reload_conf(); -- Read the resulting guarantee back: sync_state tells you which standbys are -- actually counted toward the quorum right now, not which ones are listed. SELECT application_name, state, sync_state, write_lag, flush_lag, replay_lag FROM pg_stat_replication ORDER BY sync_state, application_name; -- And the slots, because an inactive slot is the failure mode, not the standby. SELECT slot_name, active, wal_status, safe_wal_size FROM pg_replication_slots ORDER BY slot_name; ``` [Figure: Three topologies side by side. On the left, a SQL Server availability group on Linux at cluster type External: Pacemaker and Corosync as the external cluster manager above three instances — a primary and a synchronous secondary that can only be used for availability purposes, plus a configuration-only replica that holds the availability-group configuration in master, has no user databases, uses a witness endpoint, takes no failover-mode clause, and can be hosted on any edition including Express. A hexagon explains that a two-node configuration does not work for an availability group. In the centre, the Standard failover cluster instance branch: an active node and a passive node over a single shared-storage cylinder documented as a potential single point of failure, with the Software Assurance condition on the passive node. On the right, the Postgres equivalent: an operator holding fencing and leader election, a primary and two readable standbys under a quorum synchronous_standby_names expression, replication slots with their WAL-pinning failure mode, and a WAL archive to object storage as an independent second copy of the data.] ## Failure modes you inherit on each side The edges are real and different. On SQL Server under Linux the first is layer confusion: cluster type External means "Pacemaker is used underneath the AG" and "requires that the failover mode is set to External as well" — a mismatch is a cluster that looks configured and will not fail over. Second, "quorum and fencing a failed node must be implemented properly from a Pacemaker perspective" — the database's health model does not save you from a split brain the cluster layer permitted. Third, what the script above flags: on Linux the cluster reconfigures the commit guarantee, so a T-SQL value never re-read is a guarantee you only believe you have. Alert on what you read back — a failover you have never rehearsed is a plan, not a capability. On Postgres the dominant edge is the replication slot. A slot keeps the primary's WAL until the standby consumes it — fine until the standby goes away and stays away: WAL accumulates, the volume fills. max_slot_wal_keep_size is the designed backstop, pg_replication_slots.wal_status the signal: alert on an inactive slot with a shrinking safe_wal_size, not standby lag alone. The second edge fails the other way. synchronous_standby_names lists candidates; pg_stat_replication.sync_state reports which count now. A listed standby showing async — usually an application_name mismatch — has silently left the candidate set: the guarantee holds, on fewer machines than you think. Fall below num_sync counted standbys and commits wait rather than proceed unsynchronised — write latency and hung sessions, not silent loss. Alert on the count of sync_state in ('sync','quorum') and on commit wait time. ## Audit evidence and the patched-Postgres problem An assessor asks either engine the same two questions: show me privileged access is logged, and the data encrypted at rest. On SQL Server both answers are in Standard — transparent data encryption is a Yes in the editions table, and audit is native. On Postgres the first is community code and in-engine encryption at rest is not; volume encryption (LUKS, encrypted disks) clears the control without per-tablespace keys or protection from filesystem access. pgaudit is an unambiguous win on evidence quality: object-level logging keyed to a role makes that role's grant list *the* documented scope. > **The optionality trap** — Both parity answers trade a licence for a distribution dependency. pg_tde "is bundled as a component of Percona Server for PostgreSQL and requires its patches to function". Babelfish's extensions "depend on patches to community PostgreSQL". Neither is proprietary — the Babelfish extensions ship under the Apache License 2.0, so the source is forkable. But "move to Postgres" and "move to community Postgres" are different projects with different exit ramps, and a plan that conflates them has swapped a per-core licence for a patched build whose cadence it does not control. That is a trade, not a gotcha: a patched distribution with an open licence and a public modified tree beats a per-core proprietary one. It is still not stock community PostgreSQL, and the difference surfaces at upgrade time. Same shape as a licence-driven fork migration between search engines: the code is open; the question is whose release schedule you depend on. ## Exit ramps and the long game Published support horizons set the deadline, and one has already passed. SQL Server 2016's extended support ended on 14 July 2026 — a 2016 estate is not approaching a deadline, it is inside paid Extended Security Updates, Year 1 running to 13 July 2027. SQL Server 2017's extended support ends on 12 October 2027: one migration cycle for anything with an ETL layer. Both tables print an expiry a calendar day after the announced date — "7/15/2026 6:59:59 AM", "10/13/2027 6:59:59 AM" — beneath a header saying dates are Pacific, which does not reconcile. Plan against Microsoft's own announcement: "As of today, July 14, 2026, SQL Server 2016 has reached end of support." On 2016 you pay for time; on 2017 you spend it. The Postgres side is a cadence, not a cliff: the project "releases a new major version containing new features about once a year" and "supports a major version for 5 years after its initial release". PostgreSQL 19 was still at Beta 4 when this was written in early October 2026, its announcement putting a release candidate in "early October" and noting "the PostgreSQL 19 GA may also occur in October" — so 18 is what you plan against today, a status worth re-checking. The asymmetry is in the exits. Leaving SQL Server costs a schema and dialect conversion, an ETL re-platform and an HA rebuild. Leaving Postgres later costs the conversion and the HA rebuild — not the whole list, because the engine is not also your scheduler, ETL product and OLAP server. Consolidation is convenient until it is what you are undoing, and the arithmetic that makes a per-socket hypervisor licence hard to leave applies to a per-core database one. So run the test, not the comparison. If the workload fits Standard's ceilings and tolerates per-database failover units, stay, and spend the difference on rehearsing the failover. If the only thing Enterprise buys you is a readable secondary and one failover unit across many databases, that is exactly the pair Postgres gives away. And if you move, decide whether you are moving to Postgres or to somebody's Postgres. Both are defensible. Only one ends the conversation. ### FAQ Q: Is Postgres better than SQL Server? A: That is the wrong comparison, and it produces the wrong answer about half the time. The useful question has three outcomes. SQL Server 2025 Standard already includes transparent data encryption, columnstore, In-Memory OLTP, Query Store, Resource Governor and SQL Server Agent — all six are listed as available in Standard in Microsoft's own editions table, with columnstore and In-Memory OLTP footnoted as capped by the edition's memory limits rather than withheld — so most of what teams assume forces an Enterprise upgrade does not. Two things do: readable secondary replicas, which Microsoft's SQL Server on Linux availability-group documentation says are "only supported with SQL Server Enterprise edition", and availability-group coverage of more than one database, since Standard's basic availability group "supports two replicas, with one database". If neither of those binds, Standard is defensible. If both do, the real comparison is Enterprise at $15,123 per two-core pack against Postgres, where quorum synchronous replication and readable standbys are in community code. Q: How much does SQL Server 2025 cost per core? A: Microsoft's published open no-level estimated retail prices are $15,123 per two-core pack for Enterprise and $3,945 per two-core pack for Standard, with subscription equivalents of $5,434 and $1,418 per year. Core SKUs are sold only in two-core packs, and licensing a physical server requires licences for every physical core with a minimum of four per physical processor. A two-socket, sixteen-physical-core server is therefore eight packs: $31,560 on Standard and $120,984 on Enterprise. Microsoft states on the price sheet that these are open no-level estimated retail prices and directs customers to a reseller for their specific pricing, so treat any figure derived from them as a model rather than a quote. Note also which core count you are using: per-VM licensing counts virtual cores plus every hardware thread they map to, requires Software Assurance or a subscription, and is a different rule from the physical-core count above. Q: Can SQL Server Standard fail over multiple databases as a single unit? A: Yes — with a failover cluster instance, not with availability groups. Standard supports Always On failover cluster instances on two nodes (Enterprise supports sixteen), and an FCI "is redundant at the server-instance level", so every database on the instance fails over together without an Enterprise licence. The per-database failover unit is a property of basic availability groups, which support "two replicas, with one database". The FCI branch has two real costs. First, it "must use shared storage among all nodes of the FCI for database and log storage", and Microsoft notes "the shared storage has the potential of being the single point of failure" — one copy of the data, and no readable secondary. Both come from the Windows-scoped FCI page; the Linux shared-disk cluster concepts page documents the same arrangement. Second, it is not licence-free: Microsoft states that licences without Software Assurance or a subscription "do not include disaster recovery or high availability rights, including use of fail-over clusters", so without SA the passive node needs its own licences — a second $31,560 at list price on a sixteen-core server. Even with SA the benefit is one passive high-availability replica per licensed OSE, no larger than the primary. Q: Why does a SQL Server availability group on Linux need three instances? A: Because arbitration happens inside SQL Server rather than at the cluster layer. Microsoft states that "arbitration under Linux happens in SQL Server, where all the metadata is stored" and that "having just a two-node configuration doesn't work for an AG". To maintain quorum and enable automatic failovers at cluster type External, an availability group must have either three synchronous replicas — Enterprise edition only — or two replicas plus a configuration-only replica. That third instance stores the availability-group configuration in master, holds none of the user databases, and receives its configuration data synchronously from the primary. Crucially for a licensing decision, the CREATE and ALTER AVAILABILITY GROUP reference pages both document that this configuration-only replica "can be hosted on any edition of SQL Server, including Express Edition", so it costs an instance and an endpoint to operate but not a licence. An FCI is different: all FCI failover arbitration happens at the cluster layer, so Microsoft notes that the quorum mechanisms provided by Pacemaker "can be fine" there — which is not the same as saying fencing needs no design. Q: What is the Postgres equivalent of REQUIRED_SYNCHRONIZED_SECONDARIES_TO_COMMIT? A: A quorum expression in synchronous_standby_names. PostgreSQL's documentation states that "the keyword ANY, coupled with num_sync, specifies a quorum-based synchronous replication and makes transaction commits wait until their WAL records are replicated to at least num_sync listed standbys", so ANY 1 (standby_a, standby_b) is the direct analogue of requiring one synchronised secondary before commit. On the SQL Server side, REQUIRED_SYNCHRONIZED_SECONDARIES_TO_COMMIT "sets a minimum number of synchronous secondary replicas required to commit before the primary replica commits a transaction", with a default of 0 and a maximum of the replica count minus one. One practical difference is worth carrying into the migration: at cluster type EXTERNAL that setting "is changed when the availability group is added to a cluster resource", and on Linux "the value is configured automatically by the cluster on the AG resource itself" — so the cluster owns it. In self-managed Postgres the value stays where you put it. Under an operator it does not: CloudNativePG's replication documentation (cloudnative-pg.io/docs/1.30/replication/) states that "direct configuration of the synchronous_standby_names option is not permitted" and that the operator populates the option itself with the names of local pods, customisable through the .spec.postgresql.synchronous stanza. A separate default on its PostgreSQL-configuration page, not tied to this setting, disables ALTER SYSTEM on new clusters altogether unless .spec.postgresql.enableAlterSystem is set to true. So the question is the same on both engines — find out which layer writes the number — and either way pg_stat_replication.sync_state tells you which standbys are actually counted right now. Q: Does moving to Postgres always mean moving to community PostgreSQL? A: No, and conflating the two is a planning error that surfaces late. The two headline parity answers for a SQL Server estate both require a patched build. Percona's pg_tde, which brings transparent data encryption to Postgres, "is bundled as a component of Percona Server for PostgreSQL and requires its patches to function". Babelfish for PostgreSQL, which understands the SQL Server wire protocol and T-SQL, carries extensions that "depend on patches to community PostgreSQL". Neither is proprietary — the Babelfish extensions are licensed under the Apache License 2.0, with a PostgreSQL-licence file in the same branch — so the source is auditable and forkable, which is a materially better position than a per-core proprietary licence. It is still a distribution dependency rather than stock community code, and it shows up at upgrade time. Decide which project you are running before you start, not during the first major-version upgrade. --- # Terraform vs CloudFormation: Who Holds the Record URL: https://stribog.com/blog/terraform-vs-cloudformation-portable-state-versus-provider-native Published: 2026-10-04 Category: Delivery Keywords: terraform vs cloudformation, cloudformation vs terraform, cloudformation drift detection limitations, drift-aware change sets, cloudformation stack refactoring unsupported resources, terraform moved block stack refactor, migrate cloudformation to terraform, cloudformation iac generator import, provider-native infrastructure as code lock-in Terraform vs CloudFormation: drift-aware change sets narrowed the old gap. What remains is where the record lives, and what each engine refuses to move. > **The short answer** — Terraform vs CloudFormation is settled by where the record of your infrastructure lives, not by the feature matrix: Terraform keeps it in a state file at a path you choose, protect and can copy; CloudFormation keeps it as a stack object inside the AWS account, reached only through the service API. CloudFormation closed the most-cited gap on 18 November 2025, when drift-aware change sets added a three-way comparison against live resource state. What decides it now is what each engine refuses to move. Most comparisons of these engines argue about syntax, the part that does not matter. One writes its record to a file you can copy onto a laptop; the other keeps it as an object inside the account it manages. What an upgrade breaks, what an auditor is shown and what leaving costs all follow from that. It is also out of date almost everywhere it is published. CloudFormation acquired a three-way comparison against live infrastructure in late 2025, and the pages ranking for this question still describe its drift story as one command — which makes it better than its reputation, and moves the honest case for Terraform somewhere other than drift. ## The 2026 scoreboard, not the 2019 one Four things moved since the comparison most teams read. Drift-aware change sets, announced on 18 November 2025 as "a three-way diff between a new template, last-deployed template, and actual infrastructure state". Stack refactoring, which relocates resources between stacks without destroying them. The IaC generator, which scans an account and models what it finds. And Terraform's licensor: its Business Source License names a Licensed Work of "Terraform Version 1.6.0 or later" that "is (c) 2024 IBM Corp." Two things did not move, and neither decides it. Terraform speaks to many providers; CloudFormation only to AWS, as the native service with no additional charge for resource providers "in the following namespaces: AWS::* and Alexa::*". That gets flattened into "CloudFormation is free", which the pricing page does not say: third-party registry extensions carry two meters — $0.0009 per handler operation beyond a free tier of 1,000 per month per account, one bucket that "includes hook and third party resource provider operations" rather than an allowance each, plus $0.00008 per second of duration above the first 30 seconds. AWS's worked example reaches $12.60 for 15,000 operations: the meter that grows when a team wraps its own services as resource types. ## Two records: a file you hold, a service object you query Terraform's record is a file at a bucket and key you chose; protecting it is your job. The S3 backend is blunt: "State locking is an opt-in feature." Native locking comes from use_lockfile, which "Defaults to false" — the .tflock object beside your state exists only because someone wrote that line. Nor has the older mechanism vanished: "DynamoDB-based locking is deprecated" and "will be removed in a future minor version". Reading it is terraform plan, whose documentation names three operations: read "the current state of any already-existing remote objects", compare "the current configuration to the prior state", and propose "change actions that should, if applied, make the remote objects match the configuration" — all inspectable, because the middle input is a file on disk. CloudFormation's record is a stack object inside the account, reached through the service API. No path, no bucket, no lock, no file to leak — a real advantage: a whole class of failure — the state file copied to a laptop, two engineers applying at once — does not exist there. The cost is symmetric: a record in the account is one you query, not one you carry. ```hcl terraform { required_version = ">= 1.10.0" required_providers { aws = { source = "hashicorp/aws" version = "~> 6.67" } } # State locking is an opt-in feature of the S3 backend. use_lockfile # defaults to false, so the .tflock object exists only because this # line is here. S3 native state locking arrived in Terraform 1.10.0, so # the required_version floor above is what makes this line valid at all: # on 1.8.x or 1.9.x it is an Unsupported argument error, not a lock. # DynamoDB-based locking is deprecated and will be removed in a future # minor version -- deprecated, not gone, so a migration off it is # planned work rather than an upgrade that happens to you. backend "s3" { bucket = "example-tfstate-eu-west-1" key = "platform/network/terraform.tfstate" region = "eu-west-1" encrypt = true use_lockfile = true } } provider "aws" { region = "eu-west-1" } resource "aws_vpc" "platform" { cidr_block = "10.42.0.0/16" enable_dns_hostnames = true tags = { Name = "platform" } } # Terraform renames existing objects to the string in the `to` field and # then creates a plan, so the resource is not destroyed during the run. # This edits the address an object is recorded under -- nothing else. moved { from = aws_vpc.main to = aws_vpc.platform } ``` [Figure: Two columns. Left, Terraform: configuration and a state file at a bucket and key the operator chooses, with a dashed tflock sibling object written only when use_lockfile is set to true, a provider read of the live AWS API, and terraform plan listing its three documented operations. Right, CloudFormation: a template and a stack object held inside the AWS account, feeding a traditional change set that compares the new template with the previous template and does not account for stack drift, and a drift-aware change set that compares out to actual state read through service APIs and writes back onto every stack resource on execution. Two dashed hexagons hang off the CloudFormation column: detect-stack-drift blind spots, covering properties not explicitly set, resource types that are not checked and so not included in the results, nested stacks, and KMSKeyId; and drift-aware change-set carve-outs, covering the unsupported-type fallback, write-only properties, immutable properties, AWS-managed properties and external tag keys.] ## Drift: what each engine actually compares Start with the older mechanism; most monitoring is built on it. CloudFormation "only determines drift for property values that are explicitly set", which "doesn't include default values for resource properties" — so "explicitly set the property value, even if you are setting it to the default value". Three exclusions follow. It "does not detect drift on any nested stacks", so initiate the operation on each. It "does not perform drift detection on the KMSKeyId property of any resources". And resource types that do not support drift detection. It also reports things that are not drift: "objects contained in property arrays will be reported as drift, when in actuality they're default values supplied... from the underlying service". Nor is the explicitly-set rule total: it "also detects drift on stack-level tags". And the status itself misleads: IN_SYNC means "the current configuration of each supported resource" matches its template, and a stack "with no resources that support drift detection will also have a status of IN_SYNC" — a stack nothing could be checked in is green. > **Three AWS pages give NOT_CHECKED four meanings. Do not build on it.** — The stack-drift page says in prose that "resources that don't support drift detection are assigned a drift status of NOT_CHECKED". Both status tables on that page define it as CloudFormation having "not checked if the resource differs from its expected template configuration", never mentioning unsupported types. DescribeStackResourceDrifts takes a third line: such resources "aren't checked, and so not included". And StackResourceDrift documents NOT_CHECKED as a value "CloudFormation does not currently return". An alert keyed to it fires on nothing; an absent row is not proof a resource was examined. Coverage has to be established before the scan, and one page answers it: the user guide's Resource type support table, which "lists AWS resource types that currently support import, drift detection, and infrastructure as code (IaC) generator operations" in a column of its own. The stack-drift page names it for this question — not the Cloud Control API supported-resources page, which records something else. Enumerate your stack's types against that column, but do not read its row count as coverage: the table "is not an exhaustive list of AWS resources". Terraform's side is one sentence, published already in Terraform vs Ansible: a refresh-only plan reports drift, and applying it writes the drifted values into state rather than reverting infrastructure. ```yaml AWSTemplateFormatVersion: "2010-09-09" Description: Minimal stack whose properties are set explicitly so drift detection evaluates them. Parameters: BucketName: Type: String Description: Name of the bucket this stack owns. Resources: AuditBucket: Type: AWS::S3::Bucket # Keep the object store if the stack is deleted or the resource is # replaced. Neither policy affects drift; both affect what an exit # costs, which is the point of writing them down at creation time. DeletionPolicy: Retain UpdateReplacePolicy: Retain Properties: BucketName: Ref: BucketName # CloudFormation only determines drift for property values that are # explicitly set, and that does not include default values for # resource properties. AWS's guidance is to set the property value # even when you are setting it to the default value -- which is the # only reason every block below is spelled out. VersioningConfiguration: Status: Enabled PublicAccessBlockConfiguration: BlockPublicAcls: true BlockPublicPolicy: true IgnorePublicAcls: true RestrictPublicBuckets: true Tags: - Key: owner Value: platform Outputs: AuditBucketArn: Description: ARN of the bucket this stack owns. Value: Fn::GetAtt: [AuditBucket, Arn] ``` ```bash # -- CloudFormation --------------------------------------------------- # detect-stack-drift only INITIATES the operation. It returns an id, and # the per-resource records come from two further calls. aws cloudformation detect-stack-drift \ --stack-name "{{stack-name}}" # -> { "StackDriftDetectionId": "{{detection-id}}" } aws cloudformation describe-stack-drift-detection-status \ --stack-drift-detection-id "{{detection-id}}" # One record per resource that HAS been checked. Resources that don't # currently support drift detection aren't checked, and so not included -- # so there is no row here to alert on, and an absent row is not proof that # a resource was examined. Nested stacks need their own detect-stack-drift. aws cloudformation describe-stack-resource-drifts \ --stack-name "{{stack-name}}" # A REVERT_DRIFT change set previews reconciliation. Executing it updates # the actual state of ALL stack resources to match the desired state, even # where a resource was not explicitly changed in the template -- so an # incident fix made outside CloudFormation is overwritten by execution. aws cloudformation create-change-set \ --stack-name "{{stack-name}}" \ --change-set-name "{{change-set-name}}" \ --template-body "file://template.yaml" \ --parameters ParameterKey=BucketName,ParameterValue="{{bucket}}" \ --deployment-mode REVERT_DRIFT # -- Terraform -------------------------------------------------------- # A refresh-only plan reports drift, and applying it writes the drifted # values INTO state rather than reverting infrastructure. A plain # terraform plan is the command that proposes bringing resources back to # the configuration. terraform plan -refresh-only ``` ## The plan you get, and what it will not promise A drift-aware change set comes from create-change-set with "the --deployment-mode REVERT_DRIFT parameter". Executing one will "update the actual state of all stack resources to match the desired state, even if a resource was not explicitly changed in the template", matching "resource properties with template values" and recreating "resources deleted outside of IaC". Against a 03:00 console fix, the next deploy of an unrelated commit reverses it. The same page documents the other path, and omitting it would make that warning dishonest: "when the template definition of a resource is updated to match its actual state, drift-aware change sets reset the drift status of the resource without modifying the resource." Overwrite and adopt are both supported; execution carries out whichever the template encodes, so the review is the control. Reviewing means reading fields that decline to commit. Replacement can be Conditional — the API reference gives RequiresRecreation of Always with Evaluation of Dynamic as an example, not a definition. Action includes Dynamic, where the "exact action for the resource can't be determined", and SyncWithActual, where the "resource will not be changed, only CloudFormation metadata will change". ResourceDriftStatus is "only present for drift-aware change sets", and the API reference defines its UNSUPPORTED as "resource type does not support actual state comparison". The field that earns its place is the fallback. Where a type does not support the three-way comparison, these change sets "fall back to a comparison of previous deployment state and desired state" — quietly the old two-way diff. The exclusion list is published rather than left to guesswork — AWS::MSK::Cluster and AWS::RDS::DBProxy are on it, so read it before the review. BeforeValueFrom then reports the fallback per property, "whether the before value comes from ACTUAL_STATE or PREVIOUS_DEPLOYMENT_STATE". Separately and more narrowly, write-only "properties containing sensitive data (passwords, secrets)" are compared against previous deployment values — about those properties, not unsupported types. ```json { "ChangeSetName": "platform-revert-drift", "StackName": "platform-network", "Status": "CREATE_COMPLETE", "ExecutionStatus": "AVAILABLE", "StackDriftStatus": "DRIFTED", "Changes": [ { "Type": "Resource", "ResourceChange": { "Action": "Modify", "LogicalResourceId": "AuditBucket", "ResourceType": "AWS::S3::Bucket", "Replacement": "Conditional", "ResourceDriftStatus": "MODIFIED", "Details": [ { "Target": { "Attribute": "Properties", "Name": "BucketName", "RequiresRecreation": "Always" }, "Evaluation": "Dynamic", "ChangeSource": "ParameterReference", "BeforeValueFrom": "ACTUAL_STATE" }, { "Target": { "Attribute": "Properties", "Name": "VersioningConfiguration", "RequiresRecreation": "Never" }, "Evaluation": "Static", "ChangeSource": "DirectModification", "BeforeValueFrom": "ACTUAL_STATE" } ] } }, { "Type": "Resource", "ResourceChange": { "Action": "Dynamic", "LogicalResourceId": "LegacyProvisioner", "ResourceType": "Custom::LegacyProvisioner", "ResourceDriftStatus": "UNSUPPORTED", "Details": [ { "Target": { "Attribute": "Properties", "Name": "Endpoint" }, "Evaluation": "Static", "ChangeSource": "DirectModification", "BeforeValueFrom": "PREVIOUS_DEPLOYMENT_STATE" } ] } }, { "Type": "Resource", "ResourceChange": { "Action": "SyncWithActual", "LogicalResourceId": "AlarmTopic", "ResourceType": "AWS::SNS::Topic", "ResourceDriftStatus": "MODIFIED" } } ] } ``` ## Failure: a rollback that strands you, a state file that already moved A comparison ranking for this question gives "Rollback on failure" as "Automatic, built in" for CloudFormation against "Manual; failed apply leaves partial state" for Terraform. The Terraform half is accurate; the CloudFormation half describes what is supposed to happen, and three documented outcomes say otherwise. First, a failed drift-aware execution: CloudFormation "will roll back resources to their actual pre-deployment state and preserve out-of-band changes made since the last deployment" — the drifted world, not the last-deployed template. Second, the classic path, which can strand you at either step. A stack in UPDATE_ROLLBACK_FAILED "can't be updated, but it can be rolled back to a working state (UPDATE_ROLLBACK_COMPLETE)" — and getting there is conditional: "in most cases, you must fix the error that causes the update rollback to fail" first, a timed-out operation being the exception. Where the only way on is skipping resources, CloudFormation sets them to UPDATE_COMPLETE, leaving them "inconsistent with the state of the resources in the stack template" — and warns that otherwise "subsequent stack updates might fail, and the stack will become unrecoverable." Third, Terraform. When an apply step errors it logs the error, "updates the state file with any changes to your resources", unlocks it and exits: "your infrastructure may be in an invalid state", and "Terraform does not automatically roll back a partially-completed apply". Recovery is to resolve the error and apply again: worse on paper, simpler at 03:00, because the record is a file you can read. ## Moving a resource: a state edit versus a service operation On Terraform a move edits a record you hold; on CloudFormation it is a request a service can refuse. Terraform's moved block is gated too, by schema. Within one state file it is cheap: Terraform "renames existing objects to the string specified in the to field" and "does not destroy the resource during the Terraform run". Across types it is conditional — "each resource type has a separate schema so objects of different types are not typically compatible", and only some providers "let you change an object from one resource type to another". One case is closed: you cannot use it "to change a managed resource... into a data resource". A move between two state files is not a moved block at all: HashiCorp recommends remove and import, "because the removed and import blocks help keep a record of the configuration history", on Terraform 1.7 or newer — over terraform state mv, "a legacy command". CloudFormation stack refactoring refuses at four points. The type must be eligible: it "only supports resource types with a provisioningType of FULLY_MUTABLE", checked with describe-type. The type must not be excluded — the stack refactoring page lists AWS::AutoScaling::AutoScalingGroup, AWS::EC2::EIP, AWS::EC2::LaunchTemplate, AWS::Route53::RecordSet, AWS::CertificateManager::Certificate and AWS::CloudFormation::CustomResource among many others: six ordinary pieces of infrastructure; AWS edits that list in place, so read the page. The stack must carry no stack policy — refactoring "doesn't support stacks that have stack policies attached, regardless of what the policies allow or deny". And it stays small: "a maximum of 5 stacks (using the AWS CLI)". Those four are a subset. The same page refuses "pseudo parameters whose values differ between the source and destination stacks", refuses "the same resource into multiple stacks", and refuses a refactor that would empty one. Nothing rides along either: it "is limited to reorganizing existing resources" — no creates, deletes, configuration changes or new parameters and conditions. [Figure: A decision flow for relocating a single resource. The Terraform path passes three gates before any edit: whether the destination is in the same state file, since a different state file means removed plus import on Terraform 1.7 or newer or the legacy terraform state mv; whether the move stays within one resource type or uses a cross-type pair the provider allows, because each type has a separate schema; and whether the destination is a managed resource rather than a data resource. It ends at the moved block, annotated that the resource is not destroyed during the run. The CloudFormation path passes four sequential gates, each with a dashed edge into a shared rejected node: the type must be FULLY_MUTABLE, must not be on the published exclusion list of six named types, the stack must have no stack policy attached, and the operation must stay within five stacks on the CLI. A dashed hexagon footnote records that these four gates are a subset rather than a checklist.] ## The resources with no equivalent, in both directions No honest score exists, so take each side's bound and escape hatch instead. The IaC generator's documented bound is scope, not a number: it "only supports AWS resources that are supported by Cloud Control API in your Region". A different page — Cloud Control API's supported resources table — lists "the public resource types published by AWS that currently support AWS Cloud Control API"; counted from it on 4 October 2026, roughly 1,300 distinct AWS::*::* type names, on a page whose header says the table was last updated 23 December 2025. One count, one page, one day — not AWS's resource-type coverage, which AWS does not publish, and not the drift table. On the Terraform side, the registry API records hashicorp/aws 6.67.0 as published 30 September 2026, and counting that version's documentation index entries with a category of "resources" gives 1,722 managed resource types — again a count from a response, not a vendor total. These are not comparable universes: Terraform resources are not one-to-one with AWS type names, so subtracting one from the other invents a gap that does not exist. Know each escape hatch as cost, not parity. CloudFormation's is the custom resource: for "resources that aren't available as CloudFormation resource types" you declare AWS::CloudFormation::CustomResource or Custom::MyCustomResourceTypeName and supply "the service token, which specifies where CloudFormation sends requests to, such as an Amazon SNS topic or a Lambda function" — an endpoint you write, run and keep alive for the stack's life. Terraform's counterpart is the hashicorp/awscc provider, "fully generated based on CloudFormation schema" and "refreshed weekly" from us-east-1, with its own caveat: "Some resources generation may be suppressed, and therefore unavailable in the provider despite being supported by CloudFormation." ## What the auditor is shown, and who can read the record Compliance work is where the comparison does not go Terraform's way. A stack event ties itself to the operation that produced it: its contents carry a Timestamp — "time the status was updated" — and a ClientRequestToken, since "all events triggered by a given stack operation are assigned the same client request token". Checked against the API reference's contents list on 4 October 2026, that list names no IAM principal. Caller identity is a separate record: a CloudTrail event "is the record of an activity in an AWS account", which "can be an action taken by an IAM identity, or service", covering "both API and non-API account activity". Terraform's record has a sharper problem. "If you are developing with Terraform locally, Terraform stores your state in a plaintext file, which includes any secret values you defined in your configuration." The record an operator holds is a secret-bearing artefact needing encryption, access control, retention and a leak response — a control surface CloudFormation does not hand you, and a comparison that omits it is selling something. For the same evidence questions when the platform is yours, see NIST SP 800-53 on Kubernetes You Own: AC, AU, CM, SC, SI. The sovereign reading is not that one record is better but that they fail differently. A file you hold can be read by anyone who obtains it — and by you, forever, with no account and no API. A record inside the account cannot leak, and cannot leave. ## What a migration actually costs, each direction No duration, headcount or cost figure appears here: none is sourceable. What is documented is where each direction stops — the input to an estimate you build yourself, the same discipline as pricing a cloud exit as a number. Going in, the IaC generator is the on-ramp, and quota-shaped throughout. The scan "is region-wide and expires after 30 days", with frequency keyed to size: 10 scans per day below 10,000 resources, 1 per day above. One generated template models at most 500 resources — also the hard ceiling of any CloudFormation template, alongside 200 parameters and 200 outputs (and a template body capped at 51,200 bytes in a request, 1 MB as an S3 object). Coming out, Terraform's generator carries a warning its documentation has never withdrawn: "Configuration generation is available in Terraform v1.5 as an experimental feature. Later minor versions may contain changes to the formatting of generated configuration and behavior of the terraform plan command using the -generate-config-out flag." It will not always succeed either — "for certain resources with complex schemas, Terraform may not be able to construct a valid configuration". Expect a baseline a human rewrites, not a translation. Both directions are an import exercise, not a conversion. Nothing in a template becomes HCL, nothing in HCL a stack. The infrastructure transfers and the record is rebuilt from the live account — the strongest argument here for writing DeletionPolicy: Retain and its Terraform equivalents early. ```bash # -- Into CloudFormation: scan, then model at most 500 resources ------- # The scan is region-wide and expires after 30 days. aws cloudformation start-resource-scan # -> { "ResourceScanId": "{{scan-arn}}" } aws cloudformation create-generated-template \ --generated-template-name "{{template-name}}" \ --resources '[{"ResourceType":"AWS::S3::Bucket","ResourceIdentifier":{"BucketName":"{{bucket}}"}}]' aws cloudformation get-generated-template \ --generated-template-name "{{template-name}}" \ --format YAML # -- Out of CloudFormation: import blocks, then generate configuration - cat > imports.tf <<'HCL' import { to = aws_s3_bucket.audit id = "{{bucket}}" } HCL # Configuration generation is experimental. The output path must not # already exist, and complex schemas may not yield a valid configuration. terraform plan -generate-config-out="generated.tf" ``` ## The long game: the record you can still read in ten years CloudFormation has a native answer to keeping the source of truth in git. Git sync monitors a repository for "a CloudFormation template file that defines a stack" and "a stack deployment file that contains parameters": "when you commit changes to the template or the deployment file, CloudFormation automatically updates the stack". It is not uniformly available: the Git sync documentation names a list of Regions that is not every commercial Region — an architectural constraint, the same shape as when a GitOps control plane must reach every cluster. Terraform's long-game risk is not technical. Its licence permits production use "provided Your use does not include offering the Licensed Work to third parties on a hosted or embedded basis in order to compete with IBM Corp.'s paid version(s)", and defines a competitive offering broadly: "a Product that is offered to third parties on a paid basis, including through paid support arrangements, that significantly overlaps" with those. Each version converts to MPL 2.0 on its own Change Date, four years from publication. That is a question about your business model, not your architecture, and the exit is a migration whose hard part is the registry. So, a decision rule rather than a winner. If everything you will ever run is in AWS, and what you most want gone is a state file with secrets in it, CloudFormation is defensible and genuinely cheaper — provided you set every property explicitly, establish drift coverage against the resource-type table rather than a status code, and know what refactoring refuses to move. If you expect to operate anywhere else, or want a record still readable when the account, the vendor relationship and the licence have all changed, the file is the point. Pick the record you can still answer questions from in ten years. ### FAQ Q: Is Terraform or CloudFormation better in 2026? A: Neither, and the honest answer depends on one question: do you want the record of your infrastructure to be a file you hold or an object inside the AWS account? CloudFormation's record is a stack object reached through the service API — there is no state file to encrypt, lock, version or leak, which removes a whole class of operational failure. Terraform's record is a file at a path you choose, which you must protect yourself (the S3 backend states that "state locking is an opt-in feature" and that use_lockfile "Defaults to false"), but which you can copy, diff and read without an account. The feature gap that used to decide this narrowed on 18 November 2025, when drift-aware change sets gave CloudFormation "a three-way diff between a new template, last-deployed template, and actual infrastructure state". What remains is portability and what each engine refuses to move. Q: Does CloudFormation drift detection catch everything? A: No, and the exclusions are documented. CloudFormation "only determines drift for property values that are explicitly set, either through the stack template or by specifying template parameters", and this "doesn't include default values for resource properties" — so AWS advises setting a property "even if you are setting it to the default value". Drift detection on a stack "does not detect drift on any nested stacks"; it is never run on "the KMSKeyId property of any resources"; and resource types that do not support it are not evaluated. It also reports some things that are not drift: "objects contained in property arrays will be reported as drift, when in actuality they're default values supplied to the property from the underlying service". Critically, a green stack proves little — "a stack, stack set, or stack instance with no resources that support drift detection will also have a status of IN_SYNC". Q: Can I alert on a NOT_CHECKED drift status to find unmonitored resources? A: No — that alert will never fire, and three AWS pages word this four different ways. The stack-drift user guide says in prose that "resources that don't support drift detection are assigned a drift status of NOT_CHECKED", but both status tables on that same page define NOT_CHECKED as CloudFormation having "not checked" the resource, with no mention of unsupported types. DescribeStackResourceDrifts says resources "that don't currently support drift detection aren't checked, and so not included" — there is no record returned at all. And StackResourceDrift documents NOT_CHECKED as a value "CloudFormation does not currently return". The only reliable method is to enumerate your stack's resource types against the Drift detection column of the user guide's "Resource type support" table, which the stack-drift page itself names as the list of resources that support drift detection. An absent row is not proof a resource was checked. Q: What does executing a drift-aware change set do to an out-of-band incident fix? A: It depends on which of the two documented outcomes you encoded. If the template still carries the old value, execution overwrites the fix: drift-aware change sets "will update the actual state of all stack resources to match the desired state, even if a resource was not explicitly changed in the template", and AWS's launch post adds that during execution CloudFormation "will match resource properties with template values and recreate resources deleted outside of IaC". If instead you adopt the change into the template, the same page documents the non-mutating path: "when the template definition of a resource is updated to match its actual state, drift-aware change sets reset the drift status of the resource without modifying the resource". The control is the review step. Read BeforeValueFrom per property first — it "indicates whether the before value for a property comes from ACTUAL_STATE or PREVIOUS_DEPLOYMENT_STATE" — because unsupported types silently "fall back to a comparison of previous deployment state and desired state". Q: Can I move a resource between CloudFormation stacks the way Terraform's moved block works? A: Sometimes, and the refusals are stricter. Stack refactoring "only supports resource types with a provisioningType of FULLY_MUTABLE", excludes a published list that includes AWS::AutoScaling::AutoScalingGroup, AWS::EC2::EIP, AWS::EC2::LaunchTemplate, AWS::Route53::RecordSet, AWS::CertificateManager::Certificate and AWS::CloudFormation::CustomResource, and "doesn't support stacks that have stack policies attached, regardless of what the policies allow or deny". It spans "a maximum of 5 stacks (using the AWS CLI)", refuses pseudo parameters "whose values differ between the source and destination stacks", refuses the same resource into multiple stacks, and refuses to leave a stack empty. Terraform's moved block is gated differently rather than not at all: it renames within a state file without destroying the resource, but each resource type "has a separate schema", cross-type moves work only where the provider allows the pair, and a managed resource cannot become a data resource. A move between two state files is not a moved block at all — HashiCorp recommends removed plus import on Terraform 1.7 or newer and calls terraform state mv "a legacy command". --- # OpenShift vs Kubernetes: Pricing the Exit to Vanilla URL: https://stribog.com/blog/openshift-kubernetes-distribution-lock-in-exit-cost Published: 2026-10-03 Category: Optionality Keywords: openshift vs kubernetes, kubernetes vs openshift, openshift lock-in, openshift exit cost, migrate openshift to vanilla kubernetes, openshift routes vs ingress, security context constraints vs pod security admission, okd vs openshift, buildconfig to shipwright migration OpenShift vs Kubernetes: plain Deployments port cleanly, but Routes, SCCs, BuildConfigs and ImageStreams do not. Where the exit cost really sits. > **The short answer** — OpenShift vs Kubernetes is not a contest between two schedulers: OpenShift Container Platform 4.22 *is* Kubernetes 1.35, with a dialect on top. Deployments, Services, ConfigMaps, StatefulSets, PVCs and NetworkPolicies move to any conformant cluster unchanged, so the exit cost concentrates in five openshift.io API groups and in a mutating security-admission layer upstream deliberately lacks. Price the dialect, not the distribution — and measure its footprint before estimating anything. The question arrives disguised as a renewal: what would not renewing cost? It comes back either as "it is all Kubernetes underneath, so nothing" or as a six-figure estimate with no working behind it. Both treat the distribution as the unit of lock-in. The unit is the set of API objects your manifests name. Distribution lock-in is quieter than cloud lock-in for one reason: the API looks identical until the moment you leave. kubectl get deployments works; the YAML is the YAML you know. What does not survive is a smaller, sharper list — countable, and vendor-documented. ## What is actually OpenShift and what is actually Kubernetes Start with the substrate. Red Hat is plain: "OpenShift Container Platform 4.22 release uses Kubernetes 1.35 with CRI-O runtime." Every upstream object in that release behaves as the upstream API says. Nothing in that layer is a migration task. The workload dialect is five API groups, the giveaway being the apiVersion line in your manifests: route.openshift.io/v1 (Route), image.openshift.io/v1 (ImageStream), security.openshift.io/v1 (SecurityContextConstraints), build.openshift.io/v1 (BuildConfig), apps.openshift.io/v1 (DeploymentConfig). None resolve on upstream Kubernetes. The script below prints more than five, as expected: config.openshift.io, machine.openshift.io and the operator groups are platform APIs that leave with the cluster rather than objects you translate. One of the five Red Hat has already told you to stop using: "As of OpenShift Container Platform 4.14, DeploymentConfig objects are deprecated... Only security-related and critical issues will be fixed", and "Use Deployment objects unless you need a capability that only DeploymentConfig objects provide." The cheapest line item on the exit, and payable whether you leave or not. ```shell #!/usr/bin/env bash # The numerator of the exit estimate: upstream objects alongside these are # already portable and cost nothing. # set -e plus pipefail would abort the whole script on the first kind this # cluster does not serve, so every probe below absorbs its own failure - # including the first one: discovery treats an unresolvable aggregated # APIService as a hard error, so oc api-resources can exit non-zero on a # cluster that is otherwise healthy. set -euo pipefail echo "== Resources this cluster serves from an openshift.io API group ==" { oc api-resources -o name || true; } | { grep -E '\.openshift\.io$' || true; } | sort echo "== Live counts for the kinds with no upstream equivalent ==" for kind in \ routes.route.openshift.io \ buildconfigs.build.openshift.io \ imagestreams.image.openshift.io \ deploymentconfigs.apps.openshift.io do count=$( { oc get "$kind" --all-namespaces --no-headers 2>/dev/null || true; } | wc -l) printf '%-40s %s\n' "$kind" "$count" done echo "== Every SCC, then which one actually admitted each running pod ==" # Anything outside the shipped set is hand-written, and those are the expensive # ones: each is a policy decision to re-express as a webhook rule. The # annotation is the only honest answer to "which SCC applied here". oc get scc -o custom-columns='NAME:.metadata.name,PRIV:.allowPrivilegedContainer,RUNASUSER:.runAsUser.type' || true { oc get pods --all-namespaces --no-headers \ -o 'custom-columns=NS:.metadata.namespace,POD:.metadata.name,SCC:.metadata.annotations.openshift\.io/scc' \ || true; } | sort -k3 ``` [Figure: Three columns. Left: the upstream Kubernetes core — workloads, Service and Ingress and NetworkPolicy, ConfigMap and Secret and RBAC, PVC and StorageClass — with one edge to the right labelled already portable. Middle: the OpenShift dialect, the five openshift.io API groups plus the haproxy annotation family, with Crane drawn as a dashed internal side note that stays on OpenShift. Right: where each dialect object lands on a conformant cluster — Ingress or HTTPRoute, TLSRoute on a Passthrough listener, an edge-terminated listener plus a BackendTLSPolicy, a no-upstream-field node for the balance algorithm, Pod Security Admission plus webhooks, Shipwright or Tekton or off-cluster builds, a registry with digest pinning, and Deployment.] ## Routes: the API you will miss most Route is the object teams are fondest of and the least tidy to replace. Its TLS block requires a termination mode, and reencrypt holds the interesting field: destinationCACertificate carries "the ca certificate of the final destination" for "health checks on the secure connection" — and optional: absent it the router "may provide its own destination CA and perform hostname validation using the short service name". Everything else per-route lives in haproxy.router.openshift.io annotations rather than fields, some with no counterpart in Ingress or the Gateway API Standard channel: the balance algorithm's "default value is source for TLS passthrough routes. For all other routes, the default is random". Where the platform owns the traffic that hatch is gone too — "Red Hat does not support adding a route annotation to an operator-managed route." > **Writing Ingress on OpenShift does not get you out** — The Ingress Controller generates a Route behind each Ingress and keeps a one-way sync: "Do not manually apply changes directly to the generated route's TLS configuration. Any manual modifications are silently overwritten the next time the parent ingress object is updated or reconciled" — "particularly important ... if you operate a GitOps-managed cluster". Their reconcile loop quietly reverts yours. Passthrough is the mapping write-ups get backwards, and the 4.22 book is blunt: "OpenShift Container Platform does not support passthrough termination for Gateway API", which "is only supported in traditional OpenShift Container Platform routes." Nothing OpenShift-side to migrate onto: it stays a Route until the workload leaves. The destination is upstream, where a TLS listener in Passthrough mode carries TLSRoute — "GA and ... part of the Standard Channel since v1.5.0", routing on the "Server Name Indication (SNI) hostname". Reencrypt goes wrong in both directions. destinationCACertificate is still a field on route.openshift.io/v1 Route in 4.22 with no counterpart on any Gateway API route object — "for upstream connections, BackendTLSPolicy is used", after termination rather than on the route. The book gives that job a mechanism: "you must combine listener edge termination with a BackendTLSPolicy CR attached to the service." Treating that as where a Route's backend CA lands is this article's comparison, not a published conversion. The timeout annotation is the mapping to resist writing. OpenShift's own 4.22 book documents two HTTPRoute clocks and names neither as its replacement: request "represents the duration of the entire request-response transaction", while backendRequest covers the gateway-to-backend leg and "is classified as an extended feature (Support: Extended)". Upstream labels the request timeout extended-support too. ```yaml # ── OPENSHIFT: the Route you have ────────────────────────────────────────── apiVersion: route.openshift.io/v1 kind: Route metadata: name: payments namespace: payments annotations: # No field for this in Ingress or in the Gateway API Standard channel - # only in an implementation's own policy CRD (Envoy Gateway's # BackendTrafficPolicy, Istio's DestinationRule), which trades one # vendor-specific surface for another rather than removing it. haproxy.router.openshift.io/balance: leastconn haproxy.router.openshift.io/timeout: 30s spec: host: payments.example.com to: kind: Service name: payments port: targetPort: https tls: termination: reencrypt # Optional: absent it, the router may supply its own destination CA. destinationCACertificate: | -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- --- # ── EDGE/TERMINATE, as an Ingress ────────────────────────────────────────── # An ALTERNATIVE to the Route above, NEVER both on this hostname - see Gate 4. # The kind is upstream; openshift-default is not. Replace it on the far side. apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: payments namespace: payments spec: ingressClassName: openshift-default tls: - hosts: ["payments.example.com"] secretName: payments-edge-tls rules: - host: payments.example.com http: paths: - path: / pathType: Prefix backend: service: name: payments port: name: https --- # ── EDGE/TERMINATE, as Gateway API ───────────────────────────────────────── # Nothing below routes until this GatewayClass exists and reports Accepted - # creating it installs the controller. controllerName must be exact. apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: openshift-default spec: controllerName: openshift.io/gateway-controller/v1 --- apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: external namespace: openshift-ingress spec: gatewayClassName: openshift-default listeners: - name: https protocol: HTTPS port: 443 hostname: payments.example.com tls: mode: Terminate certificateRefs: # Must ALREADY EXIST in openshift-ingress - the Gateway's own # namespace - or in one a ReferenceGrant permits. Not the same # object as the payments-namespace Secret the Ingress names above. - group: "" kind: Secret name: payments-edge-tls # OMIT THIS AND THE ROUTE BELOW NEVER ATTACHES: unset, namespaces.from # is implicitly Same, so a Gateway here accepts only openshift-ingress # routes. Do not use from: All - Red Hat warns that setting "can enable # hostname or domain hijacking". Label the namespace to match: # oc label namespace payments gateway-access=payments-edge allowedRoutes: namespaces: from: Selector selector: matchLabels: gateway-access: payments-edge --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: payments namespace: payments spec: parentRefs: # Cross-namespace, so name the Gateway's namespace. - name: external namespace: openshift-ingress sectionName: https hostnames: ["payments.example.com"] rules: - backendRefs: # The Service is assumed, not shown: it must expose port 443 and map # it to the pod's named https port (containerPort 8443 further down). - name: payments port: 443 # Two clocks, and the 4.22 book names NEITHER as the annotation's # replacement: request is the whole transaction, backendRequest the # gateway-to-backend leg (Support: Extended). The inner clock belongs # at or below the outer one. 30s here is the 4.22 book's own example # value - it is NOT transcribed from the 30s annotation above, and the # match is a coincidence you should not preserve. Re-decide both. timeouts: request: 30s backendRequest: 10s --- # ── REENCRYPT: THIS ARTICLE'S COMPARISON, NOT A PUBLISHED MIGRATION ──────── # destinationCACertificate is still a route.openshift.io/v1 field in 4.22 with # no counterpart on any Gateway API route object; the 4.22 book does the same # job with an edge-terminated listener PLUS this policy. apiVersion: gateway.networking.k8s.io/v1 kind: BackendTLSPolicy metadata: name: payments-backend namespace: payments # must equal the ConfigMap's namespace; a cross-namespace # ref with a ReferenceGrant is not supported here spec: targetRefs: - group: "" kind: Service name: payments validation: # Must match the HTTPRoute's hostname, not the short service name the # Route falls back to. It is also the backend SNI, so the pod certificate # must present it or list it under subjectAltNames - the 4.22 ingress # chapter caps that list at five values, each typed Hostname or URI, and # states the hostname is not itself evaluated as a SAN. hostname: payments.example.com caCertificateRefs: # Key MUST be ca.crt, up to eight PEMs. Without it the ref is invalid # and connections using it MUST fail with an HTTP 5xx. - group: "" kind: ConfigMap name: payments-backend-ca ``` Three edges belong in the plan, not mid-cutover. None of that Gateway API YAML does anything until the GatewayClass exists, and creating it is what makes the Ingress Operator install "a lightweight Istio control plane" in openshift-ingress. A listener without allowedRoutes accepts its own namespace only, so a cross-namespace route silently never attaches. And "experimental Gateway API CRDs in the gateway.networking.k8s.io group remain restricted". ## SecurityContextConstraints: the lock-in that made your images portable This gate decides whether a migration lands, and it is not SCC syntax but an asymmetry between two admission controllers. OpenShift's is mutating: it will "generate field values for security context settings that were not specified on the request", then validates. That is the whole problem: a pod spec can be incomplete on OpenShift and still run, because admission finishes it. Those values do not come from the manifest. Given "a RunAsUser strategy of MustRunAsRange with no minimum or maximum set", admission "looks for the openshift.io/sa.scc.uid-range annotation to populate range fields". The UID is a property of the project the pod landed in. Upstream will not, and says so: Pod Security Admission "is a non-mutating admission controller, meaning it won't modify pods before validating them", so you must "either modify your workloads... or use a Mutating Admission Webhook". It is coarser too — it "only supports 3 standard levels" — so a hand-written SCC becomes a webhook you own, not a namespace label. The consequence: a manifest that ran for years behaves differently after the move, with no line changed. Upstream's restricted standard forbids runAsUser of 0 but accepts "any non-zero value" or "undefined/null", so a spec leaving the UID unset passes admission with nothing filled in, and identity comes from the image. If that image resolves to UID 0 the failure lands at the kubelet: later, noisier, in a different log. > **Red Hat's 4.22 docs do not settle which SCC is the default** — The SCC chapter says restricted-v2 "is granted to all authenticated users by default" and "used in most cases". The table in that same chapter says restricted-v3 — which forces hostUsers: false — "is the most restrictive SCC provided by a new installation and will be used by default for authenticated users". The 4.22 postinstallation book does not list restricted-v3 at all and gives that role to restricted-v2. Nothing reconciles the three: read oc get scc and the pod's admitted-SCC annotation on your own cluster, not any article's word. Two more behaviours. SCC choice is not a request: "Setting privileged: true in the pod specification does not necessarily select the privileged SCC" — the highest-prioritised one the requester may use with allowPrivilegedContainer: true wins. And same-version clusters differ: restricted "is no longer available to users of new OpenShift Container Platform 4.11 or later installations" without an explicit grant. ```yaml # Target: a pod spec that states every field admission would otherwise INVENT # - allowPrivilegeEscalation, capabilities, seccomp, runAsNonRoot - and that is # already legal under upstream restricted. runAsUser is the field you cannot # finish this way: leave it unset and restricted-v2 still generates it from the # namespace range, which is precisely why the admitted UID is not portable. # Do this BEFORE you move anything. --- apiVersion: v1 kind: Namespace metadata: name: payments labels: # These do NOT choose an SCC. Pod Security Admission is a separate # validating controller that runs on OpenShift too, so it rejects pods # still failing restricted AFTER SCC mutation - in a test namespace, # which is the point of staging it here. # OpenShift syncs warn and audit (never enforce) to the most privileged # profile in the namespace. Per the 4.22 docs a label modified by hand from # its synced value DISABLES sync for that label, so the values below are # already sticky; the opt-out is belt-and-braces, not a prerequisite: # oc label namespace payments \ # security.openshift.io/scc.podSecurityLabelSync=false pod-security.kubernetes.io/enforce: restricted pod-security.kubernetes.io/enforce-version: latest pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/warn: restricted --- apiVersion: apps/v1 kind: Deployment metadata: name: payments namespace: payments spec: replicas: 3 selector: matchLabels: # DELIBERATELY NOT THE DEPLOYMENTCONFIG'S SELECTOR - though NOT because # the two controllers would fight over pods. They cannot: library-go's # MakeDeployment copies spec.selector into the ReplicationController's # selector and then adds deploymentconfig= and deployment=-, # which this Deployment's pods never carry, and both controllers claim # pods through metadata.ownerReferences, so neither adopts the other's. # The real cutover hazard is the SERVICE: one pointed at a label both # pod sets share load-balances across both while you are still testing. # A distinct label here lets the Service select exactly one set. Scaling # this Deployment to zero still does not clear the DeploymentConfig's # own pods - that is the DeploymentConfig's own replica count, not a # fight. Point the Service at this label at cutover. app: payments-next template: metadata: labels: app: payments-next spec: securityContext: runAsNonRoot: true seccompProfile: type: RuntimeDefault # runAsUser is deliberately ABSENT. Upstream fills nothing in, so # identity comes from the image's USER and the KUBELET fails a # container that would run as UID 0; OpenShift populates it from the # namespace's uid-range annotation. Pinning a UID is not portable # either: restricted-v2 rejects one outside the range, upstream # restricted takes any non-zero value. containers: - name: api # Digest-pinned: ImageStream tag resolution does not come with you. image: registry.example.com/payments/api@sha256:0000000000000000000000000000000000000000000000000000000000000000 ports: - name: https containerPort: 8443 securityContext: allowPrivilegeEscalation: false # restricted permits only false runAsNonRoot: true # Extra hardening, NOT a restricted requirement - and the reason # the emptyDir exists. Keep the pair or drop both. readOnlyRootFilesystem: true capabilities: drop: ["ALL"] # restricted-v2 allows only NET_BIND_SERVICE back seccompProfile: type: RuntimeDefault volumeMounts: - name: tmp mountPath: /tmp volumes: - name: tmp emptyDir: {} ``` What this buys even if you never leave: a security posture readable in Git rather than inferred from a namespace annotation — a policy-as-code win before a portability one. Neither project publishes a field-by-field SCC-to-Pod-Security-Standards table, and no honest article invents it. ## BuildConfigs, S2I and ImageStreams: which build objects have an upstream twin "A BuildConfig object is the definition of the entire build process", built by one of "three primary build strategies... Docker build, Source-to-image (S2I) build, Custom build". It has no upstream equivalent kind, and neither does ImageStream. Worse than the Route position: no replacement object to map fields onto, only a different way of building software. S2I has the genuine upstream analogue most comparisons miss. Shipwright publishes a source-to-image ClusterBuildStrategy — but its platform column reads "linux/amd64 only", unlike buildah, kaniko and ko. S2I "supports incremental builds, which re-use previously downloaded dependencies, previously built artifacts", and no document says how much survives the port. Red Hat's own forward path is a product: "Builds is a Shipwright-based framework for building container images on OpenShift Container Platform" — currently 1.9, announced "on OpenShift Container Platform 4.18 and later" though the same notes' matrix caps it at 4.18-4.22, with 1.8 at 4.17-4.21. A product that runs on OpenShift. > **Crane is not an exit, and is not a supported product** — The Builds 1.7 release note does advertise "a new migration tool, Crane" that "automatically converts the BuildConfig resources to Build resources". The developer article it links states the limits: "available as an upstream community capability in Crane", "not yet included in official Red Hat product offerings" and "not supported for production use." What it converts into is "Builds for Red Hat OpenShift" — which runs on OpenShift. Sideways, not out. Two paths off BuildConfig end on a conformant cluster: rebuild on upstream Shipwright or Tekton, or move builds off-cluster into whatever CI exists. Neither has a Red Hat conversion tool behind it — a cost with a long-game dividend, because a build that runs in CI runs anywhere, the same argument as owning your forge rather than renting it. ImageStream deserves credit before you drop it: a tag "contains a history of images represented as a stack of all images the tag ever pointed to", which "allows for easy rollbacks". A plain registry tag gives none of that; the substitute is digest pinning plus what your deployment system records. ## Version skew and the support window you are really buying Here is the case for paying, as strongly as the case against. As of October 2026 upstream "maintains release branches for the most recent three minor releases (1.37, 1.36, 1.35)", the current being 1.37.1. OpenShift Container Platform 4.22 ships Kubernetes 1.35 — two minors behind. On the day of writing 4.23 had no GA release; its engineering candidate 4.23.0-ec.1 already lists "Kubernetes 1.36.3" — the two-minor gap belongs to the 4.22 GA line priced here, not the 4.23 stream. What the gap buys is time, measured against upstream's own figure: patch series are supported "for a period of roughly fourteen (14) months", then EOL. Not long for a regulated estate with a change-freeze calendar. Read the ladder off the life-cycle policy itself — summaries stop a rung early. - 18 months, every minor — Maintenance Support "ends at 18 months after GA". No add-on, no qualifier. For an odd minor that is the whole ladder. - 24 months, EUS only — plus Term 1, "an optional six months ... for all EUS denoted releases", "included with all Premium subscriptions and available as an optional add-on to Standard subscriptions". Red Hat "denotes even numbered minor releases" as EUS, so 4.22 qualifies and 4.21 never did. - 36 months, EUS only — plus Term 2, "an optional twelve months" more, commencing at the end of Term 1. Most summaries stop here; it is not the ceiling. - 48 months, EUS only — plus Term 3, a further "optional twelve months" sold only as an add-on, which Red Hat says lets customers "remain with the same release of Red Hat OpenShift for a total of 48 months". The 4.22 notes do not mention Term 3. Every rung above 18 months is EUS, so planning the estate onto even minors is a standing decision. Footnote 11 says "The OCP 4.12 Extended Update Support Add-on Term 1, Term 2, and Term 3 are only available on the x86_64 architecture", then that "Commencing with OCP 4.14 the Extended Update Support Add-on Term 1 and Term 2 includes support across all supported architectures". The second sentence names Term 1 and Term 2, not Term 3 — do not assume a 48-month window on aarch64, ppc64le or s390x. Forty-eight months against roughly fourteen is a real reason to pay. The subscription cannot be priced from public sources: Red Hat posts no list price for self-managed OpenShift Container Platform. The publicly posted cards are managed-service — AWS's ROSA page states fees accrue "at an hourly rate of $0.171 per 4 vCPU used by worker nodes", plus "a $0.25 per hour cluster fee" for hosted control planes. Fees on one cloud, no proxy for self-managed. Price this exit with the method from a general exit-cost model and your own contract. ## What an auditor sees on each side of the move Migration day changes the evidence, not just the manifests. SCC bindings are clearest: restricted-v2 granted to authenticated users is a platform-maintained statement that pods cannot escalate privilege: "if you delete a default SCC, it is regenerated by the Cluster Version Operator." The far-side equivalent is a namespace label plus a webhook you wrote — weaker until test evidence stands behind it. The second artefact is newer. "Starting in 4.22... the operators that manage cluster DNS and cluster Ingress automatically install and maintain default deny-all NetworkPolicy objects in their respective namespaces" — and "any unmanaged or custom pods running in these namespaces will have their network traffic blocked." The same notes add "a new NetworkPolicy parameter for the openshift-cluster-version namespace" that "denies all ingress and egress traffic for pods that are not host-networked". Vendor-delivered default-deny on the platform's control paths. > **Discover those namespaces; do not copy a list** — The 4.22 release notes do not enumerate them, pointing at oc get networkpolicies --all-namespaces instead. Red Hat's OpenShift Dedicated book names four for the same change (openshift-dns, openshift-dns-operator, openshift-ingress, openshift-ingress-operator) — a different product, so check your own cluster rather than assume it matches. Replicating that on vanilla is ordinary work — default-deny per namespace with targeted allow rules — but it must be standing *before* cutover. The audit question is whether the control was ever absent. ## The exit, staged: no flag day required Nothing above argues for a cutover weekend. Every step runs on the cluster you already have, in dependency order, each keeping a rollback that is not a restore — provided the replacement is built so the OpenShift object it supersedes cannot fight it. [Figure: A left-to-right staged migration from OpenShift Container Platform 4.22 to a vanilla conformant cluster through four gates: DeploymentConfig to Deployment, then ImageStream and BuildConfig, then SCC to Pod Security Admission plus webhooks, then Route last as the public-traffic cutover, scoped to HTTPRoute because OpenShift Container Platform 4.22 serves only the standard-channel Gateway API v1.4.1 and so has no TLSRoute kind for a passthrough Route to be staged on. Each gate carries what is done, a dashed abort criterion and a rollback that is a scale-to-zero or a DNS change. OKD hangs off the start node on a dashed edge as a parallel destination that keeps the whole dialect with no subscription, labelled with the absence of any published OpenShift-to-OKD upgrade edge.] 1. DeploymentConfig to Deployment. Cheapest, and already recommended — but give the new Deployment its own pod label. The controllers cannot fight over pods: the ReplicationController a DeploymentConfig creates adds deploymentconfig= and deployment=- to its selector, and both claim pods through ownerReferences. The Service is the hazard — pointed at a label both pod sets share, it load-balances across both mid-test. Scaling the Deployment to zero still leaves the DeploymentConfig's pods. Abort if a workload needs a DeploymentConfig-only capability. 2. ImageStream and BuildConfig. Move builds to Shipwright, Tekton or off-cluster, pin deployments to digests, export the tag histories you would want mid-incident. Abort if assemble behaviour or incremental caching does not reproduce. 3. SCC to Pod Security Admission plus webhooks. The highest-leverage gate, done entirely on OpenShift: state every field admission would otherwise invent, then stand up the namespace labels and webhooks that replace it. runAsUser is the exception — leave it unset and restricted-v2 still generates it, so never treat the admitted UID as portable. Abort if an image still resolves to UID 0 — that failure lands at the kubelet. 4. Route last, and not as one step. An HTTPRoute on a Gateway with its own address can run beside the live Route, so DNS is both the cutover and the rollback — but not for passthrough: 4.22 vendors only standard-channel Gateway API v1.4.1, and TLSRoute reached Standard in v1.5.0, so that kind is absent, and a passthrough Route has no parallel stage: it stays a Route until the workload lands upstream. An Ingress cannot stage either: it is admitted by generating a Route, and the router keeps routes unique per host *and* path only inside one namespace — across namespaces the default policy rejects on the host alone, so staging next door fails too. Same path as the live Route and the generated one is rejected HostAlreadyClaimed; a different path — the Route above sets none, the Ingress rule sets / — and both are admitted on one hostname, with no error saying which serves a request. Validate on another hostname, or delete the Route at cutover and accept that the rollback is recreating it. Abort if an annotation has no tested replacement; the balance algorithm has no portable field at all. OKD is a parallel destination, not a step on this path. It keeps the entire dialect with no Red Hat subscription — legitimate if the objection is the bill, not the API — and sits a Kubernetes minor *ahead* of the paid line. But no upgrade edge into it is published: OKD's documented in-place update to 5.0 starts from an OKD 4.x release. Reaching it means a fresh install plus a workload move, with a Kubernetes 1.35-to-1.36 jump riding along. ```yaml # THREE SEPARATE FILES, shown in one listing. The --- lines are file # boundaries, not documents in a single apply or one kustomize build. # # base/kustomization.yaml — UPSTREAM ONLY. A file here naming an openshift.io # API group is the bug; CI should fail the build on it. apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - namespace.yaml - deployment.yaml - service.yaml - networkpolicy-default-deny.yaml --- # overlays/openshift/kustomization.yaml — every openshift.io object lives # under here, so the dialect footprint is a readable file count. apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: payments resources: - ../../base - route.yaml # route.openshift.io/v1 components: - ../../components/scc-uid-pinning --- # overlays/vanilla/kustomization.yaml — buildable TODAY, long before anyone # decides to leave: the exit plan is exercised by CI, not written under fire. apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: payments resources: - ../../base - gateway.yaml # gateway.networking.k8s.io/v1 - httproute.yaml - backendtlspolicy.yaml patches: - path: psa-labels.yaml target: kind: Namespace name: payments ``` ## The long game: buy the distribution, do not write in the dialect The conclusion is not that OpenShift is a trap. A platform handing you default-deny networking on its own control paths, admission that makes third-party images behave, and a documented 48 months on an EUS release is selling something real. The mistake is letting the purchase write itself into every manifest. So make it a standing engineering rule, not a migration project: every openshift.io object lives in one overlay, no workload depends on admission completing its pod spec, images carry a non-root USER, deployments pin digests rather than ImageStream tags. None of that is an exit — it is a cleaner way to run the platform you pay for, and the exit stays cheap as a side effect. Then the footprint is a number you can report at any time — the only optionality that survives a renewal negotiation. A team that says "forty-one Routes, two hand-written SCCs, no BuildConfigs, overlay builds green" is negotiating; one that says "it is all Kubernetes, probably fine" is guessing. The same discipline runs below Kubernetes — an immutable node OS, bare-metal ingress on Gateway API — and above it, where KubeVirt decides whether your virtual machines join this conversation. Measure the dialect quarterly, keep the vanilla overlay green in CI, and the distribution stops being a commitment and becomes a supplier. ### FAQ Q: Is OpenShift the same as Kubernetes? A: OpenShift is Kubernetes plus a dialect. Red Hat states that "OpenShift Container Platform 4.22 release uses Kubernetes 1.35 with CRI-O runtime", and every upstream object in it behaves as the upstream API documents. What is additional is five OpenShift-only API groups — route.openshift.io (Route), security.openshift.io (SecurityContextConstraints), build.openshift.io (BuildConfig), image.openshift.io (ImageStream) and apps.openshift.io (DeploymentConfig, deprecated since 4.14 and receiving "only security-related and critical" fixes) — plus the admission behaviour around them. The practical test is your own manifests: anything whose apiVersion ends in openshift.io will not resolve on upstream Kubernetes, and everything else will. OKD is the exception that proves the point — it serves the same dialect, so it is not an exit from it. Q: What is the hardest part of migrating from OpenShift to vanilla Kubernetes? A: Not Routes — SecurityContextConstraints, because the failure is silent. OpenShift's SCC admission is mutating: its documented algorithm is to "retrieve all SCCs available for use", then "generate field values for security context settings that were not specified on the request", then "validate the final settings". Upstream Pod Security Admission does the opposite and says so: it "is a non-mutating admission controller, meaning it won't modify pods before validating them", so "you will need to either modify your workloads to meet the Pod Security constraints, or use a Mutating Admission Webhook to make those changes". A manifest that has run for years on OpenShift can behave differently on vanilla without a line changing — for example a pod leaving runAsUser unset, which OpenShift populates from the namespace's openshift.io/sa.scc.uid-range annotation while upstream fills nothing in and lets the image's own USER decide. Pod Security Admission is also coarser: it "only supports 3 standard levels", so a hand-written SCC becomes a validating admission webhook rather than a namespace label. Q: What replaces an OpenShift Route on vanilla Kubernetes? A: It depends on the termination mode, and one case has no field-level answer. Edge or terminate maps to an Ingress or to an HTTPRoute on an HTTPS Terminate listener. Passthrough maps to a TLSRoute on a TLS listener in Passthrough mode — upstream states that "the TLSRoute resource is GA and has been part of the Standard Channel since v1.5.0" and that it routes on "the Server Name Indication (SNI) hostname provided during the TLS handshake". Reencrypt is the awkward one: destinationCACertificate is still a field on route.openshift.io/v1 Route in 4.22 and has no counterpart field on HTTPRoute or TLSRoute, because Gateway API does the job with a different mechanism. OpenShift's own 4.22 ingress book says that "to use re-encrypt termination in Gateway API, you must combine listener edge termination with a BackendTLSPolicy CR attached to the service". Treating that policy as where a Route's backend CA ends up is a comparison between two APIs, not a migration Red Hat publishes. Separately, the haproxy.router.openshift.io/balance annotation has no counterpart in Ingress or in the Gateway API Standard channel — only in an implementation's own policy CRD, such as Envoy Gateway's BackendTrafficPolicy or Istio's DestinationRule. Q: Is OKD a way out of OpenShift lock-in? A: It removes the subscription, not the dialect, and it is a separate cluster rather than an upgrade. OKD keeps Routes, SCCs, BuildConfigs and ImageStreams intact, and runs CentOS Stream CoreOS rather than RHEL CoreOS. Oddly, it is also a Kubernetes minor ahead of the paid line: build 5.0.0-okd-scos.0, described by its 17 September 2026 release notes, pins "Kubernetes 1.36.3: Upgraded from Kubernetes 1.35.6", and 5.0.0-okd-scos.1 of 30 September 2026 pins Kubernetes 1.36.4 — against OpenShift Container Platform 4.22's Kubernetes 1.35 and an upstream current minor of 1.37.1 as of October 2026. Those pins belong to each build, not to "OKD 5.0" as a standing property. The important operational limit is that no OpenShift-to-OKD upgrade edge is published: the OKD page states that "Updating an OKD cluster from a 4.x release to version 5.0 is an in-place update", on the stable-5.0 and candidate-5.0 channels — it does not offer an edge from OpenShift Container Platform. Reaching OKD from OpenShift means a fresh install plus a workload move, with a 1.35-to-1.36 Kubernetes jump on top — not a retargeted Cluster Version Operator. Q: How long is an OpenShift release supported, and is that worth the lock-in? A: Longer than upstream, but only on the right minors. Maintenance Support "ends at 18 months after GA" and that applies to every minor — for an odd-numbered one it is the end of the line. The three Extended Update Support Add-On terms above it are all scoped to "EUS denoted releases", and Red Hat "denotes even numbered minor releases (eg 4.12, 4.14, 4.16) as Extended Update Support (EUS) releases", so 4.22 is eligible and 4.21 is not. Term 1 adds "an optional six months" and "is included with all Premium subscriptions and available as an optional add-on to Standard subscriptions", reaching the 24 months the 4.22 release notes state for even-numbered releases. Term 2 adds "an optional twelve months" more, to 36 — where most summaries stop. Term 3 adds a further "optional twelve months", which Red Hat says "enables customers to remain with the same release of Red Hat OpenShift for a total of 48 months". Upstream, by contrast, supports "active patch release series for a period of roughly fourteen (14) months". One caveat on architecture: footnote 11 says OCP 4.12's "Term 1, Term 2, and Term 3 are only available on the x86_64 architecture", and that "Commencing with OCP 4.14 the Extended Update Support Add-on Term 1 and Term 2 includes support across all supported architectures" — a sentence that does not name Term 3, so a 48-month all-architecture window is not something to assume. --- # Docker vs LXC: Application Container or Whole Machine URL: https://stribog.com/blog/lxc-incus-docker-system-versus-application-containers Published: 2026-10-02 Category: Runtime Keywords: docker vs lxc, lxc vs docker, system containers versus application containers, incus system container, proxmox lxc container, lxc restart migration downtime, run oci images in incus, unprivileged container user namespace Docker packages one application; LXC and Incus package a running Linux system. Compare PID 1, state, migration, audit scope and exit paths. > **The short answer** — Docker vs LXC is not a performance comparison; the two package different units. Docker's unit is an application image whose entrypoint becomes PID 1 and whose writable layer dies with the container. LXC's unit — driven by Incus or Proxmox VE — is a Linux system running /sbin/init, with a package manager and a root filesystem you keep. Decide it on what you must operate afterwards: who is PID 1, what survives a restart, and what relocating a running workload costs. Bare-metal consolidation projects reach this fork in the first week, usually by accident. Someone installs Docker on the new host because that is the container they know, then runs a database, a mail relay and a monitoring stack — each of which expects a machine — on a model built for stateless replicas. ## One packages an application, the other packages a machine Incus states the split without hedging. System containers "run full Linux distributions using a shared kernel" and are "very similar to a virtual machine but sharing kernel with the host system". Application containers "run a single application through a pre-built image" — a model it credits to "the likes of Docker and Kubernetes" — arriving "pre-installed and mostly pre-configured" rather than as "a pristine Linux environment". LXC itself is not inherently the system-container tool: it is "a userspace interface for the Linux kernel containment features" that can "create and manage system or application containers". The manager on top decides which. Proxmox VE "uses Linux Containers (LXC) as its underlying container technology", abstracted behind pct; Incus is the other manager, and the two no longer gain features in the same order. Docker's side is a documented convention, not a kernel constraint, and comparisons state it backwards. Its build guidance says "Each container should have only one concern" — then qualifies that "Limiting each container to one process is a good rule of thumb, but it's not a hard and fast rule." Nothing stops you running a full init under Docker; the question is whether the images and tooling around it expect one. One constraint binds both columns equally, worth stating before anyone sells shared-kernel containers as a substitute for virtual machines: "Only Linux distributions can be run in Proxmox Containers. It is not possible to run other operating systems like, for example, FreeBSD or Microsoft Windows inside a container." [Figure: Two container stacks on one shared host kernel: the system container with /sbin/init as PID 1, supervised services, a package manager, a root filesystem it keeps, LXCFS over /proc and an unprivileged uid-0 mapping; the application container with the image entrypoint as PID 1, optional --init, immutable layers, a throwaway writable layer and a named volume as the only durable part. Namespaces, cgroups, seccomp/AppArmor and user namespaces are the shared substrate, and the Linux-only limit applies to both.] ## PID 1 is the whole argument Everything downstream follows from which process gets PID 1. In a Proxmox container the entrypoint option is documented as "Command to run as init", defaulting to /sbin/init. That is the whole proposition in one line: the distribution's own init reads unit files, runs timers, supervises services, reaps orphans and writes a journal — inside the container, with no sidecar and no supervisor shim in the image. Docker's answer is deliberately smaller. The --init flag "inserts a tiny init-process into the container as the main process, and handles reaping of all processes when the container exits", which Docker calls "superior to using a full-fledged init process such as sysvinit or systemd". A reaper, not a service manager, because the application container is not meant to need one. The interesting case is where the models meet. Building a Proxmox container from an OCI image, restore_oci_archive concatenates the image's OCI Entrypoint and Cmd. Only if that list is non-empty and its first element is not /sbin/init does the override fire — an image whose command already is /sbin/init keeps the default. When it fires, one branch does three things: writes the command into entrypoint, sets cmode to console, and auto-enables host-managed=1 on every netN device not already setting it. That third effect is the one that bites. In pct's terms host-managed means "the host (rather than the container) is responsible for the interface's IP configuration" and "The container should not run its own DHCP client or network manager on this interface". Correct for a hello-world image; a surprise if you assumed the container owned its networking. Regardless of whether that conditional fires, the same function applies the image's User to lxc.init.uid/gid/groups, its WorkingDir to lxc.init.cwd, its Env to the environment and its StopSignal to lxc.signal.halt. The entrypoint branch is conditional; those are not. ```bash #!/usr/bin/env bash set -euo pipefail incus launch images:debian/13 app-01 --storage tank # PID 1 is /sbin/init, not the workload. incus exec app-01 -- ps -p 1 -o pid,comm,args --no-headers # The distribution's own service manager supervises inside the container. incus exec app-01 -- systemctl list-units --type=service --state=running --no-pager # Install and enable a service the ordinary way. Nothing is rebuilt as an image. incus exec app-01 -- apt-get update incus exec app-01 -- apt-get install -y postgresql incus exec app-01 -- systemctl enable --now postgresql # Resource limits are applied live against the running instance. incus config set app-01 limits.memory=8GiB incus config set app-01 limits.cpu=4 incus config show app-01 | grep -E 'limits\.(memory|cpu)' ``` ## State: a throwaway layer versus a root filesystem you keep Docker's storage documentation is explicit about where the line falls: "A volume's contents exist outside the lifecycle of a given container. When a container is destroyed, the writable layer is destroyed with it." The writable layer is scratch space in a convincing disguise, and the failure mode is quiet — it works perfectly until the first docker rm. A system container inverts the default. The root filesystem is the durable object: a disk that survives restarts and upgrades, with apt writing into it. A relief, because the operational model is the one your team already has — and a liability, because you have acquired another distribution to patch. Incus refuses to guess where that filesystem lives: "There is no concept of a default storage pool in Incus. When you create a storage volume, you must specify the storage pool to use." The pool comes from the instance's configuration, set three ways — incus launch --storage , a profile whose root disk device names the pool, or the default profile. Proxmox splits state the same way: "A container can have multiple mount points. This makes it possible to use the best suited storage for each application." Backups follow the storage — "The vzdump backup tool can use snapshots to provide consistent container backups." If you build the images rather than consume them, owning the base layer is this argument one level down. ```yaml # incus profile edit app-tier # # cluster.evacuate=refresh-migrate requires Incus 7.4 or a later feature # release. It is not present in the Incus 7.0 LTS branch. name: app-tier description: System-container tier - pinned pool, explicit limits, refresh evacuation config: limits.memory: 8GiB limits.cpu: "4" # Near-live evacuation. Documented as supported only on ZFS or btrfs, and # only for containers: other storage drivers have no efficient snapshot # transfer mechanism for remote transfers. cluster.evacuate: refresh-migrate devices: root: type: disk path: / # Incus has no default storage pool. Name it, or instance creation has # nothing to place the root filesystem on. pool: tank eth0: type: nic network: incusbr0 name: eth0 ``` ## Relocating a workload: migration, daemon continuity, and the /proc question Proxmox VE states its limit in one sentence: "Running containers cannot live-migrated due to technical limitations. You can do a restart migration, which shuts down, moves and then starts a container again on the target node." Its figure — "normally only ... a downtime of some hundreds of milliseconds" — characterises container restart, not your workload; a Postgres container with a long shutdown will not match it. Two timeouts sit underneath, routinely merged. A restart migration "will shut down the Container and kill it after the specified timeout (the default is 180 seconds)"; the per-container Shutdown timeout is a different setting, defaulting to 60. Proxmox publishes its own escape hatch, routinely misattributed, so quote it exactly: "for use cases demanding maximum isolation and the ability to live-migrate, nesting containers inside a Proxmox QEMU VM remains a recommended practice." Incus has moved further. Its howto still says CRIU is not the answer — "only very basic containers (non-systemd containers without a network device) can be migrated reliably", excluding every system container — and recommends you "stop the container, move it over and then start it again". But Incus 7.4 added near-live migration: "the container's filesystem is transferred to the target server through a series of incremental snapshots while the container keeps running. Only the last incremental transfer requires the container to be stopped." That is incus move c1 --target incus02 --stateless --refresh. Incus virtual machines remain the exception, moving "without any downtime" once migration.stateful is enabled. > **Two caveats, both load-bearing** — The release notes bound it: "only supported on ZFS or btrfs and only for containers as other storage drivers don't have an efficient snapshot transfer mechanism." And it is feature-branch behaviour — Incus 7.4 and later — not something the 7.0 LTS branch carries. Standardise on the LTS for its 2031 horizon and you are standardising on stop-move-start. Docker's two relevant mechanisms are both same-host, and both routinely overstated. Live restore "helps reduce container downtime due to daemon crashes, planned outages, or upgrades", against a default where "when the Docker daemon terminates, it shuts down running containers". Four bounds: "patch releases (YY.MM.x), not ... major (YY.MM) daemon upgrades"; unchanged daemon options "such as bridge IP addresses and graph driver"; "only ... standalone containers, and not ... Swarm services"; and not Windows containers. Checkpoint and restore is the other: "This command is experimental", and "currently focused on single-host use cases". Its own page meets the cross-host question and declines it — moving a server to another machine "is possible with the current implementation, but not currently a priority". Not a claim it cannot cross hosts; a statement that nobody maintains the path. So the node-drain answer stays what it was: stop, reschedule, start, done by the scheduler above the runtime. [Figure: Two panels. Left: relocating an instance to another host under a system-container manager, branching on instance type and storage driver into Proxmox restart migration with its 180-second migration timeout, QEMU nesting where live migration is required, Incus near-live refresh migration limited to ZFS or btrfs containers on 7.4 and later, Incus stateful VM migration, and stop-move-start. Right: Docker on a single host, with live restore and its four bounds, CRIU checkpoint and restore inside the host boundary marked experimental and single-host, and stop-reschedule-start as the node-drain answer.] Operators notice on day one that /proc inside a system container describes the container rather than the host — a component, not a property of LXC. LXCFS makes /proc/cpuinfo, /proc/meminfo, /proc/stat and /proc/uptime "container aware such that the values displayed ... really reflect how long the container is running and not how long the host is running", and Proxmox ships it. Its README is equally clear that it "is useable by any runtime" — evidence of what LXCFS does, not of what Docker shows. ```bash #!/usr/bin/env bash set -euo pipefail CTID=201 TARGET_NODE=pve-02 # Unprivileged is the default when creating a container; pass it explicitly so # the intent is in the command, not in a default that differs from the schema. pct create "$CTID" local:vztmpl/debian-13-standard_13.0-1_amd64.tar.zst \ --hostname app-01 \ --unprivileged 1 \ --memory 8192 \ --cores 4 \ --rootfs local-zfs:16 \ --mp0 local-zfs:200,mp=/var/lib/postgresql,backup=1 \ --net0 name=eth0,bridge=vmbr0,ip=dhcp pct start "$CTID" # Restart migration is the only migration Proxmox VE offers for a RUNNING # container (a stopped one migrates offline with plain 'pct migrate'): it shuts # the container down, moves it, and starts it on the target node. Measure the # outage your service actually takes. start_ns=$(date +%s%N) pct migrate "$CTID" "$TARGET_NODE" --restart --timeout 180 end_ns=$(date +%s%N) echo "restart migration wall clock: $(( (end_ns - start_ns) / 1000000 )) ms" ``` ```ini # /etc/pve/lxc/201.conf - excerpt, site-specific lines elided arch: amd64 cores: 4 hostname: app-01 memory: 8192 ostype: debian rootfs: local-zfs:subvol-201-disk-0,size=16G mp0: local-zfs:subvol-201-disk-1,mp=/var/lib/postgresql,backup=1,size=200G net0: name=eth0,bridge=vmbr0,ip=dhcp,type=veth unprivileged: 1 ``` ## Security and audit scope: unprivileged by default, rootless by choice The default isolation boundary differs, and so does what upstream treats as a bug. Proxmox creates unprivileged containers by default: "The root UID 0 inside the container is mapped to an unprivileged user outside the container", and "This is the default option when creating a new container." One caveat for anyone reading the schema rather than the chapter — the unprivileged key is documented with a default of 0, with creation defaulting to 1. LXC upstream states the position plainly in both directions: "Unprivileged containers are safe by design. The container uid 0 is mapped to an unprivileged user outside of the container." And for the other kind: "those containers aren't and cannot be root-safe." The page pairs the first claim with staying current on kernel issues. > "As privileged containers are considered unsafe, we typically will not consider new container escape exploits to be security issues worthy of a CVE and quick fix. We will however try to mitigate those issues so that accidental damage to the host is prevented." — LXC upstream security policy, linuxcontainers.org/lxc/security/ That disclosure policy comes with the choice. If vulnerability management runs off CVE feeds, a privileged LXC container is a blind spot by design rather than oversight. Docker's nearest equivalent is rootless mode, and it is opt-in: it "lets you run the Docker daemon and containers as a non-root user to mitigate potential vulnerabilities in the daemon and the container runtime". Note what it moves — the daemon as well as the containers. A Proxmox unprivileged container is not the same control: it sits under a root-owned host stack. Then there is the scope of the guidance you will be measured against. NIST SP 800-190, published 25 September 2017, defines its subject as "Application container technologies ... a form of operating system virtualization combined with application software packaging." Section 2 separates the other model — "OS containers are different from the application containers that are the topic of this guide" — and its scope line puts all other virtualization outside the document. Read that for what it says rather than what would be convenient. It does not put a system-container estate outside audit scope, and nothing on NIST's page says auditors arrive holding it as a checklist. It means the most widely cited container-security publication does not, on its own, cover the model you would run — so the control set comes from the host-hardening guidance you already apply, and expect to explain that mapping. The adjacent regulatory thread runs through the Cyber Resilience Act. ## The boundary already leaks: Incus and Proxmox now run OCI images The clean story — Docker for applications, LXC for machines — stopped being true in one direction. Incus consumes OCI registries as image sources: add one with incus remote add --protocol=oci, Docker Hub being the documented example, and launch straight from it. Proxmox VE arrived a release later. Version 9.1, released 19 November 2025, added the ability to "Create LXC containers from OCI images", which "allows to create full system containers from suitable OCI images", with application containers "also available (technology preview)". The pct chapter adds the detail that matters: "A container created from an OCI image still uses the existing LXC framework." The traffic runs one way. There is no documented Docker feature for running a system container, and --init is not it. That asymmetry, rather than any benchmark, is the difference between the two ecosystems in 2026. What the OCI path gives you in Incus is a small, fixed set of instance options: oci.cwd, oci.dns.domain, oci.dns.nameservers, oci.dns.search, oci.entrypoint, oci.gid and oci.uid. There is no OCI network key — a smaller fact than it looks, because ordinary NIC devices still apply. Read it as what the image configuration can override, not the instance's whole capability. The practical value is narrower than "Docker is obsolete" and more useful: a consolidation project need not pick one unit for the whole estate on day one. That is optionality designed in rather than hoped for — the property that lets WebAssembly runtimes sit beside OCI containers on a shared scheduler. ```bash #!/usr/bin/env bash set -euo pipefail # Add an OCI registry as an Incus remote, then launch straight from it. incus remote add oci-docker https://docker.io --protocol=oci incus launch oci-docker:hello-world --ephemeral --console # A longer-lived one. These are the OCI-conditioned instance keys Incus # documents; there is no oci.* network key, so networking is an ordinary NIC # device like any other instance. incus launch oci-docker:nginx web-01 --storage tank incus config set web-01 oci.entrypoint='/docker-entrypoint.sh nginx -g "daemon off;"' incus config set web-01 oci.cwd=/ incus config set web-01 oci.uid=0 incus config set web-01 oci.gid=0 incus config set web-01 oci.dns.nameservers=192.0.2.53 incus config device add web-01 eth0 nic network=incusbr0 name=eth0 # Every oci.* key above is documented "Live update: no", unlike the limits.* # keys earlier. Setting them on a running instance is accepted and does nothing # until it restarts. incus restart web-01 ``` ```yaml # compose.yaml services: web: image: nginx:1.29 # Inserts the tiny init process. A reaper, not a service manager. init: true restart: unless-stopped ports: - "8080:80" volumes: # The only durable part: the writable layer is destroyed with the # container. - web-content:/usr/share/nginx/html volumes: web-content: ``` ## Exit ramps in both directions, and who owns the manager Exits here are cheap, because the artefacts are ordinary: a system container's root filesystem is a directory tree, and an OCI image is an OCI image. The risk sits a level up, in who owns the manager you standardise on. Canonical confirmed in July 2023 that "Canonical is now the main maintainer and the infrastructure is hosted by Canonical." Incus "started as a community fork of Canonical's LXD following Canonical's takeover", is "free of any CLA and remains released under the Apache 2.0 license", and is "maintained by the same team of developers that first created LXD". Both are real options; only one of them cannot change its terms on you. The licence position has edges, and the slogan version gets it wrong in both directions. canonical/lxd's COPYING is the "GNU AFFERO GENERAL PUBLIC LICENSE Version 3", and its contributing guide requires that "All contributors must sign the Canonical contributor license agreement (CCLA)", licensing contributions "under the project's AGPL-3.0-only license" by default — while recording that "Some files and commits are licensed under Apache-2.0". Canonical's FAQ pins the date — "Community contributions remain under Apache-2.0 - this refers to all contributions made before December 12th, 2023" — and keeps "the Go SDK client package ... under Apache-2.0". A mixed tree with a CLA on top. The image server made the split operational, with a phase-out ending "2024/05/01 | All LXD users will lose access" and a carve-out: "This does not affect any existing instances you may have running, nor ... anyone exclusively using official Ubuntu images (ubuntu: and ubuntu-daily: remotes)." The maintainer confirmed that "Incus and LXC both have access to all the images on the image server." Ubuntu's seeding explains why LXD is still everywhere regardless. The 22.04 (jammy) server seed still carries snap:lxd=5.0/stable/ubuntu-22.04; the 24.04 (noble) seed carries lxd-installer in the same position, and the Launchpad bug ties the change to the licence: "With LXD 5.20 there is a license change to AGPL and it has been decided to no longer seed the snap in Ubuntu 24.04 and later." That package is a shim — "Wrapper to install lxd snap on demand" — whose lxc script asks "Would you like to install LXD snap now (Y/n)?" when the snap is missing and a terminal is attached. So 22.04 estates still get LXD from the seed; newer ones get a prompt. None of that makes LXD a bad tool. It makes the governance question concrete: who can change the terms underneath you, and whether a fork exists you could move to — the same calculation the OpenTofu fork forced one layer up. ## The long game: what you are signing up to operate for a decade Both projects publish horizons that outlast a procurement cycle, somewhere their own documentation has not caught up with. LXC 7.0 LTS was released on 30 April 2026 and "will be supported until June 2031"; the same announcement moves the previous LTS down a gear — "LXC 6.0 will now switch to a slower maintenance pace, only getting critical bugfixes and security updates". The introduction page still lists only 6.0 and 5.0. Incus is the same shape. The 7.0.1 LTS announcement of 10 July 2026 states "This is the first bugfix release for Incus 7.0 which is supported until June 2031", while the support page still describes the five-year schedule for 6.0 — "supported until June 2029". Take the horizon from the release announcement, the support shape from the support page. LXC 7.0 sets a floor you inherit on upgrade: it "removes support for: CGroupV1, Kernels without PIDFD support, Kernels without the new mount API". On Proxmox VE 9 the cgroup part is already behind you — that chapter records v1 as removed entirely in 9.0 — but PIDFD and the mount API are a newer floor, binding the hosts rather than the containers. Which leaves the cost nobody budgets. The system container means owning a distribution's lifecycle inside every container: a package set, a patch cadence, a release upgrade, multiplied by instance count. Not an argument against it — it is the ownership you already carry on every virtual machine, and the ownership that makes the exit cheap. It is an argument for a tier split rather than a doctrine, with the consolidation arithmetic applied to operating systems rather than clusters. The decade question is not which unit wins — both will still be here. It is whether the unit you picked leaves you able to change your mind. On that axis the system container does unusually well: what you hold is a root filesystem and a configuration file, which nobody can revoke or relicense. If the workloads forcing this choice are full virtual machines, that is a different fork, and a consolidation onto Proxmox VE without Kubernetes is how most estates reach it. ### FAQ Q: What is the difference between LXC and Docker? A: They package different units. Incus documents system containers as running "full Linux distributions using a shared kernel", "very similar to a virtual machine but sharing kernel with the host system", while application containers "run a single application through a pre-built image" — a model it credits to "the likes of Docker and Kubernetes". The practical consequences follow from PID 1: in a Proxmox container the entrypoint option defaults to /sbin/init and is documented as the "Command to run as init", so the distribution's own service manager supervises inside the container. Docker's equivalent is deliberately smaller — the --init flag "inserts a tiny init-process into the container" for reaping, which Docker's own docs call "superior to using a full-fledged init process such as sysvinit or systemd". LXC itself is neutral: it is "a userspace interface for the Linux kernel containment features" that can "create and manage system or application containers". The manager on top is what makes it one or the other. Q: Can you live-migrate an LXC container between hosts? A: Not under Proxmox VE: "Running containers cannot live-migrated due to technical limitations. You can do a restart migration, which shuts down, moves and then starts a container again on the target node." Proxmox characterises that as "normally only in a downtime of some hundreds of milliseconds", which describes container restart rather than your service's recovery, and the migration "will shut down the Container and kill it after the specified timeout (the default is 180 seconds)" — a different setting from the per-container Shutdown timeout, which defaults to 60. Proxmox's own answer where live migration is required is to nest: "for use cases demanding maximum isolation and the ability to live-migrate, nesting containers inside a Proxmox QEMU VM remains a recommended practice." Incus 7.4 added near-live migration, which transfers the filesystem "through a series of incremental snapshots while the container keeps running" rather than the memory state, via incus move --stateless --refresh — but it is "only supported on ZFS or btrfs and only for containers", and it is a feature-branch behaviour, not part of the Incus 7.0 LTS branch. Q: Does Docker have an equivalent to container live migration? A: Not for moving a workload to another host. The two features operators reach for are both same-host. Live restore keeps containers running while the daemon is unavailable — Docker says it "helps reduce container downtime due to daemon crashes, planned outages, or upgrades" — but it is "only supported when installing patch releases (YY.MM.x), not for major (YY.MM) daemon upgrades", only works if daemon options "such as bridge IP addresses and graph driver" did not change, applies "only to standalone containers, and not to Swarm services", and is not supported for Windows containers. Checkpoint and restore via CRIU is marked "experimental" and "currently focused on single-host use cases"; its own page says moving a server to another machine "is possible with the current implementation, but not currently a priority (and so the workflow is not optimized for the task)". The operational node-drain answer is therefore stop, reschedule, start — performed by whatever schedules the containers, not by the runtime. Q: Can Incus or Proxmox VE run Docker images? A: Yes, and the path is documented in both. Incus adds an OCI registry as a remote with "incus remote add --protocol=oci", with Docker Hub as the documented example, and then launches from it: "incus launch oci-docker:hello-world --ephemeral --console". Proxmox VE 9.1, released 19 November 2025, added the ability to "Create LXC containers from OCI images", which "allows to create full system containers from suitable OCI images", with application containers shipping as a technology preview — and the pct chapter notes that "A container created from an OCI image still uses the existing LXC framework." Incus exposes a fixed set of OCI-conditioned instance keys (oci.cwd, oci.dns.domain, oci.dns.nameservers, oci.dns.search, oci.entrypoint, oci.gid, oci.uid); there is no OCI network key, but ordinary NIC devices still apply. The traffic runs one way: there is no documented Docker feature for running a system container. Q: Are LXC containers secure enough for production? A: Unprivileged ones, on a current kernel, are the model both projects stand behind. LXC upstream says "Unprivileged containers are safe by design. The container uid 0 is mapped to an unprivileged user outside of the container and only has extra rights on resources that it owns itself", and Proxmox makes unprivileged the default when creating a container. The policy consequence matters more than the slogan: for privileged containers, LXC upstream states it "will not consider new container escape exploits to be security issues worthy of a CVE and quick fix", so a privileged container is outside the CVE feed your vulnerability management probably relies on. Docker's comparable control, rootless mode, is opt-in and moves the daemon as well as the containers off root. On audit scope, note that NIST SP 800-190 defines its subject as "application container technologies" and says "OS containers are different from the application containers that are the topic of this guide" — so it does not by itself cover a system-container estate, and the control set has to be assembled from host-hardening guidance instead. --- # Database Migration Tools: Rollback, Review and Evidence URL: https://stribog.com/blog/database-migration-tool-selection-schema-versioning-rollback Published: 2026-10-01 Category: Data Keywords: database migration tool, schema migration tool comparison, flyway undo teams edition, liquibase functional source license, atlas migrate down community edition, sqitch revert audit trail, database schema versioning rollback, migration tool licence change self-hosted Database migration tool choice turns on rollback behaviour, reviewability and the record left behind. Flyway, Liquibase, Atlas and Sqitch, compared. > **The short answer** — Choose a database migration tool on two things the feature matrices do not compare: where its licence boundary falls relative to the revert command your incident response assumes, and what it leaves in the database afterwards. Those boundaries sit somewhere different in every project — Flyway's undo is marked Flyway Teams, Liquibase's rollback is documented as all editions, Atlas's migrate down is listed as unsupported in the Apache-2.0 Community Edition while shipping in the standard distribution, and Sqitch's revert is MIT. Then build the rollback you will actually use out of backward-compatible schema changes and a tested restore, because none of them repairs a half-applied migration. Migration tooling is the least-revisited decision in a codebase and the one most likely to be named in an incident review. It is argued on an axis — versioned SQL against declarative diff — that predicts almost nothing about the night it matters. ## The axis everyone argues about decides the least That axis describes how you author a change, not what happens when one goes wrong at 02:00. Both styles apply DDL to a live database and are bounded by the same engine semantics. Pick the one your reviewers read fluently. Three axes do separate them. Where the licence boundary falls — per command, not per product. What the tool writes into your database as a record of what happened, which outlives the tool. And whether its rollback story survives a migration that failed halfway, where the vendors' own documentation is more candid than their comparison pages. Some of this is already public: Flyway's undo sitting behind Teams, Liquibase Community moving to the Functional Source License at 5.0 with a two-year conversion back to Apache 2.0, and the broad shape of Atlas's reduced Community build against its default binary are all in the listicles ranking for this query. What is not out there is the per-command split, the three Atlas pages that contradict each other and are quoted below unreconciled, what each tool leaves behind after a revert, and the CNCF Governing Board's vote on the downstream consequence. There are no throughput numbers below — benchmarks for schema tools measure the engine, not the tool. Every fact here was fetched on 1 October 2026, and vendor edition pages change without a version bump, so re-check any gating claim before it becomes load-bearing. [Figure: Four migration tools compared on three axes: Flyway's undo marked Teams and included in Enterprise, Liquibase's rollback in all editions with the rollback-one commands in Secure, Atlas's migrate down as a build boundary with Pro layered separately, and Sqitch's revert under MIT; each writes its own history table, and only Atlas computes its revert from live state.] ## Where each tool's licence boundary actually falls "Rollback is a paid feature" describes none of these four accurately, and acting on it means either paying for something you already have or planning an incident response around a command you may not run. Flyway. The undo-migrations page is badged EDITION: TEAMS and the command reference reads "Undo Description Flyway Teams" — the floor, not an exclusive tier, since undo is included in Enterprise too. What sits outside Community is the undo command, not the ability to reverse a change: a Community user can hand-write a forward migration that undoes an earlier one. Liquibase. The rollback table splits along a line almost nobody quotes correctly. rollback, rollback-to-date and rollback-count are documented as all editions; only rollback-one-changeset and rollback-one-update are Liquibase Secure. Ordinary sequential rollback is not paywalled. Atlas. Here the boundary is a build, not a price. Its community-edition page: "The Community Edition of Atlas is licensed under the Apache 2.0 license. The standard distribution is built from the same open-source codebase plus some additional proprietary features, and is free to download and use under the terms of the Atlas MSA." The core repository's LICENSE is indeed Apache 2.0. But the same page lists migrate down — with migrate lint, migrate checkpoint and migrate rebase — among the commands "the Community Edition does not support". The revert command is absent from the Apache-2.0 build and present in a distribution that is free but not Apache 2.0. What the MSA provides is not asserted here, only that it is not that licence. Atlas Pro is a third thing, layered on that boundary rather than cutting through it, and Atlas's pages name it inconsistently enough to quote rather than summarise. The migrate down page: Pro users "can protect this flow by configuring Atlas to require approval from one or more reviewers". The community-edition page: "Running atlas login unlocks Atlas Pro features such as migration linting, testing, and CI/CD integration", with a 30-day trial "after which a license is required". The feature-compatibility page lists Pre-migration checks and Migration Linting as Pro, "exclusively available to those on the Atlas Pro Plan". Three framings, unreconciled because Atlas does not reconcile them. Sqitch is the short entry: revert belongs to a project distributed under the MIT licence, with no edition to check. ```bash # migrate diff / apply / status are marked present in BOTH columns of Atlas's # community-edition capability table: Apache-2.0 Community and standard. atlas migrate diff add_orders_index \ --dir "file://migrations" \ --to "file://schema.sql" \ --dev-url "docker://postgres/17/dev" # migrate lint is listed among the commands the Community Edition does not # support. Separately, the feature-compatibility page marks Migration Linting # as Pro and says Pro features are "exclusively available to those on the Atlas # Pro Plan" -- while that same page's RULE table marks backward-incompatible, # destructive and data-dependent changes as Open. Both are printed as found. atlas migrate lint \ --dir "file://migrations" \ --dev-url "docker://postgres/17/dev" \ --latest 1 atlas migrate apply --dir "file://migrations" --url "$DATABASE_URL" atlas migrate status --dir "file://migrations" --url "$DATABASE_URL" # migrate down is also on the Community Edition's unsupported list, and ships # in the standard distribution. It computes the plan from the live database, so # it needs a Dev Database "that will be used to plan and analyze the migration". atlas migrate down \ --dir "file://migrations" \ --url "$DATABASE_URL" \ --dev-url "docker://postgres/17/dev" \ --to-version 20260930120000 ``` ## The rollback you planned for will not save a half-applied migration Flyway publishes the most honest page any of these vendors has, and it argues against Flyway's own paid feature. Undo migrations "assume the whole migration succeeded and should now be undone", which "does not help with failed versioned migrations on databases without DDL transactions" — because "A migration can fail at any point." Which engine you run decides whether that applies. Where the database "cleanly supports DDL statements within a transaction, failed migrations will always be rolled back (unless they were marked as non-transactional)"; where it does not, "Flyway won't be able to perform a clean rollback in case of failure". MariaDB's manual confirms the edge from the other side, for the ordinary non-temporary DDL a migration runs: "even if the statement fails with an error, the transaction is committed." Not every DDL statement — that page excepts CREATE TABLE and DROP TABLE with TEMPORARY, though TRUNCATE TABLE commits even on a temporary table. Atlas comes at it from the opposite direction, and the fencing is not optional. Its migrate down page says Atlas "computes a migration plan based on the current state of the database" and reverts "regardless of the state of the latest applied migration — whether it succeeded, failed, or was partially applied". The same page bounds that twice: Atlas "generates and executes a set of pre-migration checks to ensure the computed plan does not introduce destructive changes or deletes data", and "non-schema logic such as data migrations or other business-specific operations that Atlas cannot infer automatically" is what a manual down.sql exists for. So it reverts schema state and declines to delete data; it does not repair a half-applied data change — and it needs a --dev-url Dev Database to plan against. On the checks themselves Atlas's pages disagree. The down page says Atlas "still generates and runs all defined pre-migration checks before applying a down migration, whether it's manual or automatic", skippable only with --skip-checks; the feature-compatibility page lists Pre-migration checks as Pro; the community-edition table marks them excluded. Settle on neither. Sqitch is not computed recovery either: "sqitch revert runs each revert script in turn until the target change is reached." Which leaves Flyway's own recommended alternative as the one that works everywhere — "maintain backwards compatibility between the DB and all versions of the code currently deployed in production", "complemented with a proper, well tested, backup and restore strategy". Disaster-recovery testing arrives at the same place: the restore you have rehearsed is the rollback you own. [Figure: A flowchart from a migration applied, through a mid-migration failure check and a transactional-DDL check — PostgreSQL rolls back, MariaDB implicitly commits ordinary DDL — to five recovery paths: automatic rollback, an Atlas computed schema-only down plan annotated with its dev-database, checks and data-migration limits, operator-written undo or Sqitch revert scripts, application rollback, and restore from backup, plus a leaf for a failed concurrent index build.] ## Liquibase moved the licence line under its users, and Keycloak is still stuck At Liquibase Community 5.0 — release notes dated September 2025 — the licence changed. The README now reads "Liquibase Community is licensed under the Functional Source License (FSL)", with Liquibase Secure "commercially licensed" separately; the repository LICENSE.txt is "Functional Source License, Version 1.1, ALv2 Future License". The release also introduced a one-time Contributor License Agreement. For most self-hosted teams the terms are survivable. "A Permitted Purpose is any purpose other than a Competing Use", and permitted purposes "specifically include using the Software ... for your internal use and access". The licence also grants its own exit ramp: "We hereby irrevocably grant you an additional license to use the Software under the Apache License, Version 2.0 that is effective on the second anniversary of the date we make the Software available." Every release becomes Apache 2.0 two years after it ships — which changes the shape of the exposure rather than removing it. Standing on code two years old is a different proposition for a migration tool than for a library. Where it bites is transitively, because migration tools sit embedded inside other products. On 13 October 2025 Keycloak opened an issue recording the consequence: "CNCF does not permit source available licenses, and an exception is unlikely to be given, as such an alternative solution has to be found." Its options — switch component, freeze at the pre-change version, fork and maintain, or request an exception — are all, bar the exception, "far from ideal and will require significant effort." That exception is not pending. The CNCF request was closed on 21 November 2025: "the Governing Board voted to reject this license exception request, following the recommendation of the Legal Committee, because FSL is not an open source license." As of 1 October 2026 the Keycloak issue remains open. If your estate includes a self-hosted identity provider, this is a licence change you inherited without choosing a migration tool at all — the transitive pattern that produced OpenBao and OpenTofu, arriving at the schema layer. ## Reviewability: what a human sees in the pull request Ask what a reviewer is actually reading and the four diverge sharply. With Flyway it is a versioned SQL file — the statements that will run, in order — and integrity is checked rather than assumed: validate "checks if the migration locally still has the same checksum as the migration already executed in the database". With Liquibase the reviewed artefact depends on the changelog format, and the trade-off is not the one usually described: "If you use modeled changelogs (XML, YAML, JSON), you can define custom rollback logic for your changesets. Custom rollback definitions are not supported in formatted SQL changelogs." That does not mean formatted SQL has no rollback — it has an inline --rollback syntax; only the file form, rollbackSqlFile, needs a Secure licence key. ```yaml databaseChangeLog: - changeSet: id: 0042-add-orders-customer-created-index author: platform comment: Index orders for the customer dashboard query. changes: - createIndex: tableName: orders indexName: idx_orders_customer_created columns: - column: name: customer_id - column: name: created_at rollback: - dropIndex: tableName: orders indexName: idx_orders_customer_created ``` With Atlas the reviewed artefact is generated, so the question is what checks run over it — and here the documentation conflicts, so all four statements belong side by side. The feature-compatibility page says "Atlas community version includes a basic set of linting rules, while the Pro Plan provides a more advanced suite", and its rule table marks backward-incompatible, destructive and data-dependent changes as Open. The CLI table on that same page marks Migration Linting | Pro. That page also states Pro-marked features are "exclusively available to those on the Atlas Pro Plan". And the community-edition page lists "migrate lint - Lint the migration directory for destructive or unsafe changes" under "The Community Edition does not support the following commands and integrations". > **Read the Open rows carefully** — Those Open-marked rules are not a guarantee that the Apache-2.0 Community Edition applies them: by Atlas's own list that build has no migrate lint command to apply them with. Nor is the inverse safe — nothing says the Open rows are an error. Verify against the binary you intend to run. Sqitch asks the most of the reviewer and hides the least: a deploy script, a revert script and a verify script, all hand-written SQL, all in the pull request. Nothing about the reverse path is inferred. ## lock_timeout bounds the wait, not the build The migration that takes a service down is usually not a wrong migration; it is a correct one that queued behind a lock. Two settings get confused for each other here, and only one of them does the job. lock_timeout aborts a statement "that waits longer than the specified amount of time while attempting to acquire a lock", and "this timeout can only occur while waiting for locks"; it defaults to zero, meaning disabled. statement_timeout aborts the statement itself, so wrapping one around a long concurrent index build arms a killer rather than a guard. ```sql -- PostgreSQL. CREATE INDEX CONCURRENTLY cannot run inside a transaction -- block, so this file must execute outside one -- see the per-tool switches -- below for how to arrange that in Flyway, Liquibase and Atlas. -- -- lock_timeout aborts a statement ONLY while it is waiting to acquire a lock. -- It defaults to 0, which disables it. The value below is an EXAMPLE: choose -- yours from your own write-latency budget. PostgreSQL's documentation also -- advises against setting lock_timeout in postgresql.conf, because that would -- affect all sessions. SET lock_timeout = '3s'; -- Deliberately NO statement_timeout. It aborts the statement itself once the -- statement has run too long, not the lock wait, and it also defaults to 0. -- Around a long concurrent build it would kill the build rather than bound the -- wait. The two are not interchangeable knobs: PostgreSQL's docs note that if -- statement_timeout is nonzero, it is "rather pointless to set lock_timeout to -- the same or larger value, since the statement timeout would always trigger -- first" -- so lock_timeout only does useful work below it. -- A normal build locks the table against writes while reads continue. -- CONCURRENTLY avoids locking out writes at the cost of two table scans and -- more total work. PostgreSQL documents it as taking SHARE UPDATE EXCLUSIVE -- rather than the ACCESS EXCLUSIVE lock "on the index as a whole"; the same -- mode is held on the TABLE for the duration of the build, which is why other -- DDL on this table waits even though writes do not. CREATE INDEX CONCURRENTLY idx_orders_customer_created ON orders (customer_id, created_at DESC); -- This is not fail-clean. A failure during the scan leaves an index marked -- INVALID: "ignored for querying purposes because it might be incomplete; -- however it will still consume update overhead". For a UNIQUE index whose -- SECOND scan fails, the invalid index "continues to enforce its uniqueness -- constraint afterwards". Recovery: DROP INDEX then retry, or -- REINDEX INDEX CONCURRENTLY. ``` Getting that statement out of a transaction is per-tool. Flyway's executeInTransaction in the project TOML or via FLYWAY_EXECUTE_IN_TRANSACTION configures the run; the per-migration form is a sibling script configuration file, whose parameters are "effective at the level of a specific migration rather than globally". It "does not apply to Native Connectors". ```ini # Sits in the same folder as V2__add_orders_index.sql. The file name is the # migration file name with ".conf" appended. Flyway's script-configuration # reference lists executeInTransaction as a Community-tier override meaning # "Whether to execute this migration in a transaction." executeInTransaction=false ``` Liquibase's runInTransaction defaults to true, with a warning worth repeating verbatim: "If runInTransaction is set to false and an error occurs part way through running a changeset that contains multiple statements, the Liquibase DATABASECHANGELOG table will be left in an invalid state." No Sqitch transaction flag is named here, because none is sourced. Atlas "executes each migration file in a transaction" by default; --tx-mode all wraps every pending file in one, which is not the mechanism here, since a concurrent build cannot run in a transaction block at all. The documented way is the file-level directive: ```sql -- atlas:txmode none CREATE INDEX CONCURRENTLY name_idx ON users (name); ``` ## What each tool writes down, and what that record cannot prove Each of these tools keeps its state in the database it manages, in a table of its own design: Flyway a schema history table whose name is a configurable setting defaulting to flyway_schema_history, Liquibase DATABASECHANGELOG, Atlas atlas_schema_revisions. Sqitch keeps two — its PostgreSQL registry comments one as tracking "the changes currently deployed to the database", and a separate events table as the "full history of all deployment events". ```sql -- Flyway. The table NAME is a configurable setting; flyway_schema_history is -- only its documented default, so check your project configuration first. SELECT * FROM flyway_schema_history; -- Liquibase. A rollback removes the corresponding row from this table. If a -- Secure licence key has enabled DATABASECHANGELOGHISTORY (4.27.0+, disabled -- by default), that is where the rollback itself is recorded. SELECT * FROM databasechangelog; -- Atlas. Named on the migrate down page as the migration history table. SELECT * FROM atlas_schema_revisions; -- Sqitch, PostgreSQL registry. Two tables, not one: what is deployed now, and -- the full history of events. Substitute your own registry schema name. SELECT * FROM sqitch.changes; SELECT * FROM sqitch.events; ``` An assessor's question is rarely which tool you use. It is: show me every change applied to this database in the audit period, who authorised it, and what happened to the ones you reversed. Judge each record by what it cannot prove. Liquibase's gap is structural and documented: a rollback "removes the corresponding row from the DATABASECHANGELOG table", so the primary table does not evidence the reversal. DATABASECHANGELOGHISTORY records rollbacks instead — but it is "available in Liquibase 4.27.0 and later", "only available to Liquibase Secure users", and "disabled by default". Even switched on, the page declines the role: "It is not necessarily designed to meet security or regulatory needs." That is the vendor telling you not to present it as control evidence. Flyway's post-undo record is a state on a row — Undone, "This versioned migration succeeded but has since been undone" — a durable fact about the migration, not an attribution chain. For Atlas the honest answer is a blank cell: nothing in the migrate down documentation says what the command does to rows in atlas_schema_revisions. Treat it as untested rather than inferring either answer. Sqitch is the only one of the four whose registry was designed as an event log. In its PostgreSQL registry script the event column is constrained by a CHECK to 'deploy', 'revert', 'fail' and 'merge', so failures are rows rather than absences; the primary key (change_id, committed_at) means a redeploy appends instead of overwriting; and planner_name beside committer_name separates who planned a change from who ran it. That separation exists in none of the other three registries. One caveat: the MySQL registry script models event as an ENUM of the same four values and creates no CHECK. ```sql -- Sqitch, PostgreSQL registry. Substitute your own registry schema name. -- The event column is constrained by a CHECK to deploy, revert, fail, merge, -- so a failed deploy is a row rather than a gap. The primary key is -- (change_id, committed_at): a redeploy appends instead of overwriting. SELECT e.committed_at, e.event, e.change, e.planner_name, e.planned_at, e.committer_name FROM sqitch.events AS e WHERE e.committed_at >= now() - interval '90 days' ORDER BY e.committed_at DESC; ``` ## Exit ramps: what a tool swap leaves stranded Two facts determine what leaving costs, and neither appears on a comparison page. The first: the history artefact is tool-specific and does not travel. The four tables named above are four different schemas, and none of these tools reads another's — so whatever you move to starts with no history of what came before, unless you carry it across yourself. The second: whether the changes survive as replayable SQL. Flyway's versioned SQL migrations and Sqitch's operator-written deploy and revert scripts are files a successor tool — or a human — can read and run. Liquibase's modeled changelogs carry rollback logic inside the model, though formatted SQL does have the inline --rollback form, with only rollbackSqlFile requiring a Secure licence key. Atlas's down plans are by default computed from live state rather than stored, unless a migration file carries a manual down.sql section. No commands for the move are prescribed here; that depends on what your history table actually contains. What matters at selection time is the shape of the dependency: plain SQL files plus a readable history table is a cheap exit, a proprietary changelog model plus a computed revert an expensive one. That is the calculation exit-cost modelling applies to data gravity, pushed down to the schema layer — and if your database already runs on Kubernetes with an operator, the migration tool is one of the last pieces of the data path you still choose independently. ## The long game: outliving the vendor that wrote the tool Schema history outlives tool choices, usually by several of them. Weight the decision accordingly: licence durability and readability of the record above feature count, with the revert command you intend to rely on checked against the edition you are entitled to run — per command, not per product. On that rule Sqitch scores unusually well: MIT, plain SQL scripts, and the only registry of the four designed as an event log. The counterarguments belong in the same paragraph. It is a Perl project whose README copyright names one principal author beside a corporate line that stopped in 2021 — a bus-factor risk of a different shape to a licence risk, not an absence of risk. And its revert is only as good as the revert scripts someone wrote; the manual says so in as many words. The pattern underneath this has reached three layers of the stack in three years: secrets, infrastructure-as-code, and now the schema — and, through Keycloak, identity with it. A licence line moves under a tool nobody chose deliberately, and the cost lands on whoever cannot show what their systems did. Keep the history in a table you can read with SELECT, keep the changes in files a human can replay, and the tool becomes replaceable — the only property that has ever survived a vendor. The discipline that makes infrastructure state portable applies here, one layer down. ### FAQ Q: Which database migration tool has the best rollback? A: None of them has a rollback that repairs a half-applied migration, and that is the honest framing. Flyway's own documentation says undo migrations "assume the whole migration succeeded and should now be undone" and that this "does not help with failed versioned migrations on databases without DDL transactions", because a migration can fail at any statement. Atlas computes a down plan from the live database and documents it as working "regardless of the state of the latest applied migration", but bounds that twice: by default it generates pre-migration checks "to ensure the computed plan does not introduce destructive changes or deletes data", and non-schema logic such as data migrations is what "Atlas cannot infer automatically", which is why teams write a manual down.sql. Sqitch's revert runs "each revert script in turn" that an operator wrote. So the real rollback is Flyway's own recommendation: backward compatibility between the database and every deployed version of the application code, plus a tested backup and restore strategy. Q: Is database rollback a paid feature in Flyway, Liquibase or Atlas? A: The boundary is per command and sits somewhere different in each project. Flyway's undo-migrations page is badged EDITION: TEAMS and its command reference reads "Undo Description Flyway Teams" — marked Flyway Teams, and included in Enterprise. Liquibase documents rollback, rollback-to-date and rollback-count as available in all editions; only rollback-one-changeset and rollback-one-update are Liquibase Secure. Atlas lists migrate down among the commands "the Community Edition does not support", while the standard distribution that does include it is described on the same page as "free to download and use under the terms of the Atlas MSA" — a build boundary rather than a price, since the Community Edition is the Apache-2.0 one. Sqitch's revert is MIT with no edition to check. A Flyway Community user can still hand-write a forward migration that reverses an earlier one. Q: What does the Liquibase Functional Source License change mean for self-hosted users? A: Liquibase Community moved to the Functional Source License at 5.0, release notes dated September 2025, and the repository LICENSE.txt is FSL-1.1-ALv2. For internal self-hosted use the terms are generally survivable: "A Permitted Purpose is any purpose other than a Competing Use", and permitted purposes "specifically include using the Software ... for your internal use and access". The licence also grants an additional Apache License 2.0 "effective on the second anniversary of the date we make the Software available", so each release becomes Apache 2.0 two years after shipping. The sharper problem is transitive. Keycloak opened an issue on 13 October 2025 noting that "CNCF does not permit source available licenses" and that "an alternative solution has to be found"; the CNCF licence-exception request was closed on 21 November 2025 after "the Governing Board voted to reject this license exception request ... because FSL is not an open source license". As of 1 October 2026 the Keycloak issue is still open. Q: What record does each migration tool leave in the database after a rollback? A: Different records, with different limits. Liquibase documents that "When Liquibase rolls back a changeset, it removes the corresponding row from the DATABASECHANGELOG table"; rollbacks are recorded in DATABASECHANGELOGHISTORY instead, which is "available in Liquibase 4.27.0 and later", "only available to Liquibase Secure users", "disabled by default", and whose own page says it "is not necessarily designed to meet security or regulatory needs". Flyway's schema history table — default name flyway_schema_history, a configurable setting — documents an Undone state meaning "This versioned migration succeeded but has since been undone". Sqitch's PostgreSQL registry keeps a separate events table commented as the "full history of all deployment events", with its event column CHECK-constrained to deploy, revert, fail and merge, a primary key of (change_id, committed_at) so redeploys append, and planner_name beside committer_name. For Atlas, nothing in the migrate down documentation states what the command does to rows in atlas_schema_revisions, so treat that as untested rather than assumed. Q: How do I run CREATE INDEX CONCURRENTLY from a migration tool? A: You need the statement outside a transaction block, because PostgreSQL documents that "a regular CREATE INDEX command can be performed within a transaction block, but CREATE INDEX CONCURRENTLY cannot". In Flyway, use a per-migration script configuration file — V2__name.sql.conf beside V2__name.sql — containing executeInTransaction=false; the project TOML [flyway] executeInTransaction and FLYWAY_EXECUTE_IN_TRANSACTION settings configure the whole run instead, and the parameter "does not apply to Native Connectors". In Atlas, write the file-level directive "-- atlas:txmode none" directly above the statement, or use the run-level --tx-mode none which "will not create any transaction"; --tx-mode all is the opposite of what you want here. Liquibase's runInTransaction attribute exists but carries a documented warning that setting it to false can leave DATABASECHANGELOG "in an invalid state" if a multi-statement changeset fails part way. Bound the lock wait with lock_timeout, which aborts only while waiting for a lock and defaults to 0; do not set a statement_timeout around the build, which would abort the build itself. --- # Terraform vs Ansible: State You Own, State You Converge URL: https://stribog.com/blog/terraform-ansible-state-you-own-versus-state-you-converge Published: 2026-09-30 Category: Delivery Keywords: terraform vs ansible, ansible vs terraform, ansible and terraform, ansible with terraform, terraform state drift detection, ansible check mode limitations, terraform removed block state, configuration drift ownership infrastructure, ansible idempotency command module changed_when Terraform needs a database to map config to reality. Ansible needs none. That one difference decides who owns drift, what a dry run proves, and exit cost. > **The short answer** — Terraform vs Ansible is not procedural versus declarative — it is whether the tool keeps a persistent model of the world. Terraform's documentation says it "requires some sort of database to map Terraform config to the real world"; Ansible's says "there are no servers, daemons, or databases required". Which drift is detectable, what a dry run proves, whether a deletion can be derived, and what you carry out on exit all follow from that one structural difference. This comparison is usually settled on the wrong axis. Sort the two tools by whether they are declarative and the answer predicts nothing — and the real boundary then gets discovered one incident at a time, in the place where neither tool was looking. ## Procedural versus declarative sorts modules, not tools Retire that framing carefully — narrowed, not abolished. Ansible's glossary, under Resource Model, is unambiguous: the resource models "are idempotent meaning change commands are not run unless needed, and Ansible will bring the system back to a desired state regardless of the actual state -- rather than you having to tell it how to get to the state." Convergence to a declared end state. The same project draws the limit, with one word most comparisons drop: "Most modules accept parameters that describe the desired state of the system." Most, not all. The clearest exception says so in its own source: in ansible-core 2.21.4, ansible.builtin.command declares check_mode support as partial, noting that "while the command itself is arbitrary and cannot be subject to the check mode semantics it adds creates/removes options as a workaround." The axis is real, then, but cuts *inside* Ansible's module set rather than between the tools. A playbook of file, package and service tasks is as declarative as anything in HCL; one leaning on command and shell is not, and the module documentation tells you which you are writing. The axis that does separate them is structural. Terraform's documentation states that "State is a necessary requirement for Terraform to function". Ansible's architecture page says the opposite about itself: "there are no servers, daemons, or databases required." One tool remembers what it converged on last time. The other does not. The rest is consequence. [Figure: Two reconciliation loops side by side: Terraform's configuration, state database, provider read and three-way plan comparison on the left, Ansible's playbook, module pushed over SSH, module inspecting the target and conditional change on the right, with dashed hexagons hanging off each loop listing its documented blind spots.] ## Terraform: a database of the world, and what it compares Knowing that terraform plan compares three things rather than two is most of the skill in reading its output. By default Terraform "Reads the current state of any already-existing remote objects"; it "Compares the current configuration to the prior state and noting any differences"; and it "Proposes a set of change actions that should, if applied, make the remote objects match the configuration." Three inputs: the live read, the prior state, the configuration. The default response to a mismatch is equally explicit — "By default, Terraform detects any difference in the current settings of a real infrastructure object and plans to update the remote object to match configuration." Configuration wins. Someone changed a security-group rule in a console last Thursday; the next apply quietly changes it back, and whether that is repair or outage depends on facts Terraform does not have. The counter-move arrives in the same breath. Refresh-only planning mode "creates a plan whose goal is only to update the Terraform state and any root module output values to match changes made to remote objects outside of Terraform" — for when "you've intentionally changed one or more remote objects outside of the usual workflow (e.g. while responding to an incident)". One mode asserts configuration over the world; the other reconciles Terraform's records with it. Reconciling the record is not adopting the change: refresh-only leaves the configuration untouched, so a refreshed value that still disagrees with configuration gets planned back on the next normal apply — unless you edit the configuration to match, or list that attribute under ignore_changes. Three declarations carry most of what the record will and will not notice. ```hcl # Three declarations that decide what Terraform's record will and will not notice. resource "aws_instance" "api" { ami = var.ami_id instance_type = "m7i.large" tags = { Name = "api" } lifecycle { # Drift you are deliberately choosing NOT to own. The scope is narrower than # it looks: "Terraform only ignores attributes defined by the resource type", # and you "can't apply ignore_changes to itself or to any other # meta-arguments". It is also wider than "drift declined": these arguments # are considered when planning a create and ignored when planning an update, # so a later edit to var.ami_id will NOT replace this instance either. # Everything listed here becomes somebody else's problem, so write down whose. ignore_changes = [ami, tags] } } # Stop managing an object WITHOUT destroying it. Deleting the resource block on # its own plans a destroy, and prevent_destroy does not save you: that rule # "doesn't prevent Terraform from destroying the resource if you remove the # resource configuration". After this applies, Terraform "no longer manages that # infrastructure's lifecycle" — the object keeps running, unmanaged. removed { from = aws_instance.legacy_worker lifecycle { destroy = false } } # Adopt an object that already exists. `to` is required, and so is exactly one # of `id` or `identity`. The docs table marks only `to` as required, but the # v1.16.4 decoder rejects a block carrying neither: "At least one of 'id' or # 'identity' must be specified." They are two ways of naming the same object and # cannot both be set — "You cannot use the id argument and identity argument in # the same import block." Which forms exist is the provider's business: the AWS provider # documents id for aws_instance (Terraform >= 1.5) and identity = { id = "..." } # (>= 1.12), so either names this instance. The id form is used here. import { to = aws_instance.api id = "i-0123456789abcdef0" } ``` ## Ansible: no database, by design Ansible's loop has no prior to compare against, and the architecture page explains why mechanically: Ansible "executes these modules (over SSH by default), and removes them when finished." The module lands, reads the host as it finds it, acts if the host differs from the task, then is deleted. The only truth is the host, read fresh every run. That is a genuine architectural strength, usually undersold. Nothing goes stale because nothing is kept: no state file to lock, corrupt, leak or migrate. A host rebuilt from scratch converges identically to one running three years, neither having a history the tool consults — the discipline that makes GitOps controllers trustworthy at scale. The cost lands in one place: absence carries no meaning. Delete a task from a playbook and nothing happens to the host, because Ansible has no record saying it once put something there. Deletion exists only where you wrote it — the 2.21.4 file module's state: absent is the branch where "directories will be recursively deleted, and files or symlinks will be unlinked." Terraform derives destruction from a diff against its record; Ansible cannot, having no record to diff. Omission is not neutral in the other direction either. In ansible-core 2.21.4 the file module defaults state to "the current state of the file if it exists, directory if recurse=yes, or file otherwise" — and on that file branch the path "will be modified if it exists but will NOT be created if it does not exist." Nor does it pass quietly: the module rejects any prior state that is not a file or a hard link, so a missing path fails the task — file (...) is absent, cannot continue — in a live run and under --check alike. Creating a file is state: touch, copy or template. ## Failure modes: where each model disagrees with the world Both tools fail in the same shape — reporting clean while the world moved — by different routes. Take Terraform's four separately; conflating them manufactures false coverage. - Write-only arguments. Since Terraform 1.11, providers can declare "managed resource attributes that are configured by practitioners but are not persisted to the Terraform plan or state artifacts." In the schema, absent from the record — so nothing downstream of state reports drift on them. The exception runs the other way: a provider marking one RequiresReplace forces a replacement on *every* plan. - ignore_changes. Deliberate blindness, correctly used and easily forgotten. It suppresses only attributes the resource type itself defines, and cannot be applied to itself or to another meta-argument — a narrower escape hatch than it looks. - Objects never imported. Not in state means not in the comparison. The plan says nothing about them, and saying nothing looks exactly like clean. - Plaintext local state. "If you are developing with Terraform locally, Terraform stores your state in a plaintext file, which includes any secret values you defined in your configuration." The guidance is to "Treat your state file as sensitive data" — the database that makes drift detectable is also an artefact to protect like a credential store. One of those is routinely got backwards. HashiCorp's SDKv2 guide tells provider *authors* that "It's essential that all attributes defined in the schema are recorded and kept up-to-date in state." That is advice to implementers, not a guarantee to users — and write-only arguments are the documented counter-example in the same product. The real bound on drift detection is what the provider stores, not the size of the schema. Ansible's blind spots concentrate in what teams most want to trust: the dry run. Check mode is documented as a simulation, and the third sentence is the one that matters — "Modules that support check mode report the changes they would have made. Modules that do not support check mode report nothing and do nothing." A clean --check run is not evidence that nothing would change, only that nothing reported a change. The same page adds that check mode "will not generate output for tasks that use conditionals based on registered variables" — so a conditional chain goes dark exactly where the interesting behaviour lives. command is the sharpest case, and the 2.21.4 wording is specific: check_mode support partial, diff_mode support none. Check mode works when passing creates or removes — the module then "will check for the existence of the file and report the correct changed status" — and "If these are not supplied, the task will be skipped." Skipped, not evaluated: a changed_when there never sees a result. Diff mode is selective too, and because --diff "can reveal sensitive information, you can disable it for a task by specifying diff: false." > **Silence is not evidence** — A green terraform plan and a clean ansible-playbook --check are both compatible with a materially changed system. Terraform is silent about what the provider never stored and what was never imported, Ansible about every module lacking check-mode support. Treat both as evidence of no detected change, never of no change — write that into the runbook before an auditor asks. ## Drawing the boundary so drift has an owner The consensus answer here — "they are complementary, use both" — is true and nearly worthless, stopping one sentence short of the question that matters: which tool owns which fact, and who owns the facts neither models. A boundary that fits on one page: 1. Terraform owns anything a provider API can read back. Resource identity, lifecycle, network and account topology. The test is mechanical: if a provider can READ it, it belongs in the record that tracks it. 2. Ansible owns in-OS state on a host that already exists. Packages, files, services, users, kernel parameters — anything inside a machine's operating system belongs to the tool that reads that machine directly. 3. The seam is an artefact, not a convention. terraform output -json, rendered into an inventory by a script that fails loudly on a missing output. A boundary maintained by habit erodes the first time somebody is in a hurry. 4. The gap gets a named owner. Properties outside every provider schema, write-only arguments inside it, in-OS state no task asserts, anything a command task did without a creates guard. Neither record models this class, so by default it is nobody's — which is how it becomes an unfunded liability. The fourth point is load-bearing, and organisational rather than technical. Name a role — the platform owner signing the boundary document — with a standing review of what each side declined to model: the ignore_changes list, the write-only arguments in play, the resources still outside state, the command tasks with no creates guard. Four lists, reviewed on a schedule. Same failure mode that makes golden paths without enforcement decorative: a standard nobody is accountable for is a preference. [Figure: The boundary drawn as two owned zones — Terraform owning resource identity, lifecycle and topology; Ansible owning in-OS state — joined by a seam artefact rendered from terraform output, with a dashed gap region listing drift neither tool models and a node insisting that gap gets a named owner, annotated separately with NIST CM-6(c) and CM-3 guidance.] ```bash #!/usr/bin/env bash set -euo pipefail # The handoff contract. Terraform holds these facts in state already; this script # renders them into an Ansible inventory so the boundary is a file somebody can # read, diff and review — not a convention that lives in three people's heads. # # What must NOT cross: secrets (a local state file is plaintext), and any in-OS # fact Ansible will read fresh from the host anyway. Copying either one creates a # second authority for the same fact, which is how the two records start lying. required=(app_hosts db_hosts environment) outputs="$(terraform output -json)" for key in "${required[@]}"; do if ! jq -e --arg k "$key" 'has($k)' <<<"$outputs" >/dev/null; then echo "handoff contract broken: terraform output '$key' is missing" >&2 exit 1 fi done # Redirect stdout to a .json FILE and point -i at that file. This is a STATIC # inventory read by the yaml inventory plugin, whose default extensions include # .json — which is why hosts is a mapping here. Do not chmod +x this script and # hand it to -i directly: that invokes the script plugin instead, which speaks # --list/--host and wants hosts as a LIST. Different plugin, different shape. # # all is emitted FIRST on purpose: the yaml plugin's own note is that all must be # the first entry when you set vars on it. jq '{ all: { vars: { env: .environment.value } }, app: { hosts: (.app_hosts.value | map({ (.): { ansible_host: . } }) | add // {}) }, db: { hosts: (.db_hosts.value | map({ (.): { ansible_host: . } }) | add // {}) } }' <<<"$outputs" >"${INVENTORY_OUT:-./inventory.json}" ``` ## Proving drift: the commands, both sides Drift you cannot produce on demand is an opinion. Both tools emit a scheduled, machine-readable report on their free tier with no control plane — but you build both, and on the Ansible side you switch the evidence on first. Terraform's version is an artefact, not terminal output. Its machine-readable plan format carries a resource_drift field — "a description of the changes Terraform detected when it compared the most recent state to the prior saved state" — reached with terraform show -json . Drift becomes a file you retain, diff and attach to a ticket. ```bash #!/usr/bin/env bash set -euo pipefail # Refresh-only drift report, free tier, no control plane. # # The saved plan is the EVIDENCE and is never applied. Applying a refresh-only # plan writes the remote values INTO state, which means the drift is absorbed # rather than resolved and tomorrow's report comes back empty with nothing fixed. # Reconciling the record is a deliberate decision a human makes, not a side # effect of running the report. stamp="$(date -u +%Y-%m-%dT%H-%M-%SZ)" out_dir="${DRIFT_DIR:-./drift}" mkdir -p "$out_dir" plan_file="$out_dir/$stamp.tfplan" report="$out_dir/$stamp.drift.json" terraform init -input=false >/dev/null terraform plan -refresh-only -input=false -out="$plan_file" terraform show -json "$plan_file" | jq --arg stamp "$stamp" --arg ws "$(terraform workspace show)" '{ recorded_at: $stamp, workspace: $ws, drift: (.resource_drift // []), drift_count: ((.resource_drift // []) | length) }' >"$report" count="$(jq -r '.drift_count' "$report")" echo "drift objects: $count -> $report" # Scope note for whoever reads this file in six months: an empty array proves # only that Terraform saw no drift in attributes the PROVIDER stores in state. # Write-only arguments never reach plan or state, and objects never imported are # not in the comparison at all. Note what does NOT shrink this array: # ignore_changes only stops a NORMAL plan from proposing an update toward # configuration. This array is built by comparing prior state to refreshed # state, so a drifted attribute you ignore still shows up here — which is the # useful behaviour. You keep seeing the drift you declined to correct. # Exit status is a SIGNAL, not a build failure. Wire it to a ticket, not a rollback. [ "$count" -eq 0 ] ``` Two things this report is not. It is not HCP Terraform's health assessment, a hosted feature: health assessments "are available in HCP Terraform Standard and Premium editions." And the two do not measure the same thing — that page says so: "Configuration drift differs from state drift. Drift detection does not detect state drift." Keep the words apart in your runbook. Ansible's equivalent is a scheduled convergence run where the changed count *is* the drift signal. The catch: nothing is recorded until you ask. ansible-core's configuration definitions give DEFAULT_LOG_PATH a null default described as "File to which Ansible will log on the controller. When not set the logging is disabled." Callbacks are the same story — CALLBACKS_ENABLED defaults to an empty list because "not all callbacks need enabling, but many of those shipped with Ansible do as we don't want them activated by default." ```ini [defaults] # Off by default. ansible-core's config definition for DEFAULT_LOG_PATH ships a # null default and states plainly: "When not set the logging is disabled." # Rotate and permission this file like any other audit artefact. log_path = /var/log/ansible/convergence.log # Also off by default. CALLBACKS_ENABLED is an empty list because "many of those # shipped with Ansible do [need enabling] as we don't want them activated by # default". junit ships in ansible-core and writes one machine-readable file per # run, which is what turns a convergence run into retainable evidence. Two # caveats before you rely on it: the callback reads its output directory from the # JUNIT_OUTPUT_DIR environment variable rather than from this file, defaulting to # ~/.ansible.log, and a changed task counts as a JUnit PASS unless # JUNIT_FAIL_ON_CHANGE=true — the changed bit is only inside system-out. log_path # above is the artefact that records changed unconditionally. callbacks_enabled = ansible.builtin.junit # A task skipped under check mode is the single most important line in a dry-run # report, so do not hide it. display_skipped_hosts = True ``` ```yaml --- - name: Converge the application host hosts: app become: true gather_facts: true tasks: - name: Remove the retired drop-in, because absence is never inferred # Deletion happens ONLY where state: absent is written. Deleting this task # from the playbook deletes nothing from the host — there is no record of # a previous run from which a removal could be derived. ansible.builtin.file: path: /etc/systemd/system/api.service.d/10-legacy.conf state: absent notify: Reload systemd - name: Ship the unit drop-in directory # state is always explicit. Omitting it is not a null instruction: in # ansible-core 2.21.4 the default is "the current state of the file if it # exists, directory if recurse=yes, or file otherwise", and the file branch # FAILS on a missing path — "file () is absent, cannot continue" — # rather than creating it. Use touch, copy or template to create. ansible.builtin.file: path: /etc/systemd/system/api.service.d state: directory owner: root group: root mode: "0755" - name: Write the runtime configuration ansible.builtin.template: src: api.env.j2 dest: /etc/api/api.env owner: root group: api mode: "0640" # --diff "can reveal sensitive information", so it is disabled here rather # than left to whoever runs the playbook next. diff: false notify: Restart api - name: Rebuild the search index only when the marker is absent # command ships check_mode support "partial" and diff_mode support "none". # creates is the documented workaround: under --check the module checks for # the file and reports the correct changed status. Supply neither creates # nor removes and "the task will be skipped" — a skipped result never has # its changed_when evaluated at all. Note what is NOT here: no changed_when. # Steady state is changed=false with rc=0, stdout "skipped, since … exists", # and NO skipped key — so a hand-rolled rc-and-skipped expression is reading # a field that does not exist on this result. The module's own flag is # already the drift signal; overriding it # inverts the thing this run exists to measure. ansible.builtin.command: cmd: /usr/local/bin/rebuild-index --output /var/lib/api/index.bin creates: /var/lib/api/index.bin - name: Report what the package set would change, without changing it # check_mode: true pins this task to dry-run in EVERY run, including a live # one. The check-mode changed flag IS the preview, so nothing overrides it: # a changed_when: false here would force the registered result and the # recap to changed=false and the task would report nothing at all. # Upgrades belong to a different playbook with a human on it. ansible.builtin.apt: name: nginx state: present check_mode: true register: package_preview handlers: - name: Reload systemd ansible.builtin.systemd_service: daemon_reload: true - name: Restart api ansible.builtin.systemd_service: name: api state: restarted ``` One more piece of the free-tier picture, stated narrowly. AWX — which its own README calls "one of the upstream projects for Red Hat Ansible Automation Platform", not the platform itself — has cut no release since 24.6.1 on 2 July 2024, and that README says "Releases of this project are now paused during a large scale refactoring." So the free, self-hostable control plane is static for now: scheduled convergence with retained evidence gets built from systemd timers, log_path and a callback — the files above. ## What an auditor asks, and which tool can answer Assessors do not ask which tool is more declarative. They ask two questions teams collapse into one, and the tools answer them differently. The first is about deviations from settings. NIST SP 800-53 Rev. 5 CM-6 item c asks an organisation to "Identify, document, and approve any deviations from established configuration settings" for an organisation-defined set of system components, based on organisation-defined operational requirements. A list of differences from a declared baseline — Terraform's resource_drift, Ansible's changed tasks — is the *identify* step only; document and approve stay a human record either way. Neither tool emits the list out of the box, and the Ansible side emits nothing until log_path and a callback are set. The second is about out-of-band change, and it sits in guidance rather than a control statement. Rev. 5's discussion for CM-3 says configuration change control "includes changes to baseline configurations, configuration items of systems, operational procedures, configuration settings for system components, remediate vulnerabilities, and unscheduled or unauthorized changes" — guidance text, quoted as guidance. CM-3 item e separately asks that records of configuration-controlled changes be retained for an organisation-defined period, and CM-2 asks for a "current baseline configuration of the system" under configuration control. That is what a two-tool split strains: the baseline lives in two places, the retained record in two formats. The honest answer is layered, and the layering is the point: Terraform's refresh-only report covers objects it holds in state, in attributes the provider stores; the Ansible convergence log covers in-OS state on hosts the inventory names, for modules that report changes; the gap is covered by a named owner and a review cadence, not a tool. Mapping that onto control families is the 800-53 exercise for a cluster you run yourself, carrying the trap of a benchmark read as evidence rather than a checklist: a clean report is only as broad as what was looked at. ## Exit ramps: what you carry out of each tool Leaving Ansible is mechanically cheap and intellectually expensive. There is no state to migrate because there is no state — but the playbooks encode years of knowledge about hosts, and that knowledge is the asset, not the YAML. Licence-wise nothing to plan around: the ansible/ansible repository ships the GNU General Public License Version 3. Leaving Terraform is the opposite shape: the state file is both what you must carry and what makes leaving tractable. Two blocks do the work. removed relinquishes management without destroying — "To remove a resource from Terraform state without destroying it, replace the resource block with a removed block", after which "Terraform no longer manages that infrastructure's lifecycle." import runs the same road in reverse. Generating the matching configuration rather than writing it? terraform plan -generate-config-out is still documented as experimental, warning that "the generated configuration format may change in future versions" — a poor foundation for HCL you are about to make authoritative. The licence arithmetic is where teams reason from a headline instead of the file. Terraform 1.6.0 and later ships the Business Source License 1.1 with "International Business Machines Corporation (IBM)" as Licensor, and its Additional Use Grant bars offering the work to third parties "on a hosted or embedded basis in order to compete with IBM Corp.'s paid version(s)". The same licence then carves out the case most readers are in: "Hosting or using the Licensed Work(s) for internal purposes within an organization is not considered a competitive offering." Nor is it permanent — the Change Date is "Four years from the date the Licensed Work is published", the Change License is MPL 2.0, and it "applies separately for each version" with a Change Date that "may vary for each version" — no single calendar date applies to Terraform as a whole. OpenTofu ships under the Mozilla Public License 2.0. If that is the path you are pricing, the registry, not the binary, is the hard part, and pricing an exit as a number applies here too. ## The long game: which record survives the team Over a decade the durable asset is not the tool. Latest stable releases on 30 September 2026 were Terraform v1.16.4 (23 September 2026), ansible-core v2.21.4 (8 September 2026) and OpenTofu v1.12.6 (19 August 2026); all three will move several times before these decisions are revisited. What survives is the shape of the record you keep and the boundary you wrote down. Terraform's state is a genuine organisational asset — a machine-readable inventory of what exists, which nobody reconstructs from memory after a reorganisation. It is also an access-control liability, because locally it is plaintext including configured secrets. Ansible's playbooks are a different asset: they encode how a host is supposed to be and stay runnable long after their authors leave, depending on no stored history. Neither covers the other. The team that keeps both, explicit about which is authoritative for which fact, is the one still able to answer questions about its estate in 2036 — the argument that makes provisioning you own end to end worth the setup cost. Keep three artefacts and the tooling question stops being load-bearing: the boundary document naming who owns each zone and the gap; the drift and convergence reports, retained; and the four lists of what each side declined to model. All three are portable across tools, and no vendor comparison page will hand you one. ### FAQ Q: What is the real difference between Terraform and Ansible? A: Whether the tool keeps a persistent model of the world. Terraform's documentation states that "State is a necessary requirement for Terraform to function" and that it "requires some sort of database to map Terraform config to the real world". Ansible's architecture documentation states that "there are no servers, daemons, or databases required" and that modules are executed over SSH and removed when finished. That one structural fact decides the rest: Terraform can derive a deletion from a diff against its record and Ansible cannot, Terraform can report drift on attributes the provider stores in state while Ansible reads the host fresh every run, and the two leave behind entirely different artefacts when you go. The procedural-versus-declarative framing does not sort the tools, because Ansible's own glossary describes its modules as idempotent resource models that bring a system back to a desired state. Q: Is Ansible declarative or procedural? A: Both, and the module documentation tells you which one you are writing. Ansible's glossary entry for Resource Model says the models "are idempotent meaning change commands are not run unless needed, and Ansible will bring the system back to a desired state regardless of the actual state". Its architecture page then scopes that with one important word: "Most modules accept parameters that describe the desired state of the system." Most, not all. In ansible-core 2.21.4, ansible.builtin.command declares check_mode support as partial with the note that the command itself "is arbitrary and cannot be subject to the check mode semantics", and diff_mode support as none. So the axis cuts inside Ansible's module set rather than between Ansible and Terraform: a playbook of file, package and service tasks is declarative, and one built on command and shell is not. Q: Can I detect infrastructure drift with Terraform for free? A: Yes, locally, with two commands and a scheduler. Run terraform plan -refresh-only -out=FILE, then terraform show -json FILE, and read the resource_drift field — documented as "a description of the changes Terraform detected when it compared the most recent state to the prior saved state". Never apply that plan: applying a refresh-only plan writes the remote values into state, absorbing the drift rather than resolving it. Two limits matter. The report is bounded by what the provider actually records in state, so write-only arguments (Terraform 1.11+, "not persisted to the Terraform plan or state artifacts") and objects never imported are invisible to it. Attributes under ignore_changes are not: that meta-argument only stops a normal plan proposing an update, so drift in an ignored attribute still appears in this array. And it is not the same artefact as HCP Terraform's drift detection, which is a health assessment available in Standard and Premium editions, and whose own page says it "does not detect state drift". Q: Does ansible-playbook --check prove that nothing will change? A: No, and Ansible's own documentation is explicit about why. Check mode runs without making changes, and "Modules that support check mode report the changes they would have made. Modules that do not support check mode report nothing and do nothing." A clean run therefore proves that nothing reported a change, which includes every module that was never going to report one. The same page notes that check mode "will not generate output for tasks that use conditionals based on registered variables". ansible.builtin.command is the sharpest case: in ansible-core 2.21.4 it declares check_mode support partial and diff_mode support none, and its notes say check mode works when passing creates or removes, while "If these are not supplied, the task will be skipped" — so a changed_when on that task never sees a result at all. Q: Where should the boundary between Terraform and Ansible sit? A: Terraform owns anything a provider API can read back — resource identity, lifecycle, network and account topology. Ansible owns in-OS state on a host that already exists — packages, files, services, users, kernel parameters. Make the seam an artefact rather than a convention: render terraform output -json into an inventory with a script that fails when a required output is missing, and let no secret cross it: a local state file is plaintext including any secret values defined in the configuration. Then name an owner for the drift that neither tool models. That class includes properties outside every provider schema, write-only arguments inside it, in-OS state no task asserts, and anything a command task did without a creates guard. A gap with no owner is not a gap, it is an unfunded liability. --- # LM Studio vs Ollama vs llama.cpp: One Engine, Two Wrappers URL: https://stribog.com/blog/lm-studio-ollama-llama-cpp-local-model-runtimes-compared Published: 2026-09-29 Category: Sovereign AI Keywords: lm studio vs ollama, ollama vs llama-cpp, ollama vs llama cpp, local model runtime comparison, llama-server parallel slots, lm studio licence commercial use, gguf quantisation provenance, self-hosted local inference runtime Ollama now runs upstream llama-server for GGUF, and LM Studio wraps llama.cpp too. The engine is shared; the licence, provenance and defaults are not. > **The short answer** — LM Studio vs Ollama is no longer an engine choice along the GGUF path: Ollama deleted its own vendored runner in May 2026 and now spawns upstream llama.cpp's llama-server as a subprocess, and LM Studio runs llama.cpp for GGUF too. What still separates the three is everything around the engine — llama.cpp and Ollama are MIT, LM Studio is a proprietary application that happens to be free of charge, and each ships different documented concurrency and exposure defaults. Decide the licence, who mints the artefact, and the defaults you inherit — the engine question answered itself. Local inference arrives through a laptop. Somebody installs the tool with the best first five minutes, it works, and eighteen months later it is quietly serving nine people from a workstation under a desk. The comparison that decided it was about download size and onboarding. The one that matters is what happens to the licence, the model supply chain and the defaults when the second user arrives. ## One Engine, Two Wrappers: What Actually Runs the Tokens Scope this argument before making it: it holds along exactly one path. For GGUF models — the quantised single-file format most local deployments run — the process executing your tokens is the same project under all three tools. Safetensors take a different route, discussed below. The mechanism has a date. Pull request #16031 merged into Ollama on 29 May 2026 and did something more surgical than "Ollama uses llama.cpp" suggests. > "Remove the vendored GGML and llama.cpp backend, CGO runner, GGML-based Go model implementations, and sample. llama-server (built from upstream llama.cpp via FetchContent) is now the sole inference engine for GGUF-based models. (Safetensor based models continue to run on the new MLX engine.)" — ollama/ollama pull request #16031, merged 29 May 2026 Three consequences follow, each checkable in the repository rather than inferred. The relationship is a process boundary, not a library link: the doc comment on Ollama's own NewLlamaServer constructor in llm/server.go states that all GGUF models are served via the upstream llama-server subprocess. The build is pinned: a root file named LLAMA_CPP_VERSION names the exact upstream build fetched, reading b11232 at the time of writing. And the transition is not finished: llama/compat/ is documented as a temporary in-process layer for already-published Ollama GGUFs whose metadata or tensor layout upstream does not load directly, translating them in memory at load time so users need not re-pull. This is a reversal, not a constant. In May 2025 Ollama thanked the GGML team for a tensor library it reached directly from Go, calling that access "a portable way to design custom inference graphs." That is the arrangement #16031 deleted. The README still lists llama.cpp under Supported backends, as before — #16031 removed Ollama's own vendored runner, not a README entry. The proprietary vendor says the same thing in its own words. LM Studio's documentation states that it supports running LLMs on Mac, Windows and Linux using llama.cpp; separately, that on Apple Silicon Macs it also supports Apple's MLX. Its 0.4.0 release post is blunter still: parallel requests work thanks to llama.cpp's open-source continuous batching implementation, adopted into LM Studio's engine. [Figure: Topology showing LM Studio, Ollama and llama.cpp above two engines — upstream llama-server on the GGUF path and Apple MLX on the Safetensors path — with Ollama's route drawn through pull request 16031, a FetchContent build pin, a spawned subprocess boundary and a temporary compatibility layer, and a licence boundary separating the MIT components from LM Studio's proprietary application.] Resist the obvious next sentence. A shared engine project is not a shared binary, and certainly not measured parity. Ollama pins one upstream build while LM Studio numbers its own llama.cpp runtime separately — 2.0.0 at the 0.4.0 release, switched through the lms runtime CLI, with an llama.cpp 2.43.0 extension packs line appearing more recently in the changelog of LM Studio Bionic, a separately versioned agent product, not the classic app. Three numbering schemes are live at once: llama.cpp's own current stable release is v0.5.0, published 23 September 2026 at the time of writing, while the b-numbered prerelease channel Ollama's pin draws from is equally current — b11243 shipped 29 September 2026, six days later. > **The performance question is open, and the search results disagree with themselves** — Two pages ranking for these terms disagree about whether the wrapper costs you anything. thinkdifferent.blog reports from its own Apple Silicon measurements that GGUF in LM Studio gives speeds "nearly identical to Ollama, because it's the same llama.cpp underneath." codersera.com says Ollama "wraps llama.cpp in a Go process that adds ~50% overhead on Mac," citing ~89 tok/s for raw llama.cpp Metal against ~43 for Ollama on the same M4 Max. That page was published on 26 May 2026 and last updated the same day, three days before #16031 merged, so whatever it measured was the vendored-runner Ollama; it attaches no version or date to the ~89/~43 pair itself, and nothing on it has been revised since. The same pair also reappears further down the page against different silicon — a Mac mini M4 Pro rather than the M4 Max — which is a second reason to read it as an order-of-magnitude impression rather than a measurement. The architectural claim is dated and checkable; the throughput claim is neither, so treat it as open and benchmark your own hardware. ## Free, Open, and the Gap Between Them With the engine question settled, procurement's is the one left standing — and each tool answers it in a specific document. - llama.cpp — MIT. The LICENSE file, copyright "The ggml authors." Patch it, fork it, vendor it into a product, redistribute the result. - Ollama — MIT. Its LICENSE file, same grant. A fork that changes the registry default or the concurrency behaviour is yours to run and to ship. - LM Studio — a licence to use. The App Terms grant "a non-exclusive, non-transferable license to use the Software solely for Your personal and / or internal business purposes and solely in accordance with the Documentation." That is an end-user licence, not an open-source grant. Be fair about the price: the criticism is not about cost. LM Studio removed its separate commercial-licence requirement in an announcement dated 8 July 2025 — teams no longer need to get in touch before using it at work. Its command-line tool lms is even MIT, which means the automation surface is open while the application driving it is not. The gap between free and open is a gap in optionality, visible only under stress. Free tells you this quarter's invoice. Open tells you what happens when the vendor changes its terms, ships a regression you need patched this week, or stops shipping. Under MIT you fork, patch and keep running; llama.cpp even exposes a public llama library target with a public include directory in its build definition, so embedding it in your own binary is a supported shape rather than a workaround. Under an end-user licence you file a ticket and wait. Both are legitimate positions — but only one is a dependency you can service yourself, and only one survives the vendor. ## Who Mints the GGUF You Are Running A model tag is not a model. ollama pull and LM Studio's browser both resolve a friendly name to a quantised artefact somebody else produced, at a precision somebody else chose, from source weights you never saw converted. Ollama's own import documentation says plainly that it does not quantise GGUF models during import, and that you should prepare and quantise them first with a tool such as llama.cpp's llama-quantize. That decision is already outside the wrapper. Take the rest of it. llama.cpp documents both halves. convert_hf_to_gguf.py converts a Hugging Face repository straight to GGUF, and llama-quantize converts that high-precision GGUF — typically F32 or BF16 — to a quantised format of your choosing. Run both and the precision trade-off becomes yours to make deliberately rather than inherit; if you are choosing that precision against a fixed card, the VRAM sizing arithmetic is the constraint to work backwards from. ```bash #!/usr/bin/env bash # Usage: LLAMA_CPP_BUILD=b11232 ./mint-gguf.sh google/gemma-4-E2B-it Q4_K_M # Run from a BUILT llama.cpp checkout: convert_hf_to_gguf.py sits at the repo # root, llama-quantize is a build artefact (cmake -B build && cmake --build # build --target llama-quantize), and the Python conversion requirements are # installed. A packaged llama.cpp has llama-quantize on PATH but ships no # convert_hf_to_gguf.py, so the checkout is the environment that has both. set -euo pipefail REPO="${1:?usage: mint-gguf.sh [quant-type]}" QUANT="${2:-Q4_K_M}" # Record the engine build you converted WITH — conversion and inference are # the same project, and both move. Read it off your checkout or the pin. BUILD="${LLAMA_CPP_BUILD:?set to the upstream llama.cpp build tag, e.g. b11232}" OUT="$(basename "$REPO")" # llama-quantize is a build artefact, not something a fresh checkout puts on # PATH. Resolve it BEFORE the download, so a missing binary fails in a second # rather than after tens of gigabytes have moved. QUANTIZE="${LLAMA_QUANTIZE:-$(command -v llama-quantize || echo ./build/bin/llama-quantize)}" [ -x "$QUANTIZE" ] || { echo "no llama-quantize at $QUANTIZE — build it (cmake --build build" >&2 echo "--target llama-quantize) or point LLAMA_QUANTIZE at one." >&2 exit 1 } # GNU vs BSD userland: ask, do not assume. hash_file() { if command -v sha256sum >/dev/null 2>&1; then sha256sum "$1" | cut -d' ' -f1 else shasum -a 256 "$1" | cut -d' ' -f1; fi } size_of() { stat -c%s "$1" 2>/dev/null || stat -f%z "$1"; } # 1. Source weights -> high-precision GGUF. --remote streams the HF repo. python convert_hf_to_gguf.py \ --outfile "${OUT}-bf16.gguf" --outtype bf16 --remote "$REPO" # 2. High-precision GGUF -> the quantisation YOU chose, not one you were handed. # The bf16 intermediate is tens of gigabytes: delete it once step 3 verifies. "$QUANTIZE" "${OUT}-bf16.gguf" "${OUT}-${QUANT}.gguf" "$QUANT" # 3. The record. Hash the bytes that will actually be loaded. cat > "${OUT}-${QUANT}.provenance.json" </dev/null 2>&1; then sha256sum "$1" | cut -d' ' -f1 else shasum -a 256 "$1" | cut -d' ' -f1; fi } size_of() { stat -c%s "$1" 2>/dev/null || stat -f%z "$1"; } emit() { printf '%s\t%s\t%s\t%s\n' \ "$(hash_file "$2")" "$(size_of "$2")" "$1" "$2" } # Tools that store models as named files: match on the extension. scan_files() { local holder="$1" root="$2" [ -n "$root" ] && [ -d "$root" ] || { echo "skip: $holder root unset or absent" >&2; return 0; } find "$root" -type f -name '*.gguf' -print0 | while IFS= read -r -d '' f; do emit "$holder" "$f"; done } # Ollama stores layers content-addressed instead: $OLLAMA_MODELS/blobs/ # sha256-, no extension ever, model tensors and metadata layers side # by side. An extension filter finds NOTHING here and says so silently, which # is the wrong answer to give anyone about to migrate. Match the blob name, # then keep only files whose first four bytes are the GGUF magic. scan_blobs() { local holder="$1" root="$2" [ -n "$root" ] && [ -d "$root/blobs" ] || { echo "skip: $holder blobs dir unset or absent" >&2; return 0; } find "$root/blobs" -type f -name 'sha256-*' -print0 | while IFS= read -r -d '' f; do [ "$(head -c 4 "$f" | tr -d '\0')" = GGUF ] || continue emit "$holder" "$f" # filename already carries the digest; rehash anyway done # so every row in the table was computed the same way } printf 'sha256\tbytes\tholder\tpath\n' # Point these at the directories your installs actually use. scan_files llama.cpp "${LLAMA_MODELS:-}" scan_files lm-studio "${LMSTUDIO_MODELS:-}" scan_blobs ollama "${OLLAMA_MODELS:-}" ``` The long game is not choosing correctly today. All three will change again: Ollama's engine position reversed twice inside twelve months, and the pin, the runtime numbering and the compatibility layer are all explicitly transitional. Sovereignty at this layer means the durable assets sit outside whichever wrapper wins the laptop: the source weights, the conversion command, the hash of the artefact you serve, the file recording which defaults you reversed, and the test proving your client survives a runtime swap. Keep those five and the wrapper becomes what it should have been — an implementation detail you replace on a Tuesday. Same discipline the rest of a sovereign AI stack rests on, one layer further down than most teams look. ### FAQ Q: Is LM Studio the same as Ollama under the hood? A: Along the GGUF path they now run the same engine project. Ollama's pull request #16031, merged on 29 May 2026, deleted its vendored GGML and CGO runner and made llama-server — built from upstream llama.cpp via FetchContent — the sole inference engine for GGUF-based models, spawned as a subprocess rather than linked as a library. LM Studio's documentation states it runs LLMs on Mac, Windows and Linux using llama.cpp, and its 0.4.0 release post credits llama.cpp's continuous batching implementation for its parallel requests. Same project is not the same binary, though: Ollama pins one upstream build in its LLAMA_CPP_VERSION file while LM Studio versions its own llama.cpp runtime separately, and the two ship different documented concurrency and exposure defaults. Safetensors models take a different path entirely, through Apple MLX. Q: Is LM Studio open source, and can I use it commercially? A: Those are two different questions with two different answers. Commercially, yes and free: LM Studio removed its separate commercial-licence requirement in an announcement dated 8 July 2025, so using it at a company no longer requires getting in touch first. Open source, no: its App Terms grant a non-exclusive, non-transferable licence to use the software for personal or internal business purposes in accordance with the documentation, which is an end-user licence rather than a grant to modify, fork or redistribute. Its lms command-line tool is separately MIT-licensed. By contrast both llama.cpp and Ollama ship MIT LICENSE files, so you may patch, fork, vendor and redistribute either one. Free is a price; open is a grant, and only the grant tells you what happens when you need a fix the vendor will not ship. Q: Should I use llama.cpp directly instead of a wrapper? A: Use it directly when you want the things a wrapper abstracts away. You get the full documented llama-server flag surface — the slot count, the KV strategy, the per-slot context lever, the API key, the bind address, the metrics endpoint — rather than a subset a wrapper chose to expose. You get an MIT licence and a public llama library target with a public include directory, so embedding it in your own binary is a supported shape. And you control which upstream build you run instead of inheriting a pin. The cost is real: no model browser, no registry, no one-line install, and you mint and manage your own GGUF artefacts. For a shared team endpoint that is usually the right trade, because every one of those costs is a one-off and every default you inherit is permanent until somebody notices it. Q: How many concurrent users can llama-server handle, and how do I configure it? A: That depends on your model and hardware, but the controls are documented and three of them interact. The -np/--parallel flag sets the number of server slots and defaults to -1, meaning auto. The --kv-unified flag uses a single unified KV buffer shared across all sequences and is documented as enabled by default when the slot count is auto — so out of the box the KV buffer is shared rather than hard-partitioned per slot, which is not the same as the common assumption that total context is divided by the slot count. If you want a per-slot context ceiling, --kv-unified-per-slot N is the documented lever; when set without -c/--ctx-size the shared KV pool is sized to n_parallel*N. Continuous batching is separately enabled by default. Set all three explicitly rather than inheriting them. Q: Is it safe to expose LM Studio or llama-server to my team's network? A: Not on the defaults, and both projects document why. llama-server's --api-key defaults to none and its /health endpoint is documented as public with no API key check even when a key is configured, so reachability is always discoverable. Its --metrics endpoint is disabled unless enabled, leaving a shared endpoint unobservable. LM Studio does not require authentication by default; API Tokens are opt-in from the Developers Page under Server Settings, with permissions selected per token, and its own serve-on-network documentation warns that any bind other than 127.0.0.1 exposes the server beyond localhost and recommends enabling authentication first. Both bind to loopback by default, which is the one default that protects you — treat changing it as the moment every other default needs revisiting, not as a configuration detail. --- # Git, GitHub, GitLab: What You Own and What You Rent URL: https://stribog.com/blog/git-github-gitlab-forge-what-you-own-versus-rent Published: 2026-09-28 Category: Delivery Keywords: git vs github vs gitlab, github vs gitlab, github vs bitbucket vs gitlab, git forge migration portability, export github issues with gh api, github enterprise importer limitations, gitlab project export excluded items, git bundle mirror repository backup, forge exit cost inventory Git moves history, refs and tags losslessly. Issues, CI, packages and permissions move only through a lossy importer — the vendors document what it drops. > **The short answer** — Git vs GitHub vs GitLab is not a three-way product comparison — Git is a protocol and an object format you own outright, and GitHub and GitLab are tenancies over it. History, refs, tags and notes move losslessly with git clone --mirror or a git bundle; issues, CI definitions, registries, rulesets and run history move only through a per-vendor importer whose gaps the vendor publishes. The question is not which forge has the better feature table, but how much of your delivery system sits in a shape only one vendor can read. The usual comparison agrees: Git is the version control system, GitHub and GitLab are platforms on top, here is a table of runners and seat prices. All true, and none of it names what decides a migration's cost: which half of your delivery system is a format and which half a tenancy. Invisible while you stay; the whole project the day you leave. ## Git Is a Protocol; the Forge Is a Tenancy Git is a content-addressed object store plus a wire protocol. Every commit, tree and blob is named by the hash of its own contents, so a copy verifies against itself, not against the server that sent it, and any implementation of the protocol can read it. A forge is the application wrapped around that repository, with its own data model: issues, pull or merge requests, CI definitions and run history, artifacts, registries, rulesets, webhooks, runner minutes — none of it in Git. GitHub, GitLab and Bitbucket are three tenancies over one protocol whose models do not agree. The vendors' own tooling makes that concrete. GitHub Enterprise Importer publishes its supported sources — "Azure DevOps (ADO) Cloud, Bitbucket Server and Bitbucket Data Center 5.14+, GitHub.com, GitHub Enterprise Server (GHES) 3.4.1+, GitLab (GitLab.com or maintained self-hosted versions)" — and states that it "does not currently support migrations from GHE.com." A repository is universally readable. A forge's contents need a named, versioned, directional importer — and even the vendor's own has directions it will not go. The seam between the layers is the push hook. Git documents post-receive as "invoked by git-receive-pack[1] when it reacts to git push and updates reference(s) in its repository," running "once after all the proposed ref updates are processed." That is the extension point a forge replaces with hosted automation. Everything a forge sells happens on the far side of a push, which is why none of it returns when you clone. ## What Actually Moves: Refs, Objects, Tags, Signatures Take the half you own, then prove you have it. git clone --mirror is the instrument: compared with --bare it "not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.)" All refs, notes included — a copy of the repository, not its default branch. git bundle create backup.bundle --all gives the same content as one file. Bundles are "used for the 'offline' transfer of Git objects without an active 'server' sitting on the other side of the network connection," and the direction is fixed: "there's no corresponding \"write\" support, i.e. a git push into a bundle is not supported." > "But note again that this is only for the refs, i.e. you will only include refs and commits reachable from those refs. You will not include other local state, such as the contents of the index, working tree, the stash, per-repository configuration, hooks, etc." — git-bundle(1), Git documentation Read that as an inventory of what a Git-level backup is not. The stash and working tree are nobody's disaster. Hooks are, because hooks are where teams quietly put policy — and Git closes the loop: "client-side hooks are not copied when you clone a repository. If your intent with these scripts is to enforce a policy, you'll probably want to do that on the server side." On a hosted forge, the server side is the vendor's feature surface. Two things before you trust the copy. Git LFS objects are not in the mirror — GitHub documents git lfs fetch --all and git lfs push --all as separate steps — and an LFS blob is not a Git object, so the fetch lands in the mirror directory, not the bundle. Archive both, or archive half. And check the bundle, don't assume it — but not with git bundle verify, whose man page promises a nonzero exit "if the bundle file is invalid." It is scoped to "checks on the bundle format itself as well as checking that the prerequisite commits exist and are fully linked in the current repository": header and prerequisites, never the pack. On git 2.53.0 a bundle truncated mid-pack reports itself okay and exits zero; cloning from it dies. So the clone is the proof — unpack the bundle into a throwaway mirror, fsck that, delete it. ```bash #!/usr/bin/env bash # Usage: ./own-half.sh # Takes the portable half of one repository and proves the copy is complete. set -euo pipefail SRC="${1:?usage: own-half.sh }" WORK="${2:?usage: own-half.sh }" NAME="$(basename "${SRC%.git}")" mkdir -p "$WORK" cd "$WORK" # 1. All refs, not just local branches: --mirror maps remote-tracking refs and notes. git clone --mirror "$SRC" "$NAME.git" cd "$NAME.git" # 2. LFS objects are NOT in the mirror, and are NOT in the bundle either. # A missing git-lfs must be loud: a silent skip would still exit 0 here. if ! command -v git-lfs >/dev/null 2>&1; then LFS_OBJECTS='UNKNOWN - git-lfs not installed, LFS NOT CAPTURED' echo "WARNING: git-lfs not installed; LFS objects NOT captured" >&2 elif ! LFS_OBJECTS=$(git lfs ls-files --all | wc -l | tr -d '[:space:]'); then LFS_OBJECTS='UNKNOWN - git lfs ls-files failed, LFS NOT CAPTURED' echo "WARNING: git lfs ls-files failed; LFS objects NOT captured" >&2 elif [ "$LFS_OBJECTS" -gt 0 ]; then git lfs fetch --all # lands in $NAME.git, not in the bundle fi # 3. One file, offline-transferable, holding every ref and reachable commit. git bundle create "../$NAME.bundle" --all # 4. Prove the copy by reading it back. git bundle verify does NOT do this: # it checks the header and the prerequisites, and exits 0 on a bundle whose # pack is cut short or corrupt. The fsck below walks the MIRROR's objects, # not the bundle's. Only unpacking the bundle reads the bundle's pack. git bundle verify "../$NAME.bundle" # prerequisites and format header only git fsck --full --strict # the mirror's object database git clone --quiet --mirror "../$NAME.bundle" "../$NAME.restore.git" git -C "../$NAME.restore.git" fsck --full --strict # Under set -e a failed restore or fsck exits BEFORE this line, on purpose: # the broken copy stays for inspection. Remove it by hand before a rerun. rm -rf "../$NAME.restore.git" # needs room for a second copy # 5. Record what the copy does and does not contain, beside the copy. cat > "../$NAME.manifest" < **Date-stamp these lists, and check yours** — Every exclusion quoted here was read from vendor documentation on 28 September 2026. These lists are versioned: an older self-managed release shows a different one. GitLab names the artefacts itself: import_export.yml for the file-export path, and stage.rb alongside it for direct transfer, in the release you run. Read those before committing to a cutover date. ## CI Definitions Are a Rewrite, Not a Migration Pipeline definitions are a re-authoring job, not a data-migration problem. GitLab's own GitHub Actions migration guide settles it: a keyword mapping table, not a converter. It pairs env with variables, jobs with stages, runs-on with tags, steps with script and uses with include — and gives "Not applicable" for both on and run, the keywords carrying a workflow's triggers and its commands. Read that as an inventory of concepts to re-express, not a translation dictionary. The uses-to-include row is a conceptual equivalence: a marketplace action is a versioned unit of behaviour, an include pulls in YAML. Nothing sourced here supports a claim about what proportion of a pipeline translates cleanly. Assume every workflow is hand-re-authored. Which suggests the design move that shrinks the question. Keep every forge-specific concern — trigger, runner, checkout, artifact upload — in a thin outer layer and put the build in a repo-local script, which then lives in the zone the protocol carries. That discipline makes a self-hosted Forgejo and Woodpecker stack reachable rather than a rebuild, and is why self-hosted runners under ARC change where builds run, not what they are. ```yaml name: build on: push: branches: [main] pull_request: jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 # The entire build is one repo-local script. It moves with the repository. - name: Build, test and package run: ./ci/build.sh env: ARTIFACT_DIR: dist - uses: actions/upload-artifact@v7 with: name: dist path: dist/ ``` ```yaml stages: - build build: stage: build tags: - linux rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH - if: $CI_PIPELINE_SOURCE == "merge_request_event" variables: ARTIFACT_DIR: dist script: - ./ci/build.sh artifacts: paths: - dist/ ``` ## The Metered Surface: Minutes, Artifacts, Packages Part of what a forge sells is not data. It is consumption, which does not migrate — it stops, and reappears as a line item elsewhere. For private repositories on standard runners, GitHub documents included Actions minutes per month of 2,000 on Free, 3,000 on Pro and Team and 50,000 on Enterprise Cloud, and "at the start of each month, the minutes used by the account are reset to zero." Public repositories on standard runners are not billed against that quota; larger runners are charged for regardless. Storage is coupled to it: "the artifact storage amounts shown are shared with GitHub Packages," so artifacts and published packages compete for one number. Packages carry the subtler dependency. GitHub documents that "the permissions for a package are either inherited from the repository where the package is hosted, or can be defined for specific users or organizations," and that "some registries only support permissions inherited from a repository." Read that architecturally: for those registries the access model for your artifacts is a function of your repository layout on one forge. It does not travel with the images, which argues for running the registry yourself. Job logs and artifacts are the purest case. GitLab migrates the pipeline and job records — status, stage, timestamps — on the direct-transfer path, but excludes the logs and the artifacts; the file-export path likewise omits "CI/CD job traces and artifacts." If that output is evidence — provenance for a signed supply chain, or an audit input — it has to land somewhere you control while the builds run, not be extracted afterwards. ## Permissions and Audit Evidence: the Least Portable Layer Access control is where forge data models diverge most. GitHub rulesets are the sharp example: "push rules apply to the entire fork network for a repository, ensuring every entry point to the repository is protected." GitHub documents the mechanics, not the portability. The inference is this article's: a permission shape defined across a graph of repositories rather than on any one of them has no field to import it into elsewhere. GitLab confirms it from the other direction: imported GitHub branch protection rules "are mapped to one of the following: GitLab branch protection rules, Project-wide GitLab settings," with "Require a pull request before merging" arriving as the "No one" option in "Allowed to push and merge." Reasonable engineering, not equivalence — and that delta is what an auditor asks you to explain. Commit signatures sit on the boundary. The signature bytes are inside the commit or tag object, so they move with the history. The verification badge is not: it is rendered by whichever forge is reading, against whatever keys it knows. GitLab documents a known issue where "when importing projects that used GitHub auto-merge, the imported project in GitLab can have merge commits labeled unverified if the commit was signed with the GitHub internal GPG key." Nothing was lost; the evidence stopped rendering, because what vouched for it was the tenancy. Then the layer with no migration path. If your compliance position depends on showing who pushed to a protected branch two years ago, and that record exists only inside the forge you are leaving, it does not come with you. The durable version is the copy already streaming elsewhere — the argument that also governs GitOps reconciliation history across many clusters. ## Building the Exit Inventory With gh api None of this is actionable as prose. As a count it is: one row per repository, split into portable, importable with losses, and must be re-authored. Run it before you have a migration project; it tells you whether you want one. Budget disk: it mirrors every repository in the org. Three documented traps shape the script. The first silently double-counts: "GitHub's REST API considers every pull request an issue, but not every issue is a pull request. For this reason, 'Issues' endpoints may return both issues and pull requests in the response. You can identify pull requests by the pull_request key." Counts read straight off that endpoint are issues plus pull requests, and nothing surfaces the error until reconciliation. Rulesets repeat the shape: includes_parents defaults to true, so a per-repository query also returns org-level rulesets and an org-wide sweep counts each once per repository. Pass includes_parents=false; count org rulesets once, against orgs/{org}/rulesets. The second is the primary budget. A personal access token gets 5,000 REST requests per hour; GitHub documents 15,000 for requests made on your behalf by a GitHub App owned by a GitHub Enterprise Cloud organization. The arithmetic: at eight sweeps per repository several hundred repositories approach a personal token's budget, fewer still once a sweep paginates past its first 100 items — read /rate_limit rather than guess. The third stops a naive sweep within a minute. On top of the hourly budget sit secondary limits: "no more than 100 concurrent requests are allowed. This limit is shared across the REST API and GraphQL API," and a per-endpoint ceiling — the rule is against "too many requests to a single endpoint per minute," where "no more than 900 points per minute are allowed for REST API endpoints." A fan-out across repositories hammers one endpoint pattern, tripping those long before anything hourly runs out. The correct shape is boring on purpose: serial pagination, a pause between sweeps, back-off on rejection. ```bash #!/usr/bin/env bash # Usage: ./rented-half.sh # Exports forge metadata to JSONL. Serial by design: the secondary limits cap # concurrency at 100 requests and REST endpoints at 900 points per minute. # Token: the hooks sweep is admin-gated. Fine-grained needs Webhooks (read), # documented at the List endpoint; classic tokens conventionally need # read:repo_hook or repo. Without it hooks is rejected, not empty. set -euo pipefail REPO="${1:?usage: rented-half.sh }" OUT="${2:?usage: rented-half.sh }" PAUSE="${PAUSE_BETWEEN_SWEEPS:-20}" mkdir -p "$OUT" budget() { # 5,000/hour on a personal token; 15,000/hour for an app owned by a # GitHub Enterprise Cloud organization. Read it, do not assume it. gh api /rate_limit --jq '.resources.core | "\(.remaining)/\(.limit)"' } sweep() { local name="$1" path="$2" attempt for attempt in 1 2; do # Write .partial first: a failed sweep must not leave a short .jsonl on # disk shaped exactly like a complete export. if gh api --paginate --jq '.[]' "$path" > "$OUT/$name.jsonl.partial"; then mv "$OUT/$name.jsonl.partial" "$OUT/$name.jsonl" printf '%-18s %6s objects budget %s\n' \ "$name" "$(wc -l < "$OUT/$name.jsonl")" "$(budget)" >&2 sleep "$PAUSE" return 0 fi printf '%s: rejected on attempt %s, backing off 60s\n' "$name" "$attempt" >&2 sleep 60 done rm -f "$OUT/$name.jsonl.partial" # leave no short file beside the good ones return 1 } failed=() echo "starting budget: $(budget)" >&2 sweep issues_and_pulls "repos/$REPO/issues?state=all&per_page=100" || failed+=(issues_and_pulls) sweep issue_comments "repos/$REPO/issues/comments?per_page=100" || failed+=(issue_comments) sweep pulls "repos/$REPO/pulls?state=all&per_page=100" || failed+=(pulls) sweep labels "repos/$REPO/labels?per_page=100" || failed+=(labels) sweep milestones "repos/$REPO/milestones?state=all&per_page=100" || failed+=(milestones) sweep releases "repos/$REPO/releases?per_page=100" || failed+=(releases) sweep hooks "repos/$REPO/hooks?per_page=100" || failed+=(hooks) # includes_parents defaults to TRUE. Leave it alone and every org-level ruleset # is written into every repository's file, so an importer replays one ruleset N # times. Org rulesets belong to the org: export them once, from orgs//rulesets. sweep rulesets "repos/$REPO/rulesets?per_page=100&includes_parents=false" \ || failed+=(rulesets) if [ "${#failed[@]}" -gt 0 ]; then printf 'incomplete sweeps: %s — rerun these alone, not the whole script\n' \ "${failed[*]}" >&2 exit 1 fi # Only past the failure check: "Issues" endpoints return pull requests too. # Split on the pull_request key or every count downstream is wrong in the same # direction. A short file must never produce a plausible-looking split. jq -c 'select(has("pull_request") | not)' \ < "$OUT/issues_and_pulls.jsonl" > "$OUT/issues_only.jsonl" printf 'issues %s, pull requests returned as issues %s\n' \ "$(wc -l < "$OUT/issues_only.jsonl")" \ "$(( $(wc -l < "$OUT/issues_and_pulls.jsonl") - $(wc -l < "$OUT/issues_only.jsonl") ))" ``` Aggregate per repository and the inventory writes itself: the Git side counted from a mirror, the forge side through the API. Left columns portable, right columns backlog. ```bash #!/usr/bin/env bash # Usage: ./exit-inventory.sh > exit-inventory.csv # A rejected API request is never a zero here: the cell reads ERR, the repo is # named on stderr, and the script exits non-zero. Local tooling that cannot run # is a DIFFERENT failure: it reads UNKNOWN and does not fail the run. # Token: hooks is admin-gated (fine-grained Webhooks read; classic # read:repo_hook or repo). orgs//rulesets wants fine-grained # Administration (WRITE) at org level for one stderr line — drop that count # rather than escalate a read-only token, if that trade is the wrong one. # Run gh auth setup-git first: the mirror clone authenticates through git's # credential helper, not the gh token, so a private org prompts or fails. # Disk: $WORK ends up holding a full mirror clone of every repository. set -euo pipefail ORG="${1:?usage: exit-inventory.sh }" WORK="${2:?usage: exit-inventory.sh }" mkdir -p "$WORK" incomplete=() # API rejections — fatal tooling=() # local commands that could not run — reported, not fatal stale=() # mirrors that would not refresh — reported, not fatal # Probe git-lfs ONCE. An absent binary is not a rejected request and must not # name every repository in the org as incomplete. LFS_TOOL=1 if ! command -v git-lfs >/dev/null 2>&1; then LFS_TOOL=0 echo "WARNING: git-lfs not installed; lfs_objects reads UNKNOWN in every row" >&2 fi # Back off on rejection, as the sweep script does, then give up loudly. api() { local attempt out for attempt in 1 2 3; do if out=$(gh api "$@"); then printf '%s' "$out" return 0 fi if [ "$attempt" -lt 3 ]; then sleep $((attempt * 30)) # no dead wait after the final attempt fi done return 1 } count() { local out out=$(api --paginate --jq '.[].id' "$1") || return 1 printf '%s\n' "$out" | grep -c . || true } repos=$(api --paginate --jq '.[].full_name' "orgs/$ORG/repos?per_page=100") || { echo "FATAL: could not list repositories for $ORG" >&2 exit 1 } # Org-level rulesets exist once, not once per repository. Count them here, and # pass includes_parents=false below so they are not re-counted in every row. if org_rulesets=$(count "orgs/$ORG/rulesets?per_page=100"); then printf 'org-level rulesets (counted once, not per repository): %s\n' \ "$org_rulesets" >&2 else echo "WARNING: could not count org-level rulesets for $ORG" >&2 fi printf 'repo,size_kb,refs,tags,lfs_objects,issues,pulls,releases,rulesets,hooks,workflows\n' while read -r repo; do [ -n "$repo" ] || continue name="${repo##*/}" bad=0 # Re-running against the same workdir is the designed path, so REFRESH the # mirror rather than reuse it. Skipping this reports last run's size, refs # and tags beside today's live API counts: a wrong row, not an old one. # (git remote update rejects --quiet; a mirror's refspec is +refs/*:refs/*, # so a pruning fetch of origin is the same update.) if [ -d "$WORK/$name.git" ]; then git -C "$WORK/$name.git" fetch --prune --quiet origin || { echo "WARNING: mirror refresh failed for $repo; Git columns are stale" >&2 stale+=("$repo") } else git clone --quiet --mirror "https://github.com/$repo.git" "$WORK/$name.git" fi # Owned side: counted from the object database, no API, no rate limit. # tr strips the leading spaces BSD wc pads counts with; GNU wc does not. refs=$(git -C "$WORK/$name.git" for-each-ref --format='%(refname)' | wc -l | tr -d '[:space:]') tags=$(git -C "$WORK/$name.git" tag --list | wc -l | tr -d '[:space:]') size=$(du -sk "$WORK/$name.git" | cut -f1 | tr -d '[:space:]') if [ "$LFS_TOOL" -eq 0 ]; then lfs=UNKNOWN elif ! lfs=$(git -C "$WORK/$name.git" lfs ls-files --all | wc -l | tr -d '[:space:]'); then lfs=UNKNOWN tooling+=("$repo") fi # Rented side: split issues from pull requests on the documented key. if kinds=$(api --paginate "repos/$repo/issues?state=all&per_page=100" \ --jq '.[] | has("pull_request")'); then issues=$(grep -c '^false$' <<<"$kinds" || true) pulls=$(grep -c '^true$' <<<"$kinds" || true) else issues=ERR pulls=ERR bad=1 fi releases=$(count "repos/$repo/releases?per_page=100") || { releases=ERR; bad=1; } # includes_parents defaults to TRUE: without this, three org rulesets on a # 200-repo org emit 600 rows. Per-repository rulesets only. rulesets=$(count "repos/$repo/rulesets?per_page=100&includes_parents=false") || { rulesets=ERR; bad=1; } hooks=$(count "repos/$repo/hooks?per_page=100") || { hooks=ERR; bad=1; } workflows=$(api "repos/$repo/actions/workflows" --jq '.total_count') || { workflows=ERR; bad=1; } [ "$bad" -eq 0 ] || incomplete+=("$repo") printf '%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s\n' \ "$repo" "$size" "$refs" "$tags" "$lfs" "$issues" "$pulls" \ "$releases" "$rulesets" "$hooks" "$workflows" sleep 5 done <<<"$repos" if [ "${#tooling[@]}" -gt 0 ]; then printf 'LFS NOT COUNTED - local git lfs could not run, nothing was rejected: %s\n' \ "${tooling[*]}" >&2 fi if [ "${#stale[@]}" -gt 0 ]; then printf 'STALE GIT COLUMNS - mirror refresh failed, size/refs/tags predate this run: %s\n' \ "${stale[*]}" >&2 fi if [ "${#incomplete[@]}" -gt 0 ]; then printf 'INCOMPLETE ROWS - an API request was rejected, those counts are not zeroes: %s\n' \ "${incomplete[*]}" >&2 exit 1 fi ``` ## Pricing the Exit The inventory gives units, not a number: converting them to hours is a property of your team, not the forge. Any total arriving without your own blended rate is a guess wearing a decimal point. - Pipelines to re-author — one per workflow file, weighted by how much logic sits in the YAML rather than a script. - Protected branches, rulesets and CODEOWNERS — redesigned rather than mapped, re-reviewed by whoever signs off access. - Registries — every package and image re-published, because both GitLab paths exclude registry contents. - Webhooks, deploy keys and tokens — every forge credential, all on the exclusion lists. - Repositories over the documented ceiling — release assets near 10 GB, against GitHub Enterprise Importer's documented limit. - Metadata that will not import cleanly — the delta between your export counts and what the destination documents it accepts. Published seat prices belong here, but only for what they price: the recurring cost of staying. Read from the pricing pages on 28 September 2026, GitHub lists Team at $4 USD per user/month and Enterprise at "Starting at $21 USD per user/month" — each figure printed twice at the same value, once plain and once footnoted "for the first 12 months." GitLab lists Premium at $29 per user/month billed annually, a seat price rather than a platform total: the same page carries $12 of included GitLab Credits per user/month against Premium and $24 against Ultimate, both footnoted as a limited-time promotional offer, so AI usage past the allowance meters on top of the seat at "the standard rate of $1/credit." Ultimate itself reads "Get custom pricing." Which makes the question tractable: a recurring public cost of staying against a one-off countable cost of leaving, shrinking each time you move a piece from the outer zone to the inner — the generic exit-cost model applied to a system whose vendors publish their limits. [Figure: Eight-step forge exit sequence, each step annotated with what it does not carry. Stand up the destination and take a mirror plus bundle, proven by cloning the bundle back and running git fsck on that copy, because git bundle verify checks only the prerequisites and the format header. Export metadata through the API paced against published primary and secondary rate limits. Import metadata and diff the counts. Re-author CI and prove parity on a canary repository. Re-publish packages and images. Re-author rulesets, branch protection and CODEOWNERS. Flip the push mirror so the old forge becomes the replica. Cut over webhooks, deploy keys and tokens. Three dashed abort paths hang off the sequence: a failed restore clone or fsck, a secondary-limit rejection requiring back-off and resume, and a canary parity failure that rolls back to the re-authoring step.] ## Designing So the Exit Stays Cheap Run a continuous mirror, and know what it covers. Forgejo documents push mirrors to GitHub, GitLab and Bitbucket, so a self-hosted forge can hold the authoritative copy while the commercial one stays a replica. The caveat: a mirror carries Git data only. Forgejo's migration imports releases and their assets; the request to have that run continuously was closed, its own author noting the feature "does already exist, but is mutually exclusive with a code mirror." A live code mirror therefore carries history alone; pair every one with a scheduled release export against the API. Even mirroring touches the permission model: a push mirror to GitHub needs Contents permission, plus Workflows if the repository holds anything under .github/workflows. Keep CI logic in repo-local scripts. Every line of build behaviour inside the repository moves by protocol; every line in the workflow YAML is re-authored. It turns the largest re-authoring bucket into the smallest, at no cost on the next pipeline you touch. Publish to a registry you could re-point. Registry contents are excluded from both GitLab migration paths and nothing carries them for you. A registry you operate, or one addressed through a name you control, turns an image migration into a DNS change. The logic extends to developer environments: anything living only inside the forge's runtime is something you rebuild. Export metadata on a schedule, not on an incident. The export script above belongs in a cron job writing to storage you control. A few thousand API calls a week turns the outer zone from a hostage into a dated copy, so leaving argues over a backlog you can count, not over whether migration is possible. The long game is the simplest part. Git's object format has already outlived SourceForge's dominance, Google Code, Gitorious and Bitbucket's Mercurial support; on that record, a team still running in 2040 that never changed forge would be the exception. Not a forecast about which vendor fails — an observation about how tenancies end. The protocol is the durable layer; the tenancy is a lease renewed monthly, on somebody else's schedule. ### FAQ Q: What is the difference between Git, GitHub and GitLab? A: Git is a distributed version control system — a content-addressed object format plus a wire protocol, implemented by many programs and owned by nobody. GitHub and GitLab are hosted platforms built over that protocol, each adding its own data model: issues, pull or merge requests, CI/CD definitions and run history, package and container registries, permission systems, webhooks. The practical consequence is an ownership boundary, not a feature comparison. Anything in the Git layer moves to any other implementation losslessly with a mirror clone or a bundle, and is verifiable against its own hashes. Anything in the platform layer lives in one vendor's schema and moves only through that vendor's importer, which both GitHub and GitLab document with explicit exclusion lists. Bitbucket sits in the same position, a third tenancy over the same protocol. So the choice is not about which has the better runner or AI assistant; it is a decision about how much of your delivery system you are willing to hold in a shape only one vendor can read. Q: Does a git clone --mirror back up everything in my repository? A: It backs up the Git layer completely and nothing else. Git documents that --mirror "not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.)," so a mirror is a full copy of the history, branches, tags and notes. Two gaps matter. First, Git LFS objects are not included — GitHub's own duplicate-a-repository procedure lists git lfs fetch --all and git lfs push --all as separate steps. Second, and much larger: none of the forge data model is in there: no issues, no pull requests or review threads, no CI definitions or run history, no packages, no rulesets, no webhooks. A mirror plus a git bundle is the right backup of what Git owns, and Git's bundle documentation is explicit that even that excludes the index, the working tree, the stash, per-repository configuration and hooks. Treat the mirror as one of two backups you need, not as the backup. Q: What does not transfer when migrating from GitHub to GitLab, or the other way? A: Both vendors publish the answer, which is the useful part. As documented on 28 September 2026, GitLab's direct-transfer path excludes CI/CD job logs, CI/CD variables, container registry images, deploy keys and tokens, encrypted tokens, job artifacts, pipeline schedule variables, pipeline triggers and webhooks on sensitivity grounds, and lists the container registry, package registry, environments, feature flags, merge request approval rules and dependencies, remote mirrors and wiki comments, among others, as not supported — though approval rules related to project settings are imported. Users themselves, and any personal access tokens they created, are excluded. GitLab's file-export path additionally omits CI/CD job traces and artifacts, security policies and activity logs for Git-related events such as pushes and tag creation. Going the other way, GitHub Enterprise Importer documents that a repository with more than 10 GB of releases data cannot have its releases migrated at all through GitHub Enterprise Importer, with --skip-releases as the workaround. These lists are versioned, so check the documentation for the version you are running. Q: How do I export all my GitHub issues before leaving? A: Use gh api with --paginate, and account for three documented behaviours or your numbers will be wrong. First, GitHub's REST API "considers every pull request an issue," so the Issues endpoints return both and you must split them on the pull_request key — otherwise every issue count you report is inflated by your pull requests. Second, pace the sweep against the primary rate limit: 5,000 requests per hour on a personal access token, or 15,000 for requests made by a GitHub App owned by a GitHub Enterprise Cloud organization. Third, the one that actually breaks naive scripts: respect the secondary limits — GitHub allows no more than 100 concurrent requests across REST and GraphQL, and, per endpoint per minute, no more than 900 points for REST endpoints. A parallel fan-out across repositories hits those within a minute. Paginate serially, pause between endpoint sweeps, read /rate_limit rather than assume a budget, and back off on rejection rather than retry immediately. Q: Is self-hosting a forge cheaper than GitHub or GitLab? A: That is the wrong comparison to lead with, because the two costs have different shapes. Seat pricing is recurring and public: read on 28 September 2026, GitHub lists Team at $4 USD per user/month and Enterprise at "Starting at $21 USD per user/month," each shown beside a footnoted first-twelve-months figure, while GitLab lists Premium at $29 per user/month billed annually with $12 of promotional included credits alongside it, and Ultimate as quote-only. Self-hosting replaces that with hardware, operational attention and an upgrade cadence you own. The more decision-relevant number is the one nobody publishes: the cost of re-authoring the part of your delivery system that only exists in one vendor's data model. That cost is countable — pipelines, rulesets, registries, webhooks — and it shrinks every time you move a piece into the layer the protocol carries. Run the inventory first. A team whose build logic already lives in repo-local scripts and whose images already go to a registry it controls is looking at a very different number from one whose pipelines are a thousand lines of vendor-specific YAML. --- # ISO 27001 vs SOC 2: Certificate or Auditor's Opinion URL: https://stribog.com/blog/iso-27001-vs-soc-2-certification-versus-attestation-report Published: 2026-09-27 Category: Compliance Keywords: iso 27001 versus soc 2, soc 2 vs iso 27001, soc 2 report vs iso 27001 certificate, soc 2 type 2 report period, iso 27001 certificate, soc 2 attestation vs certification, iso 27001 and soc 2 dual compliance, iso 27001 soc 2 evidence reuse ISO 27001 ends in an accredited certificate on a cycle; SOC 2 Type 2 ends in an auditor's opinion over a past period. What each one lets you prove. > **The short answer** — ISO 27001 versus SOC 2 is not a choice between two grades of one audit — the programmes terminate in structurally different artefacts. ISO 27001 ends in a certificate from an accredited certification body, on a cycle an accreditation body's restatement of ISO/IEC 17021-1:2015 puts at three years by default. A SOC 2 Type 2 engagement ends in no certificate at all, but a practitioner's opinion on suitability of design and operating effectiveness "throughout a specified period" that has already closed. Build the evidence generators once; only the two descriptive documents are irreducibly audit-specific. Search the comparison and you get control-mapping tables: Annex A 8.9 beside CC7.1, a percentage of claimed overlap, a closing note on which your market wants. They are not wrong. They answer a question nobody is stuck on, and skip the one that decides what you build — what lands on the desk, and what it can say. ## Two Artefacts, Not Two Flavours of One Audit There is no such thing as being "SOC 2 certified." The AICPA's attestation standards, as amended by SSAE No. 21, define the engagement a SOC 2 sits in as an assertion-based examination engagement: one in which "the practitioner obtains reasonable assurance by obtaining sufficient appropriate evidence about the responsible party's measurement or evaluation of the underlying subject matter against criteria in order to be able to draw reasonable conclusions on which to base the practitioner's opinion." That definition, at AT-C section 105.12, holds the answer: the deliverable is an opinion, expressed rather than conferred, about something already past. "Assertion-based" names who speaks first — management measures and asserts, the practitioner opines on that assertion. ISO 27001 is the other shape: a management-system standard whose conformity is certified. A third-party body audits the management system, a decision is issued, and out comes a document with a scope and an expiry. It enumerates no controls tested and no findings. It asserts a status. The distinction is the one separating a certifiable standard from its guidance document one level down: certification and attestation are different instruments, not different rigour settings. The criteria themselves also differ in kind. The trust services criteria are, in the AICPA's own words, "control criteria established by the Assurance Services Executive Committee (ASEC) of the AICPA for use in attestation or consulting engagements" — a professional body's criteria for its members' engagements, not a standard published by a standards body, and not something an organisation is certified against. ISO 27001 is the latter. [Figure: Flow diagram: four shared evidence generators feed one dated hashed store, which two mappings read — one to trust services criteria, one to ISMS clauses and Statement of Applicability entries. The SOC 2 path runs management's system description through an examination engagement to an opinion with tests and results plus a conditional restricted-use alert. The ISO path runs the ISMS scope and SoA through a certification-body audit and a separate decision to a certificate carried on a cycle, hanging from an accreditation chain.] ## What the Auditor's Opinion Actually Covers Read the opinion page of a Type 2 report as scope limits, not a verdict. The trust services criteria state what the engagement covers: "the suitability of design and operating effectiveness of controls included in management's description of a service organization's system relevant to one or more of the trust services criteria ... throughout a specified period." Four constraints sit in that sentence: the controls are management's own, the criteria only those the engagement names, operating effectiveness is in scope alongside design, and all of it is bounded by a period. The same document supplies the reason a Type 2 is worth reading: it "also includes a detailed description of tests of controls performed by the service auditor and the results of those tests" — the substance, naming what was sampled and what happened. A Type 1 has neither half: it "does not contain an opinion on the operating effectiveness of controls nor a detailed description of tests of controls ..." A buyer asking for evidence that controls ran wants a Type 2, whatever the request email says. A SOC 3 keeps the opinion but drops the tests and results, which is what makes it publishable. Then the part that surprises procurement. Under AT-C section 205.64 the report "should include an alert, in a separate paragraph, that restricts the use of the report" in listed circumstances — among them where "the criteria used to evaluate the subject matter are available only to specified parties." Paragraph .65 then requires that alert to state the report is "intended solely for the information and use of the specified parties," identify them, and state it "is not intended to be, and should not be, used by anyone other than the specified parties." Conditional, not a blanket rule — read the alert paragraph in the report you hold. > **Sourcing note, and an edition caveat** — Provenance differs by quote. The AT-C section 105 definition and its paragraph number come from the current text as amended by SSAE No. 21, published openly by the AICPA: that amendment renumbered the definitions paragraph to .12, named the engagement assertion-based, and superseded AT-C section 205 for reports dated on or after 15 June 2022. The restricted-use paragraphs .64 and .65 and the qualified-opinion requirement survived word for word — only the qualified-opinion number moved, .70 to .72, cited here at .72. The trust services criteria quotes are an Internet Archive snapshot of the 2017 edition, which "includes March 2020 updates" rather than the later revised-points-of-focus refresh — nothing quoted here falls in that delta. ## What the Certificate Actually Says A certificate asserts that an information security management system, inside a stated scope, conforms to the standard. Read what it leaves out: no sample sizes, no exceptions, no population. The audit behind it was a management-system audit, and the document is a status rather than a result set. Which controls were tested last year is a question it was never built to answer — that answer lives in the Statement of Applicability and the ISMS records behind it. Its weight comes from the chain above the issuing body, which is itself accredited against ISO/IEC 17021-1, the general standard for certification bodies. Accreditation bodies run mutual-recognition arrangements so a certificate travels between jurisdictions, and that layer consolidated recently: Global Accreditation Cooperation Incorporated "has commenced full operations as of 1 January 2026, including the launch of its own Multilateral Recognition Arrangement (MRA)" — a change Stribog's earlier ISO 27001 article covers. An unaccredited certificate is a different artefact in similar typography. One detail inside the certification body is the closest ISO analogue to auditor independence. Per an accreditation body's published restatement of ISO/IEC 17021-1:2015 — the International Accreditation Service's abridged Section 9, used here because the standard is paywalled — clause 9.5.1.1 requires that "Decision makers are different from those who carried out the audits ..." The people who audited you do not decide your certification, and that separation sits in the rules governing certification bodies, not in your ISMS. The certificate does record which edition it is against, and the 2013 transition is closed. IAF MD 26, the mandatory document that governed it, set in its key-timescale table a deadline for transitions of certified clients at 36 months from the last day of the publication month of ISO/IEC 27001:2022 — 31 October 2025 — and required that "All certifications based on ISO/IEC 27001:2013 shall expire or be withdrawn at the end of the transition period." So an in-date accredited certificate today is a 2022-edition certificate. ## The Two Calendars Are the Real Difference The ISO calendar first. Clause 9.1.3.2 of that accreditation-body restatement sets the programme as "two-stage initial audit, surveillance audits in the first and second years following the certification decision, and a recertification audit in the third year prior to expiration of certification," and fixes the origin: "The first three-year certification cycle begins with the certification decision." Not the audit date, not the certificate's print date. Clause 9.1.3.3 sets the cadence: surveillance "at least once a calendar year, except in recertification years," with the first "not be more than 12 months from the certification decision date." Those are not the same constraint: a calendar year alone would permit a January decision and a December surveillance 23 months later, and the 12-month ceiling forbids it. Three years is the general default, not a universal rule. NOTE 3 to clause 9.1.3.2 records that "If specified by the industry specific certification scheme, the certification cycle can be different from three years." Confirm the length against your certification body's programme, not against an article. Audit duration for an ISMS comes from ISO/IEC 27006-1:2024 and that programme too — and where exact wording reaches a contract, buy the standard. The SOC 2 calendar has no cycle, only a sequence of closed periods with an issuance lag behind each. Which raises the question the mapping tables never reach: what does this report say about today? Nothing. The further the question date drifts from the period's end, the less of your current posture it touches. A bridge letter from management narrows the gap but is not an examination and carries no opinion. The honest answer to "are your controls operating?" cites the last closed period and its end date, then says what has changed since. Moving certification body does not restart the clock: IAF MD 2 requires that where the pre-transfer review finds no problems, "the certification cycle shall be based on the previous certification cycle and the accepting certification body shall establish the audit programme for the remainder of the certification cycle." You inherit your position in the cycle, expiry included. Stribog's ISO/IEC 42001 certification-audit article works those transfer clauses on the same 17021-1 machinery; the point here is that the cycle belongs to the certification, not the certifier — which makes changing bodies a move rather than a restart. [Figure: Two calendars on one axis. The SOC 2 row shows consecutive closed report periods, a report issued after the period end, then an uncovered gap before the question date. The ISO row shows the certification decision as the origin, first surveillance no more than 12 months later, second-year surveillance, and recertification in year three before expiry, with a note that an industry scheme may set a different cycle length. Both rows feed a retention-sizing rule: longer than any single report period, and further than the cycle's far edge.] ## One Evidence Pipeline, Two Artefact Sets Here is the payoff the percentage-of-overlap framing obscures: the question is not what fraction of controls coincide, but which artefacts machines generate and which humans write. An audit event recording who deleted a secret at 03:14 UTC is the same fact whether a service auditor samples it inside a period or a certification-body auditor reads it as ISMS operation. Build the generators once, land them in one dated store, let two mappings read it differently. Start where both programmes start: the control plane's record of who did what. Audit policy decides whether an access event is admissible evidence or merely a count, and rule ordering is the whole design — "When an event is processed, it's compared against the list of rules in order. The first matching rule sets the audit level of the event." The levels are None, Metadata, Request and RequestResponse, and a broad Metadata catch-all above a specific rule silently downgrades exactly the event you will need. ```yaml # Passed to kube-apiserver as: # --audit-policy-file=/etc/kubernetes/audit/audit-policy.yaml # # Rule order IS the policy. The first matching rule wins, so every # narrow high-value rule must appear ABOVE the broad catch-all. apiVersion: audit.k8s.io/v1 kind: Policy omitStages: - RequestReceived rules: # 1. Drop the highest-volume, zero-evidence-value chatter first, so it # never reaches the rules below and never consumes retention. - level: None users: - system:kube-scheduler - system:kube-controller-manager - system:kube-proxy verbs: ["get", "list", "watch"] - level: None nonResourceURLs: - /healthz* - /livez* - /readyz* - /version - /metrics # 2. Secret and credential access: metadata only, deliberately. # RequestResponse here would write the secret material into the # audit log, turning the evidence store into a credential store. - level: Metadata resources: - group: "" resources: ["secrets", "serviceaccounts/token"] - group: "authentication.k8s.io" resources: ["tokenreviews"] # 3. Authorisation changes: full request AND response. This is the # access-review trail both programmes reconcile against, and # "who granted cluster-admin, and what exactly did they grant" # is unanswerable from metadata alone. - level: RequestResponse resources: - group: "rbac.authorization.k8s.io" resources: - roles - rolebindings - clusterroles - clusterrolebindings # 4. Interactive access to a running workload — the event that most # often needs explaining, and the one least often logged usefully. - level: RequestResponse resources: - group: "" resources: - pods/exec - pods/attach - pods/portforward # 5. Policy and admission configuration: changing the guard is itself # an auditable act. - level: RequestResponse resources: - group: "admissionregistration.k8s.io" resources: ["validatingwebhookconfigurations", "mutatingwebhookconfigurations"] - group: "kyverno.io" resources: ["clusterpolicies", "policies"] # 6. Everything else that mutates: bodies kept, reads excluded. - level: Request verbs: ["create", "update", "patch", "delete", "deletecollection"] # 7. Residual catch-all. Last, always. - level: Metadata ``` Retention decides whether any of it is evidence when somebody asks. The three kube-apiserver flags are familiar — --audit-log-maxage for days retained, --audit-log-maxbackup for file count, --audit-log-maxsize for rotation size — and the question is what number goes in them. The defensible floor is the SOC 2 one: the longest period you will be examined over, plus the issuance lag behind it. A period longer than the window leaves its early months unevidenced — controls may have operated perfectly and nothing survives to sample. Reaching a cycle's far edge is a design choice, not a requirement: neither ISO 27001 nor the 17021-1 restatement asks for three years of raw API events, since a surveillance audit samples current ISMS operation. Treat it as conservative headroom, then check maxbackup and maxsize cannot end the window early. A SIEM you operate decouples it from local disk. ```yaml # kubeadm-config.yaml # kubeadm init --config kubeadm-config.yaml # kubeadm upgrade apply --config kubeadm-config.yaml # # maxage here is a conservative upper bound rather than a requirement: # it reaches past the far edge of a three-year certification cycle. The # defensible floor is smaller - the longest report period plus the # issuance lag behind it. maxbackup and maxsize are then sized so # rotation cannot expire the window before maxage does: 400 files x # 200 MB = 80 GB, which is about 66 MB of audit volume per day across # the 1200 days. Note that maxsize is the only rotation # trigger kube-apiserver has; there is no time-based equivalent. apiVersion: kubeadm.k8s.io/v1beta4 kind: ClusterConfiguration kubernetesVersion: v1.36.0 clusterName: sovereign-prod controlPlaneEndpoint: api.cluster.example.com:6443 apiServer: extraArgs: - name: audit-policy-file value: /etc/kubernetes/audit/audit-policy.yaml - name: audit-log-path value: /var/log/kubernetes/audit/audit.log - name: audit-log-maxage value: "1200" - name: audit-log-maxbackup value: "400" - name: audit-log-maxsize value: "200" extraVolumes: - name: audit-policy hostPath: /etc/kubernetes/audit mountPath: /etc/kubernetes/audit readOnly: true pathType: DirectoryOrCreate - name: audit-log hostPath: /var/log/kubernetes/audit mountPath: /var/log/kubernetes/audit pathType: DirectoryOrCreate etcd: local: dataDir: /var/lib/etcd ``` The rest of the shared set follows the same rule: dated, hashed, indexed once. Backups are the clearest case, because a backup without a restore result is an assertion rather than evidence, and because the dating has a trap in it. Velero lets the time zone live in the schedule — "The format is CRON_TZ= " — and its v1.18 documentation gives the reason: "Specifying timezones can reduce disputes in the case of daylight saving time changes." ```bash #!/usr/bin/env bash # Collect one day's evidence artefacts into a dated, hashed index. # # Programme-agnostic by design: this script knows nothing about trust # services criteria or ISMS clauses. It produces facts with dates and # digests. Mapping those facts to either programme happens downstream. set -euo pipefail ROOT="${EVIDENCE_ROOT:-/srv/evidence}" COLLECTED_AT="$(date -u +%Y-%m-%dT%H:%M:%SZ)" DAY="${COLLECTED_AT%%T*}" OUT="${ROOT}/${DAY}" INDEX="${OUT}/index.jsonl" mkdir -p "${OUT}" : > "${INDEX}" # Emit one index row. Digest is computed from the bytes on disk, so a # later re-hash either reproduces it or proves the artefact changed. index_row() { local path="$1" kind="$2" local digest bytes digest="$(sha256sum "${path}" | cut -d' ' -f1)" bytes="$(wc -c < "${path}" | tr -d '[:space:]')" printf '{"path":"%s","kind":"%s","collected_at":"%s","sha256":"%s","bytes":%s}\n' \ "${path#"${ROOT}/"}" "${kind}" "${COLLECTED_AT}" "${digest}" "${bytes}" >> "${INDEX}" } collect() { local name="$1" kind="$2" shift 2 local dest="${OUT}/${name}" if "$@" > "${dest}.partial" 2> "${dest}.err"; then mv "${dest}.partial" "${dest}" rm -f "${dest}.err" index_row "${dest}" "${kind}" else # Fail loudly per artefact, but do not abandon the run: a missing # artefact discovered now is repairable, one discovered at audit # time is a gap in a closed period. echo "collect FAILED: ${name} ($(tr -d '\n' < "${dest}.err" | cut -c1-200))" >&2 rm -f "${dest}.partial" fi } # Access review: every binding that can escalate, as at this timestamp. collect rbac-clusterrolebindings.json access_review \ kubectl get clusterrolebindings -o json collect rbac-rolebindings.json access_review \ kubectl get rolebindings --all-namespaces -o json # Policy reports: admission and background results, per namespace. collect policy-reports.json policy_report \ kubectl get policyreports,clusterpolicyreports --all-namespaces -o json # Backup state, including the restore results that make a backup # evidence rather than an intention. collect velero-backups.json backup \ velero backup get -o json collect velero-restores.json restore_result \ velero restore get -o json # Control-plane audit trail. kube-apiserver has exactly one rotation # trigger, --audit-log-maxsize, and names each backup after the # rotation instant. So on any cluster below that threshold no dated # file appears for days, and the day's events are still sitting in the # open segment. Collect both: rotated files carrying today's date, and # the live log. audit_rows=0 for f in /var/log/kubernetes/audit/audit-"${DAY}"*.log; do [ -e "${f}" ] || continue cp "${f}" "${OUT}/$(basename "${f}")" index_row "${OUT}/$(basename "${f}")" audit_log audit_rows=$((audit_rows + 1)) done # The open segment, under a name that says so: it was live when copied # and it may overlap tomorrow's copy. if [ -e /var/log/kubernetes/audit/audit.log ]; then cp /var/log/kubernetes/audit/audit.log "${OUT}/audit-open-${DAY}.log" index_row "${OUT}/audit-open-${DAY}.log" audit_log audit_rows=$((audit_rows + 1)) fi # Seal the day: a digest over the index itself, so a later reader can # prove the row set was not edited after collection. sha256sum "${INDEX}" | cut -d' ' -f1 > "${OUT}/index.sha256" echo "collected $(wc -l < "${INDEX}") artefacts into ${OUT}" # Fail closed on the audit trail specifically. A day with zero audit # artefacts is a collection failure, not a quiet zero, and the # reconciler downstream cannot catch it: an absent kind produces no # row, so it raises no rejected-artefact error. Sealed first, so the # partial day stays verifiable. if [ "${audit_rows}" -eq 0 ]; then echo "collect FAILED: no audit-log artefact for ${DAY}" >&2 exit 1 fi ``` Then one reconciler reads that index and emits two mappings. Its most useful property is failing closed: an artefact mapping to neither programme is either a collection step nobody needs or a mapping somebody forgot. The SOC 2 evidence article covers the retention and immutability plumbing underneath; what follows is only the mapping layer. ```python #!/usr/bin/env python3 """Map one day's evidence index onto both programmes' artefact sets. Reads the dated index produced by collect-evidence.sh and emits two mappings over the same rows: trust services criteria references for the SOC 2 side, ISMS clause and Statement of Applicability references for the ISO 27001 side. Fails closed. An artefact kind that maps to neither programme exits non-zero, because in practice that means one of two things: a collection step that serves no audit, or a mapping that was never written. Both are worth a build failure and neither is worth discovering during fieldwork. """ from __future__ import annotations import argparse import json import sys from pathlib import Path # The mapping table is the only programme-aware code in the pipeline. # Criteria references are illustrative placeholders: confirm every one # against the criteria your engagement actually names and against your # own Statement of Applicability before relying on them. MAPPING: dict[str, dict[str, list[str]]] = { "audit_log": { "tsc": ["CC6.1", "CC7.2"], "isms": ["A.8.15", "A.8.16", "ISMS-9.1"], }, "access_review": { "tsc": ["CC6.1", "CC6.2", "CC6.3"], "isms": ["A.5.15", "A.5.18", "ISMS-9.1"], }, "policy_report": { "tsc": ["CC7.1", "CC8.1"], "isms": ["A.8.9", "ISMS-8.1"], }, "backup": { "tsc": ["A1.2"], "isms": ["A.8.13"], }, "restore_result": { "tsc": ["A1.2", "A1.3"], "isms": ["A.8.13", "A.5.29", "ISMS-9.1"], }, } # Every row must carry these before it can be cited as evidence. A row # missing one is a collection bug, and it fails closed like an unknown # kind rather than raising KeyError halfway through a run. REQUIRED_FIELDS = ("path", "collected_at", "sha256") def load_index(path: Path) -> list[dict]: rows: list[dict] = [] with path.open(encoding="utf-8") as handle: for lineno, line in enumerate(handle, start=1): line = line.strip() if not line: continue try: rows.append(json.loads(line)) except json.JSONDecodeError as exc: raise SystemExit(f"{path}:{lineno}: malformed index row: {exc}") if not rows: raise SystemExit(f"{path}: index is empty — nothing was collected") return rows def main() -> int: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("index", type=Path, help="path to a dated index.jsonl") parser.add_argument("--out", type=Path, required=True, help="output directory") args = parser.parse_args() rows = load_index(args.index) soc2: list[dict] = [] iso: list[dict] = [] rejected: list[str] = [] for row in rows: kind = row.get("kind", "") absent = [field for field in REQUIRED_FIELDS if not row.get(field)] if absent: rejected.append( f"{row.get('path', '?')} (incomplete row, missing {', '.join(absent)})" ) continue entry = MAPPING.get(kind) if entry is None: rejected.append(f"{row['path']} (kind={kind or 'missing'})") continue common = { "artefact": row["path"], "collected_at": row["collected_at"], "sha256": row["sha256"], } soc2.append({**common, "criteria": entry["tsc"]}) iso.append({**common, "references": entry["isms"]}) args.out.mkdir(parents=True, exist_ok=True) (args.out / "soc2-evidence-map.json").write_text( json.dumps(soc2, indent=2, sort_keys=True) + "\n", encoding="utf-8" ) (args.out / "isms-evidence-map.json").write_text( json.dumps(iso, indent=2, sort_keys=True) + "\n", encoding="utf-8" ) print( f"mapped {len(soc2)} of {len(rows)} artefacts into both maps, " f"{len(rejected)} rejected" ) if rejected: print("rejected artefacts (failing closed):", file=sys.stderr) for item in rejected: print(f" {item}", file=sys.stderr) return 1 return 0 if __name__ == "__main__": sys.exit(main()) ``` ## What Does Not Reuse: System Description vs ISMS Scope Two documents here are written by humans about the same infrastructure, and they are not even the same genre. Management's description of the system is a narrative about a period: the system as it operated throughout that window, changes included, with the practitioner examining the description as well as the controls in it. Past tense, temporal boundary. An ISMS scope statement is a standing declaration of what the management system covers now, and the Statement of Applicability is its register — which Annex A controls apply, which are excluded, why. Present tense, organisational boundary. The SoA is not internal paperwork, which is the common misreading. It "is found in 6.1.3 of the main requirements for ISO 27001, which is part of the broader 6.1, focused on actions to address risks and opportunities," and it is "an integral part of the mandatory ISO 27001 documentation that needs to be presented to an external auditor when the ISMS is undergoing an independent audit." An audited artefact — cited here from a vendor explainer rather than the standard, because clause 6.1.3 sits behind the ISO paywall. Since both describe one estate, the failure mode is disagreement, not duplication. A system description naming three clusters and an SoA scope naming two is a finding on one side and an awkward question on the other. The fix is not to merge them but to derive the shared facts from one place: one inventory of clusters, namespaces, data stores and third parties, cited by both. The estate changes, the inventory changes once, and both documents go stale detectably. Concentration and substitutability of those vendors is its own discipline, and both programmes ask about it. ## Failure Modes That Cost a Period or a Cycle Each programme breaks in ways the other cannot. Which failures are recoverable and which cost a year is most of the planning value. - A qualified opinion. AT-C section 205.72: the practitioner "should express a qualified opinion" when misstatements are "material, but not pervasive to the subject matter," or when sufficient appropriate evidence could not be obtained and the possible effects "could be material but not pervasive." Read the second limb: you can be qualified for missing evidence about a control that worked. The report issues, qualified, and the period is closed. - Retention shorter than the period. The quietest failure of the set: everything looks healthy until fieldwork reaches the far end of the window and finds rotated-away logs. - A missed surveillance date. The 12-month ceiling on the first surveillance audit runs from the certification decision, not from a convenient internal anniversary. Suspension and withdrawal are the certification body's instruments, and a lapsed certificate is not explained away in a security review. - A boundary changed mid-cycle. A cluster in a new jurisdiction, or a new production data store, changes the scope both descriptive documents assert. If neither moves, one artefact describes an estate that no longer exists. - Treating a Type 1 as a milestone. No operating-effectiveness opinion, no tests-and-results section. A design checkpoint before a period starts, useful. An answer to a buyer asking whether controls ran, not the artefact they asked for. The common shape: none of these are control failures. Every one is an evidence, calendar or scope failure — categories a control-mapping table cannot show, and that a dated store with two mappings over it makes visible months ahead. ## Which Buyer Asks for Which, and Why the Asks Differ The pattern operators notice — US enterprise security reviews reaching for the report, EU and international tenders for the certificate — is an observation about markets, not a measured statistic, and the structure explains much of it. A certificate is issued to be shown: short, attributable to an accredited body, answerable as of today. A Type 2 report is long, rich, and frequently arrives with a paragraph naming who may use it. One suits a form field; the other suits a reviewer under NDA who will read the tests-and-results section. On the European side there is a thread of hard law pointing at the ISO standards. Recital 3 of Commission Implementing Regulation (EU) 2024/2690 states that the technical and methodological requirements in its Annex "are based on European and international standards, such as ISO/IEC 27001, ISO/IEC 27002 and ETSI EN 319401, and technical specifications, such as CEN/TS 18026:2024." A regulator naming reference points, not a mandate to certify — but a tender asking for the certificate is usually downstream of language like that. The overlapping EU regimes converge on the same standards repeatedly. Sell into both markets and the answer is both artefacts on one pipeline, which is why the reuse boundary matters more than the control mapping. The mapping says the programmes ask similar questions. The boundary says what you build once. ## The Long Game: Cadence Over Certificates Both programmes punish the same posture: treating an audit as a project. A project has a deadline, a scramble and a gap afterwards, and the gap is where the retention window quietly closes and the surveillance date quietly arrives. Organisations for which neither audit is eventful run the shorter cadence continuously — daily collection, dated and hashed, mapped both ways, failing closed on anything unclassified. That posture outlasts the specifics on this page, which is the point of building it rather than buying a dashboard. Criteria get revised; exposure drafts become standards; accreditation arrangements consolidate, as they did on 1 January 2026. A pipeline whose output is dated, hashed, programme-agnostic facts survives all of it, because only the mapping layer has to change — and the mapping layer is a table. Which is the sovereignty argument in its least rhetorical form. The evidence is generated by infrastructure you operate, retained on terms you set, mapped by code you can read. Change certification body and you keep your position in the cycle. Change auditor and the store is unaffected. The certificate and the opinion are other people's documents about your systems. The store underneath is yours. ### FAQ Q: Is SOC 2 a certification? A: No, and the distinction is structural rather than pedantic. A SOC 2 is an assertion-based examination engagement under the AICPA attestation standards — SSAE No. 18 as amended by SSAE No. 21 — defined at AT-C section 105.12 as one in which the practitioner "obtains reasonable assurance by obtaining sufficient appropriate evidence about the responsible party's measurement or evaluation of the underlying subject matter against criteria" in order to base an opinion on it. Management measures and asserts; the practitioner opines on that assertion. The deliverable is a report expressing that opinion about a period that has already closed — not a certificate, and not a status you hold. Nobody issues it as a credential, and there is no accredited body behind it in the sense there is for ISO 27001. "SOC 2 certified" is a phrase with nothing behind it. The accurate form names the artefact and its period — a current Type 2 report covering the period ending on a stated date — and that is what a reviewer who knows the difference is checking for. Q: How long is an ISO 27001 certificate valid? A: Three years is the general default, and it runs as a cycle rather than a simple document expiry. An accreditation body's published restatement of ISO/IEC 17021-1:2015 — the International Accreditation Service's abridged Section 9 — puts the programme at a two-stage initial audit, surveillance in each of the first two years after the certification decision, and recertification in year three before the certificate expires. Two details matter more than the headline number. The cycle's origin is the certification decision itself, not the audit date and not the certificate's print date. And surveillance runs at least once a calendar year outside recertification years, with the first one capped at 12 months from the decision date — a tighter constraint than the calendar-year rule alone, which would otherwise permit a January decision and a December surveillance nearly two years later. Three years is not universal either: a note to clause 9.1.3.2 allows an industry-specific certification scheme to set a different cycle length. Confirm yours with your certification body. Q: Can the same evidence be used for both ISO 27001 and SOC 2? A: Most of it, yes — and that is the practical reason to run both. Machine-generated artefacts are programme-agnostic: API-server audit events, policy reports, scheduled backups with their restore results, and dated access reviews are the same facts whichever auditor reads them. Collect them once into a dated, hashed store and maintain two mappings over the same rows: trust services criteria references on the SOC 2 side, ISMS clause and Statement of Applicability references on the ISO side. What does not reuse is the descriptive layer. Management's description of the system is a narrative about a closed period; the ISMS scope statement and SoA are standing declarations about now. Those two documents cannot be merged, though both should derive their shared facts — clusters, namespaces, data stores, third parties — from one inventory so they cannot disagree. Q: Why do some SOC 2 reports say they cannot be shared? A: Because the attestation standards require a use-restricting alert in specific circumstances, and SOC 2 reports commonly fall into one of them. AT-C section 205.64 requires the report to carry an alert, in a paragraph of its own, restricting who may use it — one of the listed triggers being criteria that are available only to specified parties. Where that applies, paragraph .65 requires the alert to say the report is intended solely for those parties' information and use, to identify them, and to state that nobody else should use it. This is conditional rather than automatic, so read the alert paragraph in the report you actually hold rather than assuming either way. If you need something freely distributable, a SOC 3 keeps the operating-effectiveness opinion but omits the detailed tests and their results. Q: What is the difference between a SOC 2 Type 1 and a Type 2? A: A Type 2 opines on operating effectiveness over time; a Type 1 does not opine on it at all. Per the trust services criteria, a Type 2 engagement covers both the suitability of design and the operating effectiveness of the controls in management's system description, throughout a specified period, and its report also carries a detailed description of the service auditor's tests of those controls and the results. A Type 1 has neither half: no operating-effectiveness opinion, and no tests-and-results description. So a Type 1 answers whether the controls were designed suitably as at a point in time. It is a reasonable design checkpoint before a period begins. It is not the artefact a buyer asking for evidence of operation is asking for, and presenting it as one tends to cost more credibility than starting the period would have cost time. --- # PCI DSS Self-Assessment: When the SAQ Path Closes URL: https://stribog.com/blog/pci-dss-self-assessment-questionnaire-saq-eligibility Published: 2026-09-26 Category: Compliance Keywords: self assessment questionnaire pci dss, pci dss self-assessment questionnaire, saq eligibility criteria, saq d for service providers, pci dss validation levels service provider, visa level 1 merchant report on compliance, compliance-accepting entity acquirer saq, saq a script eligibility criteria iframe Which self-assessment questionnaire PCI DSS lets you use is an architecture test and a Visa volume test, and neither verdict is the merchant's to issue. > **The short answer** — The self-assessment questionnaire PCI DSS gives you is not a difficulty setting you choose: PCI SSC "does not define compliance requirements for any organization or set compliance validation responsibilities." Eligibility is two gates — an architectural one, where your payment path must consist only of the system types a form permits, and a volume one, where Visa's regional compliance-validation page routes merchants over 6 million Visa transactions a year (all channels) to a QSA's Report on Compliance instead. Know which side of both you sit on, dated, and confirmed by the entity that accepts your validation. Search the term and you get lists: each form, one paragraph, ending on "pick the one that matches your setup." Nobody picks. Two parties decide, on unrelated criteria, and the answer moves when a transaction count creeps up or an appliance lands in the wrong segment. ## Who Decides Whether You May Self-Assess at All Start with the disclaimer on the Council's own blog — the sentence the roundups skip. PCI SSC "does not define compliance requirements for any organization or set compliance validation responsibilities"; those requirements "are set by brands, acquirers, payment facilitators, etc., which are often referred to as compliance enforcing entities." The forms are instruments. Both the obligation to validate and the permission to self-assess come from elsewhere. Naming that elsewhere precisely matters, because three PCI SSC documents use three phrases for it — "compliance enforcing entities" in the 30 January 2025 blog post, "the entity to which the SAQ will be submitted" in the October 2024 SAQ bulletin, and in FAQ 1473 the one naming the role: > "Compliance-accepting entities (typically, payment brands and acquirers) are responsible for determining the PCI DSS validation and reporting methods of their merchants and service providers, including how compliance is to be evidenced–for example, whether via a Report on Compliance (ROC) or a Self-Assessment Questionnaire (SAQ)." — PCI SSC FAQ 1473 The question is never which form reads most manageable, but which entity accepts your validation. Two consequences follow. First, a self-assessment starts with a conversation: the October 2024 bulletin says merchants "should confirm they meet all eligibility criteria for a particular SAQ before commencing their self-assessment", and are "encouraged to first contact the entity to which the SAQ will be submitted". Second, eligibility criteria do not travel. Asked whether they can guide requirement applicability in a Report on Compliance, FAQ 1331 answers that SAQs "should not be used as a 'guide'" for that "unless explicitly reviewed, discussed and agreed upon with the merchant's compliance accepting entity" — discouraged, with a route to agreement, not prohibited. Scope and evidence are a separate discipline; this is the gate before it. ## The SAQ Family Is a Map of Architectures, Not a Difficulty Ladder The forms are not tiers, and reading them as easy-to-hard is the root of most wrong answers. Each carries eligibility criteria describing a payment architecture, and FAQ 1443 states the constraint without wiggle room: "only system types defined in the eligibility criteria may be used in that environment", and environments "containing any other types of systems would not be eligible". Eligibility is a property of what you built, read off an inventory and a segmentation diagram. One sourcing note, because the version is load-bearing. The October 2024 bulletin states that the v4.0 SAQs "will remain active until PCI DSS v4.0 is retired on 31 December 2024" — so the v4.0.1 forms are the current ones, and they sit behind the Council's document-library licence gate. The retired v4.0 PDFs of April 2022 are still publicly served and are what is quoted below — an indication of the architectures the criteria describe, not the form to submit. The bulletin names eligibility-criteria clarifications only in SAQs A, A-EP and C-VT, but that list is expressly non-exhaustive — it opens "including:" and closes by directing readers to "the Document Changes table at the beginning of each SAQ for specific details". Confirm every form below against its own v4.0.1 Document Changes table with your compliance-accepting entity. Read off those v4.0 forms, the architectures are narrow. SAQ B: account data only via imprint machines or standalone dial-out terminals, stored on no computer system. SAQ C: an Internet-connected payment application system — a point-of-sale, typically — storing no electronic account data and connected to no other system in the merchant environment, a segmentation rule in eligibility clothing. SAQ P2PE: account data only via a validated PCI-listed P2PE solution, with merchants that "do not have access to clear-text account data on any computer system", every control in the P2PE Instruction Manual implemented. All three state they are not applicable to e-commerce channels or service providers. SAQ D for Merchants is the residual, for merchants that "do not meet the criteria for any other SAQ type". SAQ B-IP is worth quoting, because its exclusion is what operators trip over. It covers merchants taking account data only via "standalone, PCI-listed approved PIN Transaction Security (PTS) point-of-interaction (POI) devices with an IP connection to the payment processor" — then: "merchants using SCRs or SCRPs are not eligible for this SAQ", SCR and SCRP being Secure Card Reader classifications. Being on the PTS list does not make a device a B-IP device; its classification does. Eligibility can hinge on a procurement choice. The criteria are not a device budget. FAQ 1443 says the intent is not "to prohibit more than one of the permitted system types being on the same network zone, as long as the permitted systems are all isolated from other types of systems". A *kind* of system appearing breaks eligibility; a count rising does not — which makes default-deny network policy an eligibility control. [Figure: A decision diagram read top to bottom. Gate one is a diamond asking whether the entity is being assessed as a merchant or as a service provider for this payment flow. The service-provider branch leads to a container with two nodes: FAQ 1133's rule that SAQ D for Service Providers is the only SAQ for SAQ-eligible service providers and all other SAQs are merchant-only, feeding a hexagon recording that the same form applies only to service providers a payment brand has defined as eligible to self-assess, so SAQ-eligible is a status conferred rather than a form chosen. The merchant branch leads to gate two, a container of six architecture nodes: imprint or dial-out only for SAQ B; standalone IP-connected PTS POI devices excluding those classified SCR or SCRP for SAQ B-IP; an isolated Internet-connected payment application for SAQ C; a validated PCI-listed P2PE solution for SAQ P2PE; a hexagonal e-commerce node for embedded form or redirect, recording that A and A-EP criteria were clarified in v4.0.1 and must be read off the current form, with FAQ 1588's split between embedded form and redirect; and a residual node for own-site capture and everything else, SAQ D for Merchants. A separate container holds FAQ 1443's rule as a dashed input into gate two: only system types defined in the criteria may be used, environments containing any other type are not eligible, and the constraint is segmentation rather than device count. Every architecture branch and the service-provider hexagon terminate at a node for the compliance-accepting entity, carrying PCI SSC's instruction to confirm eligibility before commencing and to first contact the entity to which the SAQ will be submitted. Two dashed edges break out to a hexagonal Report on Compliance node — one where the brand requires a ROC of a service provider, one where the volume gate closes the SAQ path for a merchant.] ## If You Are a Service Provider, There Is Exactly One SAQ Settle the role before any architecture question; it is the gate that closes fastest. FAQ 1133 leaves nothing open: "SAQ D for Service Providers is the ONLY SAQ for SAQ-eligible service providers. All other SAQs are for merchant use only." The v4.0 form (April 2022) says the same of itself. The clause that matters more sits immediately before it: the form "applies to all service providers defined by a payment brand as being eligible to complete a self-assessment questionnaire". SAQ-eligible is a status a brand confers, not a shape an architecture earns. A platform can be trivial — one embedded form, no account data on any system it runs — and still be told to produce a ROC. So "the platform only embeds the processor's form, therefore SAQ A" is not an argument a service provider has. It has two destinations: SAQ D for Service Providers, if a brand has said it may self-assess at all, or an assessor. No lighter form exists to chase, so the effort belongs in making the SAQ D evidence cheap, as with owning your SOC 2 evidence. The determination is not self-certified, and it is worth asking rather than answering: a product that never sees a primary account number but serves the page collecting one is where assumption and answer diverge. Record the role, who agreed it and the date, beside the level. Concentration risk in a single vendor is the same question pointed the other way. ## The Iframe Boundary: SAQ A Eligibility Is Now a Scripts Problem Two changes to SAQ A sit on the record from different dates, and running them together makes the Council look self-contradictory. The October 2024 bulletin lists "Adding a requirement to SAQ A and removing a requirement from SAQ C". Then on 30 January 2025 PCI SSC announced SAQ A r1: removal of Requirements 6.4.3 and 11.6.1 "for payment page security" and 12.3.1 "for a Targeted Risk Analysis", and in their place an eligibility criterion that merchants "confirm their site is not susceptible to attacks from scripts that could affect the merchant's e-commerce system(s)". Read that as PCI DSS dropping payment-page script controls and you have made the most expensive error this subject offers. The same post says the opposite: v4.0.1 Requirements 6.4.3, 11.6.1 and 12.3.1 "become effective as of 31 March 2025", and the SAQ A modifications "do not remove or diminish the underlying requirements within PCI DSS." What moved is where the claim gets made — out of a reporting line, into an eligibility condition. A failed requirement produces a finding; a failed criterion removes the form. Which pages the criterion reaches is a question for FAQ 1588, not for inference. It "only applies to e-commerce merchants with a webpage that includes a TPSP's/payment processor's embedded payment page/form (for example, one or more inline frame(s) (iframes))", and does "not apply" where the page "redirects customers from the merchant's webpage to a TPSP/payment processor". Embedding and redirecting are not two styles of one integration; under SAQ A r1 they are different eligibility positions, and a front-end team can move you between them in a sprint. FAQ 1588 gives two ways to satisfy it, with different engineering shapes. Either use "techniques such as, but not limited to, those detailed in PCI DSS Requirements 6.4.3 and 11.6.1 to protect the merchant's webpage from scripts targeting account data", deployable "by the merchant or a third party" — or obtain confirmation from the processor supplying the form that its solution "includes techniques that protect the merchant's payment page from script attacks." The second is a procurement artefact, not a control. Either way one question stays open — what is on the page today — and a static inventory answers it for the markup you ship, repeatably enough to schedule. ```python #!/usr/bin/env python3 """payment_page_inventory.py — the script and frame origins your page serves. Reads the checkout page's delivered markup and classifies every