The EU AI Act for High-Risk AI Systems: An On-Prem Compliance Path

Most engineering leaders first meet the EU AI Act (Regulation (EU) 2024/1689) as a legal-department problem: a matrix of obligations, a RACI chart, a GRC tool procurement. That framing is wrong in a specific and expensive way. For a high-risk AI system — the tier that captures the AI most companies actually deploy in production — the Act's requirements are not satisfied by policy documents. They are satisfied, or not, by the architecture underneath the model: where inference runs, who holds the logs, whether the deployed system can be reproduced, and whether you can hand an auditor evidence rather than an assurance. This article ignores the compliance-theatre layer entirely and reads the Act as what it is for a platform team: an infrastructure specification.

We have argued the broad convergence thesis elsewhere — that NIS2, DORA, and the AI Act collectively push regulated workloads toward self-hosted infrastructure. This piece does something narrower and deeper. It drills into a single regulation, the AI Act, and its technical control surface for high-risk systems: how classification works, what Article 12 record-keeping actually demands of a logging pipeline, what conformity assessment produces as evidence, and why post-market monitoring is the observability stack you should already be running. The conclusion is not ideological. It follows from the text of the Regulation and from what a third-party model API can and cannot technically deliver.

What 'High-Risk' Actually Means: Annex III and the Article 6 Filter

The Act sorts AI into four tiers. Prohibited practices (Article 5 — social scoring, untargeted facial-recognition scraping, workplace and school emotion recognition, most real-time public biometric identification) have been banned since 2 February 2025. High-risk systems carry the full weight of Chapter III. Limited-risk systems owe only transparency duties under Article 50 (label AI-generated content, disclose that a chatbot is a machine). Minimal-risk systems — the vast majority — owe nothing beyond voluntary codes. The entire compliance burden concentrates in the high-risk tier, and the first engineering task is knowing whether you are in it.

There are two routes into high-risk. Article 6(1) captures AI that is a safety component of a product already regulated under EU sectoral law listed in Annex I — medical devices, machinery, lifts, radio equipment. Article 6(2) captures the use-case systems enumerated in Annex III, and this is where most software teams land. Annex III lists eight areas: biometrics; safety components of critical infrastructure; education and vocational training; employment and worker management; access to essential private and public services (explicitly including creditworthiness scoring and life/health insurance risk pricing); law enforcement; migration, asylum, and border control; and the administration of justice and democratic processes.

Click to zoom

Article 6(3) is the escape hatch, and it is narrower than teams hope. An Annex III system is *not* high-risk if it performs only a narrow procedural task, improves the result of a completed human activity, detects deviation from prior decision patterns without replacing human judgement, or performs a preparatory task to an assessment. But there is a hard carve-out: any system that profiles natural persons is always high-risk, regardless of the filter. And even when the filter applies, you must document the assessment and register the system — the Act does not let you self-exempt silently. The practical upshot: a credit-decision model, a CV-screening system, or a clinical triage assistant is high-risk, full stop, and the engineering obligations attach.

The Provider Control Surface: A Dozen Articles That Become Infrastructure

Chapter III, Section 2 is where the Act stops being abstract. For a provider of a high-risk system, roughly a dozen articles specify requirements that a platform team implements, not a lawyer. Article 9 mandates a lifecycle-long risk-management system. Article 10 governs training, validation, and test data — provenance, representativeness, documented bias examination. Article 11, expanded by Annex IV, requires technical documentation drawn up *before* the system reaches the market and kept current thereafter. Article 12 requires automatic event logging. Article 13 requires transparency to deployers; Article 14, built-in human oversight; Article 15, accuracy, robustness, and cybersecurity by design, including resistance to data and model poisoning and adversarial examples.

Then the retention and process articles: Article 17 requires a documented quality-management system, which for a financial-services or healthcare provider can nest inside an existing sectoral QMS rather than duplicate it. Article 18 requires you to retain the technical documentation, QMS records, and declaration of conformity for ten years after the system is placed on the market. Article 19 requires the provider to keep the automatically generated logs under its own control for a period appropriate to the purpose — at least six months — longer where other law, such as GDPR or DORA, demands. Article 20 sets the corrective-action and withdrawal duties when non-compliance is discovered. Article 72 requires an active post-market monitoring system for the system's whole life.

Read the list again with an architect's eye and a pattern emerges. Several of these obligations are only *auditable* if you control the runtime. You cannot produce Annex IV technical documentation for a model whose weights, versioning, and evaluation you never see. You cannot keep Article 19 logs under your own control if they are written to a vendor's managed logging plane. You cannot demonstrate Article 15 cybersecurity of a system whose serving stack changes underneath you without notice. The Act does not mention Kubernetes, on-prem, or sovereignty once — but its evidentiary requirements quietly assume you can reach inside the system and prove things about it. A black-box inference API cannot grant that reach.

Article 12 Record-Keeping: Why Log Custody Is an Architecture Decision

Article 12 is the requirement most directly translated into a data pipeline, so it rewards precision. It requires that a high-risk system technically enable the automatic recording of events (logs) over the system's lifetime. Article 12(2) frames the purpose: the logging must let you identify situations that may cause the system to present a risk or undergo a substantial modification, and it must support post-market monitoring under Article 72. Article 12(3) gets specific for the highest-sensitivity case — remote biometric identification systems must, at minimum, log the period of each use, the reference database checked, the input data that produced a match, and the identity of the humans who verified the results.

A subtlety worth internalising: Article 12 itself sets *no* retention period. The duty to keep the logs lives in Article 19 (provider, at least six months) and Article 26 (deployer, at least six months) — and sector law routinely lengthens it, five years being common under DORA-aligned financial rules. So the design constraint is not merely 'log the inferences.' It is: generate a tamper-evident record of every high-risk inference, and retain it, under your custody, for years, in a form an auditor can read and trust. That is a records-management problem with legal weight, and it is where a hosted API leaves you exposed — you have contractual access to a vendor's log store, which is a categorically weaker thing than custody of your own.

The technical shape of an Article 12 record is unglamorous and specific. Each high-risk inference emits a structured event capturing the input (hashed, if the raw input is itself sensitive), the output, the exact model version, the requesting identity, and any human-oversight action. Those events are hash-chained so that any later tampering is detectable, and written to an append-only store — object-lock or WORM — that even your own administrators cannot silently rewrite within the retention window. None of this is exotic; it is the same self-hosted observability discipline you already apply to operational telemetry, pointed at a legal requirement and hardened for immutability.

// One append-only record per high-risk inference — the technical shape of an
// Article 12 log entry. Emitted at the serving layer, hash-chained, and written
// to a WORM / object-lock store held inside your own legal perimeter.
{
  "ts": "2026-07-05T09:14:22.481Z",
  "system_id": "eu-hr-credit-decisioning",   // registered high-risk system (Art. 49)
  "model": {
    "name": "credit-risk",
    "weights_sha256": "b1946ac92492d2347c6235b4d2611184",
    "revision": 41                            // pinned, versioned, reproducible
  },
  "request": {
    "input_sha256": "e3b0c44298fc1c149afbf4c8996fb924",  // hash, not raw PII
    "operator": "svc:loan-decisioning",
    "deployer": "eu-retail-bank-unit"
  },
  "output": { "decision": "refer", "score": 0.612, "explanation_ref": "s3://audit/shap/9f86.." },
  "oversight": { "human_reviewer": null, "override": null },   // Art. 14 hook
  "prev_hash": "9f86d081884c7d659a2feaa0c55ad015",            // -> tamper-evident chain
  "record_hash": "2c26b46b68ffc68ff99b453c1d304134"
}

Technical Documentation and Traceability: Annex IV Against a Black Box

Article 11 requires technical documentation, and Annex IV specifies its contents in nine categories. Read as an infrastructure checklist, they are demanding: a general description of the system and its intended purpose; the development process, architecture, and design specifications; the model's capabilities, limitations, and expected accuracy; the metrics used and why; the Article 9 risk-management detail; a log of lifecycle changes; the harmonised standards applied; a copy of the Article 47 declaration of conformity; and the Article 72 post-market monitoring plan. Crucially, this documentation must be accurate about the system *as deployed* and kept current as it changes.

This is where the black-box problem becomes unavoidable. You cannot write truthful Annex IV documentation about the architecture, the versioning, or the change history of a model you consume as an opaque API — you do not have those facts, and the provider's model card is not a substitute for documentation of *your* deployed system. When you self-host inference on your own Kubernetes, by contrast, the documentation writes itself out of the artefacts you already control: the pinned model digest, the container image SHA, the serving-stack configuration, the evaluation harness and its results, the GitOps history of every change. Traceability stops being a document you assemble under audit pressure and becomes a property of the system.

Click to zoom

Traceability has a second dimension the Act cares about: being able to reconstruct, after the fact, exactly what a system did and why. Article 12 logs tell you *that* an inference happened; reproducibility tells you that you can re-run it. A pinned model digest plus a versioned serving configuration means an incident six months later can be reproduced bit-for-bit — the same weights, the same tokenizer, the same runtime. That is impossible against a hosted endpoint that silently upgrades its model, and it is the difference between an investigation that concludes with evidence and one that concludes with a shrug. For the most sensitive deployments, confidential computing and remote attestation extend the same idea into a hardware-signed proof of exactly which code ran on which hardware.

Conformity Assessment and CE Marking: Producing Evidence, Not Promises

Before a high-risk system is placed on the market, it must pass a conformity assessment (Article 43), carry a CE marking (Article 48), and be registered in the EU database (Article 49). For most Annex III systems — points 2 through 8 — the route is internal control under Annex VI: you assess your own conformity against the requirements, with no third party involved. Biometric systems under Annex III point 1 are the exception; they may require a notified body under Annex VII, and where no harmonised standard yet exists or the system serves law enforcement or migration authorities, the third-party route becomes mandatory. Article 47 then requires a written, machine-readable EU declaration of conformity, retained for ten years; Article 48 permits a digital CE marking for AI delivered digitally.

Internal control sounds permissive — self-assessment, no auditor at the gate — but it inverts the burden in a way that favours controllable infrastructure. Because *you* attest conformity, *you* must hold the evidence that backs the attestation, and it must be reproducible if a market-surveillance authority later demands it. The declaration of conformity is only as strong as the artefacts behind it. This is why a conformity assessment is, at bottom, an evidence-production problem, and why the architecture that makes evidence cheap to produce and hard to fake wins. The right shape is a conformity bundle assembled from artefacts under version control, not a binder compiled by hand.

conformity-bundle/eu-hr-credit-decisioning/rev-41/
├── declaration-of-conformity.json      # Art. 47 — signed, machine-readable, Annex V
├── technical-documentation/            # Art. 11 + Annex IV (nine sections)
│   ├── 01-system-description.md
│   ├── 02-architecture-and-serving.md  # pinned image SHA, vLLM config, GPU topology
│   ├── 04-performance-metrics.json     # eval harness output, per-subpopulation
│   ├── 05-risk-management.md           # Art. 9 record
│   └── 06-change-log.md                # GitOps history -> every substantial modification
├── model/
│   ├── weights.sha256                  # b1946ac9… — the exact artefact served
│   └── eval-report-rev-41.json         # reproducible on demand from pinned inputs
├── logs/
│   └── retention-policy.yaml           # Art. 19 -> object-lock, >= 6 months / 5 years
└── post-market-monitoring-plan.md      # Art. 72

A word on standards, because it shapes the timeline. Compliance with a harmonised standard published in the Official Journal grants a presumption of conformity under Article 40 — the cheapest path to a defensible assessment. As of the original 2026 deadline, those standards did not exist: CEN-CENELEC's JTC 21 accelerated its programme in late 2025 but targeted first publications only for the end of 2026. ISO/IEC 42001, the AI-management-system standard, is a useful voluntary baseline, but a management-system certificate is not the same as the Act's mandated record-keeping — it organises how you govern AI without, by itself, producing the Article 12 and Article 19 logs the Act requires. The absence of ready standards is precisely what bought the Digital Omnibus deferral. It is not a reason to wait; it is a reason to build the evidence machinery now, standards-agnostic.

Post-Market Monitoring: Article 72 Is the Observability You Already Run

Article 72 requires providers to establish a documented post-market monitoring system, proportionate to the risk, that actively and systematically collects and analyses the system's performance data across its entire operational life — including data fed back from deployers. The monitoring plan is itself part of the Annex IV technical documentation, and the Commission was due to issue a template for it in early 2026. Article 20's corrective-action duties and Article 73's serious-incident reporting sit on top: when monitoring detects that the system has drifted out of its declared performance envelope or caused harm, defined notification clocks start.

For anyone who has run production infrastructure, this is familiar equipment wearing a regulatory label. Post-market monitoring is drift detection, performance SLOs, alerting, and incident response — the same stack you build for reliability, now also serving as compliance evidence. The requirement to analyse deployer-supplied data is a data-flow you must design: telemetry from where the model is used, flowing back to where it is governed. When both the inference and the observability run on infrastructure you own, that flow is a network policy and a retention bucket. When inference is a third-party API, you are limited to whatever the vendor chooses to meter back to you — which is rarely what Article 72 wants and never under your control.

This is the quiet efficiency of the on-prem path: the Act's post-market monitoring obligation and your own operational need for observability are the same system. You are not building a compliance apparatus bolted onto the side of production; you are pointing the telemetry you already need at a second purpose. The regulated-fintech and healthcare deployments we have supported treat the Article 72 monitoring plan as a view over the existing metrics-and-logs platform, not a parallel build — which is only possible because the platform is theirs to instrument.

GPAI on Your Own Infrastructure: Deploying Open-Weight Models Under Chapter V

General-purpose AI models — the foundation and open-weight models most teams now build on — have their own regime in Chapter V, in force since 2 August 2025. Article 53 sets baseline provider obligations: technical documentation (Annex XI), information for downstream integrators (Annex XII), a copyright-compliance policy, and a published summary of training data. Article 51 classifies a GPAI model as carrying systemic risk when its cumulative training compute exceeds 10²⁵ FLOPs, which triggers the heavier Article 55 duties — adversarial evaluation, systemic-risk mitigation, incident reporting to the AI Office, and model-and-infrastructure cybersecurity. Open-source models are exempt from the baseline documentation duties *unless* they cross the systemic-risk threshold.

The line that matters for a platform team is provider versus deployer. Consume an open-weight model unchanged and you are a deployer; the model provider carries the Chapter V obligations. But fine-tune it, or assemble it into a high-risk system, and you can inherit provider obligations for the result — the Annex IV documentation, the Article 12 logging, the conformity assessment now attach to *your* system. This is the deployment reality behind most enterprise AI: you are not calling someone's API, you are running an open-weight model, adapted, on your own hardware. That is a sovereignty advantage — the weights, the fine-tuning data, and the inference capacity are yours — and it is also the point at which the high-risk control surface becomes your responsibility to evidence.

The voluntary GPAI Code of Practice, published in 2025, gives a read on the compliance landscape: roughly two dozen companies signed its first cohort — most major frontier labs, though Meta declined and no major Chinese lab joined — ahead of the Commission's enforcement powers taking effect in August 2026. The signal for a self-hosting team is that the upstream provenance of the model you deploy is itself becoming a documented, contestable fact. Owning the model and its serving stack is what lets you make claims about it that hold up, rather than inheriting a supply chain you cannot inspect — the same supply-chain attestation logic that governs your container images now extends to your model weights.

Why On-Prem Inference Is the Cleanest Compliance Path

Assemble the control surface and one architectural conclusion is hard to avoid. The AI Act's high-risk obligations divide into two kinds. Some — a risk-management process, a quality-management system, a fundamental-rights impact assessment — are organisational, and you can satisfy them on any infrastructure with enough discipline. But the falsifiable ones — Article 12 log custody, Annex IV technical documentation of the deployed system, Article 15 cybersecurity you can attest, Article 72 monitoring you own, reproducible conformity evidence — are the ones a regulator can actually test, and every one of them is satisfiable *by design* only when you control the runtime. A third-party API can satisfy them contractually at best, and regulators are increasingly explicit that a contract is not evidence.

The honest counter-argument is operational cost: self-hosting inference means you carry the GPU capacity, the serving stack, and the platform. That is real, and for a low-stakes, non-high-risk feature it may not be worth it. But for a system that is genuinely high-risk — a credit decision, a hiring filter, a clinical triage tool — the evidentiary requirements are exactly the ones that a controllable runtime makes cheap and an opaque API makes impossible. The regulated-fintech cloud exit pattern generalises here: at meaningful volume, the compliance evidence available from infrastructure you own is not merely better, it is *categorically different* from what a managed endpoint can produce, and the operational premium is smaller than the risk it retires.

The EU AI Act, read honestly by an engineer, is not a legal text that happens to touch infrastructure. For high-risk systems it *is* an infrastructure specification: it demands records you can custody, systems you can document, models you can reproduce, and monitoring you can own. Those are properties of a platform, not clauses in a contract. The regulation the EU wrote for high-risk AI describes, almost line by line, a self-operated, auditable, sovereign inference stack. That is the architecture we build for — not because it is ideologically satisfying, but because it is the only one that can produce the evidence the Act requires.