Caddy, Traefik, NGINX, HAProxy: Who Writes Your Edge Config

Search for a comparison of these four and the results are throughput tables: requests per second, a percentage delta, a winner. Several figures in circulation carry no published methodology — no hardware, no payload size, no concurrency. Behind a 1 Gbit/s access link, the judgement worth making is that the uplink binds before the proxy does. Put a 10 Gbit/s link and heavy TLS termination on one core and that stops holding — then measure, on your hardware. Below it, the benchmark is irrelevant rather than wrong.

The question an operator has arrives at 3am: what did this proxy do to in-flight connections when the config changed, and what is it running right now? Four different answers, and they follow from where the config came from, not from performance. This is the layer above bare-metal ingress with MetalLB and kube-vip — that article decides how packets reach a node; this one decides which L7 process you then operate.

Three Ways an Edge Config Gets Written

Three config-provenance models, not four, because NGINX and HAProxy share one. Hand-written: a human edits a file in version control and a signal tells a running process to re-read it. Discovered: the proxy learns its routing elsewhere. Traefik "gets its routing configuration from providers: whether an orchestrator, a service registry, or a plain old configuration file," and in its own Docker quick start "the whoami application routing configuration comes from docker in the form of a label attached to the whoami container." Owning API: the process holds the configuration and writes it back to disk — Caddy is configured through an administration REST endpoint whose default address is localhost:2019, and "the latest configuration will be saved to disk after any changes (unless disabled)."

Any fifth proxy sorts into one of the three, and drift, reload semantics and auditability follow from which. A Caddyfile is short not because Caddy is less capable but because convention carries the file — it serves all sites over HTTPS by default, keeping managed certificates renewed and redirecting port 80 to 443 automatically.

{
	# Bind the admin API to a permissioned unix socket rather than
	# the default localhost:2019 TCP listener.
	# A packaged caddy user often cannot create a socket under /run;
	# pick a directory it owns, or adjust the unit.
	admin unix//run/caddy/admin.sock

	email ops@example.com

	# Do not try to install the local root into the system trust store.
	# Without this, first use of the root prompts for a password, which
	# on a headless box has nobody to answer it. Distribute it yourself.
	skip_install_trust
}

edge.example.com {
	encode zstd gzip

	reverse_proxy app-a:8080 app-b:8080 {
		lb_policy least_conn
		health_uri /healthz
		health_interval 5s
		health_timeout 2s
	}
}

internal.example.local {
	# Locally-trusted self-signed cert instead of a public ACME CA.
	tls internal

	reverse_proxy metrics:9090
}

The discovered model looks nothing like that, and the difference is not syntax. The routing configuration is not in a file you review; it is an attribute of the workload.

services:
  traefik:
    image: traefik:v3.7.14
    command:
      - "--configFile=/etc/traefik/traefik.yml"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "./traefik.yml:/etc/traefik/traefik.yml:ro"
      # acme.json must ALREADY EXIST on the host as an empty file, mode
      # 0600, before the first up: Docker creates a missing bind source
      # as a directory, and the resolver then cannot write the ACME store.
      #   touch acme.json && chmod 600 acme.json
      - "./acme.json:/acme.json"
      # :ro stops Traefik writing the socket file. It does NOT make the
      # Docker API read-only — see the security note below.
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

  whoami:
    image: traefik/whoami:v1.10
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`edge.example.com`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls.certresolver=myresolver"
      - "traefik.http.routers.whoami.middlewares=ratelimit@docker"
      - "traefik.http.middlewares.ratelimit.ratelimit.average=50"
      - "traefik.http.services.whoami.loadbalancer.server.port=80"

Click to zoom

What Each One Does When You Change It

The benchmark cannot tell you this; all four projects document it precisely. NGINX is strictest about never serving a broken config. On a HUP: "The master process first checks the syntax validity, then tries to apply new configuration, that is, to open log files and new listen sockets. If this fails, it rolls back changes and continues to work with old configuration." A typo in nginx.conf costs a failed reload, not an outage.

events {
    worker_connections 4096;
}

http {
    resolver 127.0.0.53 valid=30s ipv6=off;

    upstream app {
        zone app 64k;
        least_conn;
        server app-a.internal:8080 max_fails=3 fail_timeout=10s;
        server app-b.internal:8080 max_fails=3 fail_timeout=10s;
    }

    server {
        listen 443 ssl;
        http2 on;
        server_name edge.example.com;

        ssl_certificate     /etc/ssl/edge/fullchain.pem;
        ssl_certificate_key /etc/ssl/edge/privkey.pem;
        ssl_protocols       TLSv1.2 TLSv1.3;

        location / {
            proxy_pass       http://app;
            proxy_set_header Host              $host;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

HAProxy's reload is a process swap, not a worker swap, and the subtlety is in the sockets. A graceful stop on SIGUSR1 "consists in only unbinding from listening ports, but continue to process existing connections until they close." In master-worker mode "the master process reacts to the SIGUSR2 signal by reexecuting itself with the -sf parameter followed by the PIDs of the workers." Between unbind and new bind there is a window. -x <unix_socket> closes it: the new process retrieves "any listening sockets from the old process, and use them instead of trying to bind new ones," which the management guide calls "useful to avoid missing any new connection when reloading the configuration on Linux." The master uses that path automatically; outside master-worker mode, the stats socket needs expose-fd listeners.

global
    log stdout format raw local0
    # Required: the ACME client below is an experimental feature.
    expose-experimental-directives
    httpclient.resolvers.prefer ipv4
    # Runtime socket. Start the process in master-worker mode with -W (or
    # -Ws under systemd); the bare global master-worker keyword is
    # deprecated in 3.3 and rejected from 3.5, though
    # master-worker no-exit-on-failure still parses. In that mode
    # the master hands listening
    # sockets to the new workers on reload, so -x is only needed outside it.
    stats socket /var/run/haproxy/admin.sock mode 660 level admin

defaults
    # Without this, the log-format below has no log address to attach to
    # and HAProxy warns that it is ignored.
    log     global
    mode    http
    timeout connect 5s
    timeout client  30s
    timeout server  30s
    option  httplog

# NOTE: HAProxy's ACME tutorial states this works with a single load
# balancer, but NOT an active-active pair. Do not copy this section onto
# a second node and expect both of them to renew.
acme letsencrypt
    directory https://acme-staging-v02.api.letsencrypt.org/directory
    contact   ops@example.com
    challenge http-01
    map       virt@acme

crt-store edge
    crt-base /etc/haproxy/ssl
    key-base /etc/haproxy/ssl
    # Declaring the certificate in a crt-store is what allows HAProxy to
    # start with no certificate on disk: a temporary key pair is used
    # until ACME issues the real one.
    load crt "edge.pem" acme letsencrypt domains "edge.example.com" alias "edge"

frontend https_in
    bind :80
    bind :443 ssl
    # HAProxy answers the HTTP-01 challenge itself, out of the acme map.
    http-request return status 200 content-type text/plain lf-string "%[path,field(-1,/)].%[path,field(-1,/),map(virt@acme)]\n" if { path_beg '/.well-known/acme-challenge/' }
    http-request redirect scheme https unless { ssl_fc }
    ssl-f-use crt "@edge/edge"
    default_backend app

backend app
    balance roundrobin
    option  httpchk GET /healthz
    server  app-a app-a.internal:8080 check
    server  app-b app-b.internal:8080 check

Traefik splits the problem in two. Elements in the install configuration "set up connections to providers and define the entrypoints Traefik will listen to (these elements don't change often)"; the routing configuration "can change and is seamlessly hot-reloaded, without any request interruption or connection loss." Routes change freely; an entryPoint or a provider connection does not — that is a restart. Caddy's documentation recommends caddy reload over caddy validate against a live instance because reload "provisions the new config within the running process... If provisioning fails, the active config keeps running; if it succeeds, the new config is applied." Offline, caddy validate is the stronger check — it "loads and provisions all of its modules as if to start the config."

Where the Running Config Stops Matching Git

Each model has its own drift vector and one command that exposes it. HAProxy's is sharpest, and documented as a trap. The Runtime API "keeps changes in memory until the next reload or restart. It does not write changes to the configuration file on disk." The management guide is blunter: "A dynamic server not present in the configuration won't be restored after a reload operation" — the CLI-created server has to be written into the file by hand. HAProxy 3.4 extends this to whole backends: add backend instantiates one in the unpublished state and publish backend activates content switching to it.

Caddy's vector runs the other way: the process writes the file. The latest configuration is saved to disk after any change by default and caddy run --resume restarts from it, so a POST to the admin API becomes durable without touching your Caddyfile. Caddy ships this as a first-class mode: the shipped caddy-api.service "uses the --resume option which will start Caddy using the autosave.json which is persisted by default." Legitimate — a problem only if half the team believes the repository is authoritative while the unit file says otherwise.

NGINX has the narrowest vector, by omission: open-source nginx has no reload-free API for changing upstream servers. Modifying a server group on the fly is an F5 NGINX Plus feature requiring NGINX Plus R13 or later. So the file genuinely is the configuration — a sovereignty property, bought with a reload on every backend change. Traefik's vector is the inverse of HAProxy's: nothing drifts in memory, because the orchestrator is the source. What drifts is review — the rule, the middleware and the resolver assignment live on a container or a custom resource, so reading traefik.yml is reading the wrong file.

#!/usr/bin/env bash
set -euo pipefail

drift=0
note() { printf '%s\n' "$*" >&2; }

# --- NGINX: -T is -t plus a dump of the config it actually parsed.
if command -v nginx >/dev/null; then
  nginx -t
  nginx -T >/tmp/nginx.running.conf
  note "nginx: parsed config dumped to /tmp/nginx.running.conf"
fi

# --- HAProxy: syntax check the file, then ask the running process
# which servers it actually has. A server only in memory is drift.
if command -v haproxy >/dev/null; then
  haproxy -c -f /etc/haproxy/haproxy.cfg
  socket=/var/run/haproxy/admin.sock
  if [ -S "$socket" ]; then
    echo "show servers state" | socat stdio "$socket" >/tmp/haproxy.state
    while read -r _ bename _ svname _; do
      [ -n "${svname:-}" ] || continue
      grep -qE "^[[:space:]]*server[[:space:]]+${svname}\b" \
        /etc/haproxy/haproxy.cfg ||
        { note "DRIFT haproxy: ${bename}/${svname} is live but not in the file"; drift=1; }
    done < <(tail -n +3 /tmp/haproxy.state)
  fi
fi

# --- Caddy: adapt the Caddyfile to JSON and diff it against what the
# admin API says is loaded. Any difference means someone used the API.
if command -v caddy >/dev/null; then
  caddy validate --config /etc/caddy/Caddyfile
  caddy adapt --config /etc/caddy/Caddyfile >/tmp/caddy.expected.json
  curl -fsS --unix-socket /run/caddy/admin.sock \
    http://localhost/config/ >/tmp/caddy.running.json
  diff <(jq -S . /tmp/caddy.expected.json) \
       <(jq -S . /tmp/caddy.running.json) >/dev/null ||
    { note "DRIFT caddy: running config differs from the adapted Caddyfile"; drift=1; }
fi

exit "$drift"

Click to zoom

TLS: Who Holds the ACME Account

This is where the four separate most sharply, and not in the order most assume. Caddy clusters. Automatic HTTPS is on by default — public DNS names get certificates from a public ACME CA such as Let's Encrypt or ZeroSSL, internal hostnames and IP addresses get locally-trusted self-signed ones — and the clustering sentence is unambiguous: "Any Caddy instances that are configured to use the same storage will automatically share those resources and coordinate certificate management as a cluster." Two replicas, one storage backend both can write, done.

Traefik does not, and its own ACME reference says so outright: "it is not possible to run multiple instances of Traefik 2.0 with Let's Encrypt enabled, because there is no way to ensure that the correct instance of Traefik receives the challenge request, and subsequent responses" — wording the current v3 page still carries. It names two escapes: Traefik Enterprise, "which includes distributed Let's Encrypt as a supported feature," or keeping Traefik Proxy and achieving "LetsEncrypt HA … by using a Certificate Controller such as Cert-Manager." This is how a working single-node Traefik becomes a broken HA pair: the second replica is an architectural decision, not a replicas: 2.

entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"

api:
  dashboard: true
  # api.insecure exposes the API and dashboard on the entryPoint named
  # "traefik". It defaults to false. Leave it that way.

providers:
  # providers.providersThrottleDuration is a sibling of the provider,
  # not a field inside it: "This option cannot be set per provider, but
  # the throttling algorithm applies to each of them independently."
  # Only the most recent refresh event inside this window is taken into
  # account; the rest are discarded. Default is 2s.
  providersThrottleDuration: 2s
  docker:
    exposedByDefault: false

certificatesResolvers:
  myresolver:
    acme:
      email: ops@example.com
      storage: acme.json
      httpChallenge:
        # Must be reachable by Let's Encrypt through port 80.
        entryPoint: web

# SINGLE INSTANCE ONLY. Traefik's ACME reference states that multiple
# instances cannot run with Let's Encrypt enabled. For two replicas,
# use Traefik Enterprise's distributed Let's Encrypt, or keep Traefik
# Proxy and move issuance to a certificate controller such as
# cert-manager. Do not scale this file to replicas: 2.

HAProxy's ACME client is newer and narrower. Per the 3.4 configuration manual the ACME section "allows to configure HAProxy as an ACMEv2 client," but "this feature is experimental meaning that expose-experimental-directives must be in the global section so this can be used," and is "limited to the http-01, dns-01 or dns-persist-01 challenges for now." HAProxy's tutorial adds the constraint: "this feature works with a single load balancer, but not an active-active pair." The stock nginx binary does not speak ACME — though the project ships one. ngx_http_acme_module "implements the automatic certificate management (ACMEv2) protocol" and is "available in a prebuilt nginx-module-acme package," but it is a separate dynamic module: install it and load_module it, and only then does the account live in nginx. Until then the renewal loop sits in an external client beside it — more moving parts, and the same decision step-ca with cert-manager makes one layer down.

Four Failure Modes These Mechanics Predict

None needs measuring. Each falls out of the mechanics above — which is what makes them worth rehearsing before they happen.

• Stale discovery. The orchestrator becomes unreachable and Traefik keeps serving the last routing configuration it saw — indistinguishable from healthy until a route you deleted is still live. First command: query the provider, not the proxy.
• Coalesced config events. Several containers restart inside Traefik's providersThrottleDuration window, and "only the most recent one is taken into account, and all others are discarded." Each event carries the provider's whole current view, not a per-container delta, so the effect is delayed convergence, not an omission: one reload per window, settling on the latest container list. Expect a briefly stale route during a bulk deploy, not a missing service. First command: wait out the window, then read the provider — docker inspect, or the route objects.
• A runtime-added server lost on reload. Someone adds a server through HAProxy's Runtime API during an incident, nobody writes it into haproxy.cfg, and the next reload — a renewal, a kernel patch — silently removes it. First command: show servers state, diffed against the file.
• A trust-store surprise on internal names. Caddy serves internal hostnames with a locally-trusted self-signed certificate, and "the first time a root key is used, Caddy will try to install it into the system's local trust store(s). If it does not have permission to do so, it will prompt for a password." On a headless box nothing answers that prompt. Fix: skip_install_trust, as the Caddyfile above sets, and distribute the root deliberately.

The Audit and Security Surface You Inherit

Each config model hands an auditor a different document and an attacker a different door. Traefik's Docker provider "requires access to the docker socket to get its dynamic configuration," and its documentation does not soften it: "Accessing the Docker API without any restriction is a security concern: If Traefik is attacked, then the attacker might get access to the underlying host." That is the price of discovery: on a single host the orchestrator is root. Mounting the socket :ro does not help — that stops Traefik writing the socket file, not the API behind it. The second exposure is a switch: api.insecure exposes the API and dashboard on the entryPoint named traefik and defaults to false; leaving it alone is the whole control.

Caddy's surface is its admin endpoint, defaulting to localhost:2019 — a reasonable default, not a boundary. Its documentation says what to do when that is not enough: "If you are running untrusted code on your server (yikes 😬), make sure you protect your admin endpoint by isolating processes, patching vulnerable programs, and configuring the endpoint to bind to a permissioned unix socket instead." Anything that reaches it can rewrite the running configuration and have it autosaved. NGINX and HAProxy have the smallest surface — a file plus a signal — though HAProxy's runtime socket deserves the same seriousness: the same power, without the persistence.

The fourth risk is not technical. In November 2025 Kubernetes SIG Network and the Security Response Committee announced Ingress NGINX's retirement — best-effort maintenance until March 2026, after which "there will be no further releases, no bugfixes, and no updates to resolve any security vulnerabilities that may be discovered," existing deployments keep working and the artifacts stay available. That date has passed. The January 2026 Steering and Security Response Committees statement quoted below calls it "critical infrastructure for about half of cloud native environments," attributing that figure to internal Datadog research whose method it does not describe.

Note what that is evidence of: not a verdict on nginx the proxy, which is maintained and shipping — the retired project was a controller built around it. It is evidence that maintainer capacity is a selection criterion carrying a feature matrix's weight, and that "widely deployed" is not "well-resourced." The same reasoning belongs in any Kubernetes upgrade and deprecated-API plan.

Exit Ramps: What Leaving Costs

Four projects, four licences. Caddy is Apache License 2.0, Traefik is MIT, nginx a two-clause BSD-style licence. HAProxy is the interesting one: it keeps the GPL for the code but applies the LGPL to its exportable include files, so it "is expressly permitted to write, compile, link and load non-GPL code using the LGPL header files and not to distribute them if it causes a legal problem." All four are open source and forkable: none can strand you the way a proprietary appliance can.

The cost of leaving is not the licence; it is that no two of them share a configuration format. A Caddyfile, an nginx.conf, a haproxy.cfg and Docker labels are four incompatible descriptions of similar intent, and the only documented bridge runs one way: Caddy's native config language is JSON, the Caddyfile is itself a config adapter, and Caddy's own adapter list includes one for NGINX config — "a config adapter could turn your NGINX config into Caddy JSON." A migration aid, not a compatibility layer. Price an edge migration the way the OpenShift lock-in and exit-cost analysis does: count the artefacts with no equivalent on the other side.

On Kubernetes there is one real portability contract: the Gateway API. Its implementations page lists Traefik Proxy, NGINX Gateway Fabric and N42 Gateway — the last described there as "a community driven Ingress and Gateway API implementation for HAProxy" — among conformant Gateway controllers. Read the bar before calling that parity: self-submitted reports against one of the two most recent releases, maintained by the implementors — not a continuously verified badge. They differ in maturity: the HAProxy-side entry is an alpha-tagged release against an older minor of the specification; Traefik's and NGINX Gateway Fabric's are stable releases against the current minor. Traefik's own documentation claims support for v1.6.2, with full HTTPRoute core coverage plus Standard-channel BackendTLSPolicy, GRPCRoute and TLSRoute, and Experimental TCPRoute. If an HTTPRoute is your routing contract, it moves. Nothing else does — which is why Argo Rollouts canary and blue-green ports more easily than the proxy underneath it.

Choosing, and the Long Game

The decision rule follows from who is on call. If configuration is written by humans, reviewed in pull requests, and the person paged at 3am wrote the file, choose NGINX or HAProxy — and between them on protocol surface, not provenance, since they share a model. HAProxy for the runtime socket, dynamic servers and backends — the socket's power being its drift vector. NGINX for the narrowest gap between file and process, paid for with a reload on every backend change.

If configuration is produced by an orchestrator and the routing decision genuinely belongs to the team deploying the workload, choose Traefik — accepting two consequences: the Docker socket or cluster read permissions it needs, and a certificate story solved outside Traefik Proxy once there is a second replica. If you want the shortest path from zero to correct TLS and can live with the process owning the config file, choose Caddy — and decide in writing whether the Caddyfile or autosave.json is authoritative, because the shipped systemd units support both.

Over a decade, three properties still matter and none is in today's release notes. The licence, because it decides whether a fork is available when maintenance is not — ingress-nginx shows that is not theoretical. The config-production model, because it decides whether your edge is audited by reading a repository or only by interrogating a live process. And whether routing intent lives in a vendor-neutral resource — the only part of this choice that survives replacing the proxy itself. If the edge is doing anything other than what the repository says, the workload-identity and mTLS design above it rests on an unverified assumption.