eBPF and Cilium Are Eating the Service Mesh: What Zero-Trust Kubernetes Networking Looks Like Without Sidecars

The service mesh market has been running a slow-motion rationalization for three years, and by early 2026 the verdict is legible: GKE Dataplane V2 ships Cilium by default, AKS offers Azure CNI Powered by Cilium as its flagship network plugin, and AWS greenfield EKS clusters are increasingly Cilium-native. When three major hyperscalers cast the same vote in their managed offerings, that is not a trend — it is a consolidation event. Cilium graduated from the CNCF in October 2023 — the first CNI to reach graduation — and its Hubble observability layer has moved from optional add-on to operational baseline. The hyperscaler vote matters not because hyperscalers are right about everything, but because they run enough clusters at enough scale to surface the failure modes that matter.

The Cilium versus Istio framing that has dominated conference talks since 2021 is, at this point, the wrong frame. It presupposes a binary choice between two tools that solve overlapping but distinct problems. The right question is harder and more interesting: what security guarantees does your threat model actually require at the network layer, and what operational complexity are you willing to carry permanently? Most teams installing Istio are solving a perceived compliance problem — "zero trust means mTLS, mTLS means Istio" — without examining whether Cilium's native WireGuard encryption and identity-based L3/L4/L7 policy enforcement already satisfies that requirement without the certificate management overhead, sidecar injection complexity, and control-plane blast radius. For the workloads where Istio's traffic management capabilities (canary splits, circuit breaking, external authorization) genuinely add value, this article is honest: use Istio ambient mode. But know precisely what you are buying and what the realistic running cost is.

This article is a production-grade walkthrough of the Cilium networking and security model, written for platform engineers who are either deploying Cilium for the first time or hardening an existing installation toward a defensible zero-trust posture. It covers the eBPF kernel architecture with specific hook points, identity-based policy with real YAML including an L7 HTTP rule, WireGuard transparent encryption, Hubble flow observability, and an honest assessment of where Istio ambient mode still justifies its operational weight. The goal is a default-deny policy corpus you can ship to a growing cluster without it becoming a maintenance liability.

What Zero Trust Actually Requires at the Network Layer in Kubernetes

Zero trust in the network context has a concrete technical meaning that gets obscured by marketing: no implicit trust based on network location. In a traditional perimeter model, traffic inside the cluster VLAN is implicitly trusted. In a zero-trust model, every connection is authenticated, authorized against an explicit policy, and logged. The key word is every — not just north-south traffic crossing the load balancer, but east-west pod-to-pod traffic, and ideally node-to-node traffic in transit.

In Kubernetes, translating that principle into enforcement requires three things: (1) a cryptographically verifiable identity for every workload that does not rely on IP addresses — which are ephemeral and trivially spoofable within a cluster; (2) policy evaluated and enforced at a kernel level, before traffic reaches the application, so that policy bypass requires compromising the node itself rather than just the pod; and (3) observability that provides a complete audit trail of what communicated with what, when, and with what verdict. The fourth requirement — mTLS for data in transit — is often assumed to be unique to Istio, but it is not. These requirements map directly to Cilium's capability set. The gap — and it is a real one — is in Layer 7 traffic management that goes beyond policy enforcement into request routing, retries, and fault injection.

The eBPF Kernel Architecture: Why In-Kernel Policy Enforcement Changes the Performance Equation

eBPF (extended Berkeley Packet Filter) is a kernel subsystem that allows verified programs to run inside the Linux kernel in response to events, without modifying kernel source code or loading kernel modules. The verification step — run by the kernel's eBPF verifier at load time — ensures programs cannot crash the kernel, leak memory, or loop indefinitely. Cilium compiles its network policy engine to eBPF bytecode and attaches it to specific kernel hook points where it intercepts packets before they ever reach userspace. The performance implication is significant: in the traditional iptables path, every packet triggers a linear traversal of rule chains. eBPF programs are JIT-compiled to native machine code and execute in nanoseconds against pre-built hash map lookups.

The three hook points Cilium uses most extensively are XDP (eXpress Data Path), TC (Traffic Control), and socket/sockops hooks. XDP attaches at the earliest possible point in the receive path — immediately after the NIC driver delivers a packet, before kernel memory allocation for the sk_buff network buffer structure. This is where Cilium performs DDoS mitigation and load-balancing for kube-proxy replacement. TC hooks attach at the Traffic Control ingress and egress queuing discipline layer, just before packets enter the network stack or leave it; this is where Cilium applies L3/L4 identity policy and enforces default-deny. Socket and sockops hooks intercept at the socket layer for L7 traffic redirection — when a connection requires HTTP or gRPC inspection, Cilium redirects it to a per-node Envoy instance (not a per-pod sidecar) via the sockops hook, gets the L7 verdict, and then applies the TC-layer allow or drop.

Click to zoom

The practical performance consequence: Cilium benchmarks consistently show 30–50% lower CPU overhead for East-West policy enforcement compared to iptables-based implementations, and 2–3x lower latency at the tail (p99) under high connection churn. These numbers matter less for low-traffic dev clusters and enormously for API gateway nodes or data-plane services handling tens of thousands of connections per second. The secondary consequence — equally important for security — is that kube-proxy replacement means there are no iptables rules to enumerate, inspect, or accidentally weaken through a kubectl flag that bypasses the chain.

Cilium's Network Policy Model: Identity-Based, Not IP-Based

Cilium implements the standard Kubernetes NetworkPolicy API for compatibility, but its native policy object — CiliumNetworkPolicy — is substantially more expressive. The critical architectural difference is that CiliumNetworkPolicy selects endpoints by identity, computed from the pod's label set plus its namespace and service account. When a new pod starts, the Cilium agent on its node assigns it an identity, distributes that identity mapping to BPF maps on all nodes, and the policy evaluation on every node in the cluster is updated automatically. This removes the stale-IP-rule problem endemic to IP-based NetworkPolicy implementations under autoscaling conditions.

A concrete L3/L4 policy that enforces namespace-scoped communication with explicit port allowances looks like this:

# L3/L4 CiliumNetworkPolicy — production-grade default-deny for the api-server service.
# Allows ingress only from pods labeled app=frontend in the same namespace,
# and egress only to the postgres service on port 5432 and to DNS on port 53.
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: api-server-policy
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: api-server
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
  egress:
    - toEndpoints:
        - matchLabels:
            app: postgres
      toPorts:
        - ports:
            - port: "5432"
              protocol: TCP
    - toEndpoints:
        - matchLabels:
            "k8s:io.kubernetes.pod.namespace": kube-system
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP

The identity model connects directly to the open-source principle: Cilium's identity computation is transparent, reproducible, and auditable. The identity assigned to any given pod is deterministically derived from its metadata — you can compute it from first principles without running special tooling. This matters for audit evidence: a compliance reviewer can inspect a CiliumNetworkPolicy and trace exactly which workloads are in scope without needing to query runtime BPF maps.

Click to zoom

L7 Policy Without Sidecars: How Cilium Enforces HTTP and gRPC Rules in the Kernel

The canonical objection to Cilium handling L7 policy has been: "Cilium is a CNI, not a service mesh — it can't parse HTTP headers." This was accurate in 2020. It is not accurate today. Cilium's L7 policy works by redirecting qualifying connections — those that match a toPorts rule with an http or kafka or grpc descriptor — to a per-node Envoy proxy via the sockops hook. The Envoy instance is managed by the Cilium operator, runs on the host network namespace (not in any pod), and handles L7 inspection for all pods on that node. The result is that a single shared Envoy instance on a node with forty pods replaces forty per-pod Envoy sidecars — with the attendant reduction in resource overhead, startup latency, and certificate churn.

An L7 HTTP rule in CiliumNetworkPolicy looks like this:

# L7 HTTP CiliumNetworkPolicy — restricts the api-server to specific HTTP paths and methods.
# Only GET /api/v1/* from frontend pods is permitted; all other paths are denied at L7.
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: api-server-l7-policy
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: api-server
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "^/api/v1/.*"
              - method: "POST"
                path: "^/api/v1/orders$"
    - fromEndpoints:
        - matchLabels:
            app: monitoring
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: "GET"
                path: "^/healthz$"
              - method: "GET"
                path: "^/metrics$"

The practical limit worth stating clearly: Cilium's L7 policy enforces allow/deny at the HTTP path and method level. It does not perform advanced traffic management like weighted routing between service versions, circuit breaking on consecutive 5xx responses, or retry budget enforcement. For those capabilities you need a proxy model — either Istio ambient or a dedicated API gateway. The policy model enforces your security perimeter; the traffic management layer optimizes the behavior of allowed traffic. These are different problems and conflating them is a significant source of architectural confusion.

Hubble: What You Get for Observability When Your CNI Is eBPF-Native

Hubble is Cilium's observability layer, and it is the strongest argument for choosing Cilium even in environments that do not immediately need L7 policy. Because Cilium owns the packet path at the kernel level, Hubble can emit a verdict-annotated flow record for every TCP connection and UDP flow in the cluster — source identity, destination identity, destination port, L7 method/path if applicable, verdict (forwarded/dropped), and the specific policy rule that produced the verdict. No sampling. No instrumentation required in the application. No sidecar required.

The practical implications for zero-trust operations: (1) You get a complete audit trail of lateral movement attempts. When a pod that should only talk to the database attempts a connection to the Kubernetes API server, Hubble records it with the DROP verdict and the specific policy rule that blocked it, with nanosecond timestamps. (2) Policy authoring is dramatically safer. Before enforcing a new CiliumNetworkPolicy in deny mode, you can run it in audit mode and observe Hubble flows that would be affected — confirming the policy matches your intent before it breaks production traffic. (3) The Hubble CLI provides real-time flow inspection:

# Stream flows for a specific pod, showing L7 HTTP verdicts
hubble observe --pod production/api-server-6b7d9f-x4p2q --follow

# Show all DROPPED flows cluster-wide in the last 5 minutes
hubble observe --verdict DROPPED --since 5m

# Filter to a specific namespace and show L7 details
hubble observe --namespace production --follow \
  --output json | jq '.flow | {src: .source.labels, dst: .destination.labels, verdict: .verdict, l7: .l7}'

# Identify which policy rule is responsible for drops (policy audit)
hubble observe --verdict DROPPED --namespace production \
  --output json | jq '.flow | {src_identity: .source.identity, dst_port: .destination.port, reason: .drop_reason_desc}'

Hubble's Relay component aggregates flows from all nodes and exposes them via a gRPC API that the hubble observe CLI queries. Hubble Enterprise (the Isovalent commercial offering) adds multi-cluster aggregation and a time-series storage backend — relevant for regulated environments where flow logs must be retained for 90+ days. The open-source Hubble stores flows in a per-node ring buffer (configurable size, typically 4096 flows); for long-term retention in the open-source stack, the recommended pattern is to tail the gRPC API and write flows to your existing observability pipeline — Loki, Elasticsearch, or an OTel collector endpoint.

OpenTelemetry's graduation from CNCF incubating to graduated status means the integration story is now stable: Hubble can emit flows as OTel spans, compatible with any graduated OTel backend. This matters for capabilities requiring unified observability across both network flows and application traces in a single pane.

WireGuard Encryption: Node-to-Node Transparent Encryption Without Certificate Management

The most operationally underrated feature in Cilium's security stack is transparent WireGuard encryption. Enabled by a single Helm value, it establishes a WireGuard tunnel between every pair of nodes in the cluster, encrypting all pod-to-pod traffic in transit at the node level. The Cilium operator automatically manages WireGuard public key exchange — nodes advertise their public keys via the Kubernetes API server, and Cilium configures the WireGuard interfaces accordingly. There is no PKI, no certificate authority, no certificate rotation schedule, no cert-manager dependency, and no CRL infrastructure to operate.

# Cilium Helm values to enable WireGuard transparent encryption
# All node-to-node pod traffic is encrypted; no application changes required.
encryption:
  enabled: true
  type: wireguard
  nodeEncryption: true          # Strict mode: refuse to start if WireGuard unavailable,
                                # preventing accidental unencrypted fallback.
  wireguard:
    userspaceFallback: false    # Require kernel WireGuard (Linux >= 5.6)
    persistentKeepalive: 0      # 0 = disabled; set to 25 if NAT traversal needed

WireGuard in this model is node-to-node, not pod-to-pod at the TLS layer. Every byte leaving a node's WireGuard interface bound for another node is encrypted. The security property you gain is protection against passive eavesdropping on the cluster's physical or virtual network — the threat model covering compromised hypervisor NICs, shared-tenant cloud networks, or a threat actor with span-port access to the switch fabric. What WireGuard does not provide: mutual authentication at the application/pod identity level (a pod on node A impersonating a different pod's identity is not prevented by WireGuard alone — that is the CiliumNetworkPolicy identity enforcement layer's job). The two controls are complementary: WireGuard encrypts the channel; Cilium identity policy enforces authorization within the channel.

Where Istio Ambient Mode Still Wins: Traffic Management Beyond Policy

Istio's ambient mesh mode reached production readiness in Istio 1.22 (May 2024), replacing per-pod sidecars with a node-level ztunnel proxy for L4 mTLS and a per-namespace waypoint proxy for L7 traffic management. This dramatically reduced Istio's resource overhead — the primary operational objection to Istio in 2020–2024 — while preserving the capabilities that Cilium genuinely does not provide: VirtualService weighted traffic splitting for canary deployments, DestinationRule circuit breaking and connection pool management, and ext_authz integration with external authorization services like OPA/Cedar.

Istio ambient mode composes cleanly with Cilium as the CNI: Cilium handles L3/L4 identity policy, kube-proxy replacement, and WireGuard node encryption; Istio ambient's ztunnel handles pod-level mTLS with SPIFFE/X.509 identity; waypoint proxies handle L7 traffic management where needed. This combination gives you the full stack, at the cost of operating both Istiod and Cilium's operator. Whether that complexity is justified depends entirely on whether you are actually using VirtualService and DestinationRule in production, and whether the ext_authz integration is a real requirement versus a theoretical one.

Click to zoom

The honest recommendation for a new cluster: start with Cilium standalone. Run the default-deny policy corpus described in the next section. Add Hubble and wire it to your OTel pipeline. Enable WireGuard. Revisit in 90 days and ask: which of the four Istio-only capabilities (canary splits, circuit breaking, ext_authz, SPIFFE identity) are operational requirements in your environment, not theoretical ones? In most cases the answer is zero or one — and if one, evaluate whether a dedicated API gateway at the ingress layer handles it with less cluster-wide operational surface than Istiod.

Designing a Zero-Trust Network Policy Corpus That Survives Cluster Growth

A default-deny Cilium policy corpus that remains maintainable as the cluster grows needs three structural properties: (1) namespace-level default deny as the baseline, applied via a cluster-scoped CiliumClusterwideNetworkPolicy; (2) service-specific allowlists as CiliumNetworkPolicy objects that live alongside the service's Helm chart or Kustomize overlay, owned by the team that owns the service; and (3) shared infrastructure allow rules — for DNS, the kube-apiserver, metrics scraping, and log collection — managed centrally and version-controlled. The failure mode of a flat policy corpus (all policies in one place, owned by the platform team) is that it becomes a bottleneck: every new service needs a platform team PR to add its allowlist. The failure mode of a fully delegated corpus (teams write their own policies without review) is policy drift and gaps.

# Step 1: Cluster-wide default-deny baseline.
# Applied first; all namespace-scoped CiliumNetworkPolicies are additive allowlists on top.
#
# IMPORTANT: A policy with ingress: [{}] or egress: [{}] is a wildcard ALLOW, not a deny.
# True default-deny = endpointSelector selects endpoints with NO ingress/egress rules.
# Cilium denies all traffic for any direction that is not explicitly allowed.
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: default-deny-all
spec:
  endpointSelector: {}        # Matches all endpoints in all namespaces.
                              # No ingress: or egress: section = deny all in both directions.
                              # Namespace-scoped CiliumNetworkPolicies are additive allowlists.
---
# Step 2: Shared infrastructure — allow DNS egress for all pods.
# Without this, DNS resolution fails after the default-deny policy applies.
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: allow-dns-egress
spec:
  endpointSelector: {}
  egress:
    - toEndpoints:
        - matchLabels:
            "k8s:io.kubernetes.pod.namespace": kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP
            - port: "53"
              protocol: TCP
---
# Step 3: Allow Prometheus scraping (metrics collection from all pods).
# Adjust ports to match your application metrics endpoints.
# 9090 is Prometheus's own API port — pod metrics ports vary (common: 9091, 8080, 2112).
# Add only the ports your pods actually expose metrics on.
apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: allow-metrics-ingress
spec:
  endpointSelector: {}
  ingress:
    - fromEndpoints:
        - matchLabels:
            app.kubernetes.io/name: prometheus
      toPorts:
        - ports:
            - port: "9091"              # Common Pushgateway / exporter metrics port
              protocol: TCP
            - port: "8080"
              protocol: TCP

The structural recommendation that survives cluster growth: make the policy corpus part of the service contract, not the platform contract. The platform team owns the cluster-wide default-deny baseline and the shared infrastructure rules. Every service owns its own CiliumNetworkPolicy in its own namespace, checked into its own repository, reviewed by the service team. The platform team provides a policy template and a kyverno test-compatible test harness (see the capabilities page for our policy governance pattern). New services scaffold their policy from the template as part of onboarding — they cannot deploy without one, because the cluster-wide default-deny blocks any undeclared traffic. The same identity-based egress policy applies cleanly to CI runner pods: self-hosted runners on Kubernetes with ARC become ordinary cluster workloads, and Cilium can restrict their egress to only the registry and the API server they need.

The long-game property of this architecture: because Cilium identities are derived from labels rather than IPs, and because the policy objects are YAML checked into version control, the corpus remains legible and auditable across team changes, cluster rebuilds, and Kubernetes version upgrades. The BPF maps are runtime state; the CiliumNetworkPolicy objects are the source of truth. A new engineer joining the team can read the policy for any service and understand its communication surface completely — no need to query runtime state, decode iptables rules, or reverse-engineer firewall logs.

The thesis of engineering sovereignty runs through this architecture at every layer. The eBPF programs are open-source and auditable. The policy objects are YAML in your repository, not opaque SaaS configuration. The WireGuard keys are managed by code you control. The Hubble flow records go to your observability stack, not a vendor's cloud. When a security auditor asks "how do you enforce east-west network segmentation?" the answer is a set of CRD objects in a Git repository with a commit history — not a screenshot from a vendor's console and a hope that the SaaS configuration hasn't drifted.

Putting It Together: The Minimum Viable Zero-Trust Cilium Stack

For a cluster starting from scratch or migrating from a flat-network CNI, the recommended deployment order is: (1) Install Cilium with kube-proxy replacement and WireGuard encryption enabled. (2) Deploy Hubble Relay and wire it to your OTel pipeline. (3) Apply the three-layer default-deny corpus above in audit mode. (4) Run Hubble in audit mode for one full week — longer if the cluster has batch workloads with irregular traffic patterns — and build the service-specific allowlists from observed flows. (5) Switch the cluster-wide policy to enforcement mode namespace by namespace, starting with non-critical workloads. (6) Enable L7 HTTP rules for services where path-level policy is a security requirement, not a curiosity.

The result is a cluster where every east-west connection is authorized by an explicit policy rule derived from workload identity, encrypted in transit by WireGuard, observable at L3/L4/L7 by Hubble, and auditable from a Git-controlled YAML corpus. That satisfies the technical requirements of zero trust for the overwhelming majority of Kubernetes workloads — without a certificate authority, without per-pod sidecar overhead, and without an Istio control plane to operate. Whether you add Istio ambient on top for traffic management is an operational decision about specific capabilities, not a compliance decision about whether your network is zero-trust.

The long game here is a cluster that does not require deep Cilium expertise to operate day-to-day. The policy corpus is readable YAML. The observability is standard OTel. The encryption is a Helm value. When the engineer who built the original Cilium installation moves to a different team, the cluster's security posture survives because the specification is in version control, not in their head.