Cilium

eBPF-powered networking for Kubernetes

What Is Cilium?

Cilium is an eBPF-based CNI (Container Network Interface) plugin for Kubernetes. It replaces traditional iptables-based networking with eBPF programs that run directly in the kernel, providing networking, observability, and security at a fundamentally different performance tier. Cilium is a CNCF graduated project and the default CNI in Google GKE, AWS EKS (optional), and Azure AKS (optional).

How Cilium Replaces kube-proxy

In a standard Kubernetes cluster, kube-proxy watches the API server for Service/Endpoint changes and writes iptables DNAT rules to implement service load balancing. Cilium replaces this entirely with eBPF hash map lookups.

    $ # Verify Cilium has replaced kube-proxy
    $ cilium status
    KubeProxyReplacement:    True
      - ClusterIP:           Enabled
      - NodePort:            Enabled
      - ExternalIPs:         Enabled
      - LoadBalancer:         Enabled
      - HostPort:            Enabled
    

    $ # View eBPF service map
    $ cilium bpf lb list
    SERVICE ADDRESS       BACKEND ADDRESS
    10.96.0.1:443         10.244.0.5:6443
    10.96.0.10:53         10.244.1.12:53, 10.244.2.8:53
    10.96.45.200:80       10.244.1.20:8080, 10.244.3.15:8080, 10.244.2.31:8080
  

eBPF-Based Network Policy

Cilium enforces Kubernetes NetworkPolicy using eBPF programs instead of iptables. But Cilium goes far beyond standard K8s NetworkPolicy — it provides L7-aware policy that can filter by HTTP path, gRPC method, Kafka topic, and DNS domain.

Policy Enforcement Layers

L7 Policy: Cilium's Unique Strength

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "api-access"
spec:
  endpointSelector:
    matchLabels:
      app: api-server
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP
      rules:
        http:
        - method: "GET"
          path: "/api/v1/products"
        - method: "GET"
          path: "/api/v1/health"

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "kafka-access"
spec:
  endpointSelector:
    matchLabels:
      app: kafka-broker
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: order-service
    toPorts:
    - ports:
      - port: "9092"
        protocol: TCP
      rules:
        kafka:
        - apiKey: "produce"
          topic: "orders"
        - apiKey: "fetch"
          topic: "orders"

rules:
  http:
  - method: "POST"
    path: "/myservice.UserService/GetUser"
  - method: "POST"
    path: "/myservice.UserService/ListUsers"

spec:
  endpointSelector:
    matchLabels:
      app: backend
  egress:
  - toFQDNs:
    - matchPattern: "*.amazonaws.com"
    - matchName: "api.stripe.com"
  - toEndpoints:
    - matchLabels:
        io.kubernetes.pod.namespace: kube-system
        k8s-app: kube-dns
    toPorts:
    - ports:
      - port: "53"

Identity-Based Security

Traditional network policy works with IP addresses. In Kubernetes, pod IPs are ephemeral — they change on every restart, reschedule, or scale event. Cilium solves this by assigning numeric identities to pods based on their labels, and enforcing policy on identity rather than IP.

    $ # View identity assignments
    $ cilium identity list
    ID       LABELS
    12345    k8s:app=frontend k8s:env=prod
    12346    k8s:app=api-server k8s:env=prod
    12347    k8s:app=redis k8s:env=prod
    

    $ # View per-endpoint eBPF policy
    $ cilium bpf policy get 1234
    IDENTITY   DIRECTION   PORT/PROTO   ACTION
    12345      ingress     80/TCP       allow
    12346      ingress     80/TCP       allow
    0          ingress     0/ANY        deny
  

Hubble: eBPF-Powered Observability

Hubble is Cilium's built-in network observability layer. Because Cilium's eBPF programs already see every packet at tc-bpf and XDP hook points, Hubble can extract flow metadata without injecting sidecar proxies or tapping interfaces. Zero additional overhead for flow visibility.

    $ # Watch live flows
    $ hubble observe --namespace production
    TIMESTAMP              SOURCE                      DESTINATION                 TYPE      VERDICT
    Mar 31 10:23:45.123    production/frontend-7b4c     production/api-server-3d2    L4/TCP    FORWARDED
    Mar 31 10:23:45.125    production/api-server-3d2    production/redis-5a1f        L4/TCP    FORWARDED
    Mar 31 10:23:45.200    production/api-server-3d2    kube-system/coredns-6d4      L4/UDP    FORWARDED
    Mar 31 10:23:46.001    production/unknown-pod-x     production/api-server-3d2    L4/TCP    DROPPED (policy denied)
    

    $ # Filter by drop verdict
    $ hubble observe --verdict DROPPED --namespace production
    Mar 31 10:23:46.001    production/unknown-pod-x     production/api-server-3d2    DROPPED (Policy denied)
    

    $ # HTTP-level flows
    $ hubble observe --protocol http --namespace production
    Mar 31 10:23:45.124    production/frontend-7b4c -> production/api-server-3d2
      HTTP/1.1 GET /api/v1/products -> 200 OK (12ms)
    

    $ # DNS flows
    $ hubble observe --protocol dns
    Mar 31 10:23:45.200    production/api-server-3d2 -> kube-system/coredns-6d4
      DNS Query A api.stripe.com -> 104.16.0.1, 104.16.0.2 (TTL: 300)
  

Cilium Service Mesh (Sidecar-Free)

Traditional service meshes (Istio, Linkerd) inject a sidecar proxy (usually Envoy) into every pod. This adds 2 extra containers per pod (init + sidecar), consumes significant CPU/memory, and adds latency to every request. Cilium offers a sidecar-free service mesh using eBPF.

Why Cilium Is Winning Over Calico/iptables-Based CNIs

Debugging Tools

        $ # List all loaded eBPF programs
        $ bpftool prog list
        6: sched_cls  name cil_from_cont  tag a1b2c3  gpl
            loaded_at 2024-01-15  uid 0
            xlated 4096B  jited 2480B  memlock 8192B
            map_ids 4,5,6,7
        

        $ # Dump an eBPF map's contents
        $ bpftool map dump id 4
        key: 0a f4 01 05  value: 00 00 30 39
        # (10.244.1.5 -> identity 12345)
        

        $ # Show program stats
        $ bpftool prog show id 6 --json | jq
      

        $ # Trace XDP actions on eth0
        $ bpftrace -e '
          tracepoint:xdp:xdp_redirect {
            @actions[args->act] = count();
          }'
        @actions[2]: 15234   # XDP_PASS
        @actions[1]: 8921    # XDP_DROP
        @actions[4]: 3417    # XDP_REDIRECT
        

        $ # Histogram of packet sizes
        $ bpftrace -e '
          kprobe:netif_receive_skb {
            @bytes = hist(((struct sk_buff *)arg0)->len);
          }'
      

      $ # Overall Cilium health status
      $ cilium status
          /¯¯\
       /¯¯\__/¯¯\    Cilium:          OK
       \__/¯¯\__/    Operator:        OK
       /¯¯\__/¯¯\    Hubble:          OK
       \__/¯¯\__/    ClusterMesh:     disabled
          \__/
      KubeProxyReplacement:    True
      Encryption:              Wireguard
      

      $ # List endpoints and their identity/policy status
      $ cilium endpoint list
      ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS
      1234       Enabled            Enabled           12345      k8s:app=frontend
      1235       Enabled            Enabled           12346      k8s:app=api-server
      

      $ # Inspect eBPF CT (conntrack) table
      $ cilium bpf ct list global
      TCP IN  10.244.1.5:45321 -> 10.96.0.10:53  expires=300 rx_packets=2 tx_packets=2
      TCP IN  10.244.1.5:52100 -> 10.244.2.8:80  expires=120 rx_packets=45 tx_packets=38
      

      $ # Run connectivity test
      $ cilium connectivity test
      ✅  All 46 tests (172 actions) successful, 0 tests skipped, 0 scenarios skipped.
    

Cilium Architecture Summary

1. K8s Event

   Service created, NetworkPolicy applied, pod scheduled

2. cilium-agent

   Translates K8s objects into eBPF map entries

3. eBPF Maps Updated

   Atomic per-entry updates, no traffic interruption

4. eBPF Programs

   Read maps at packet time — new policy effective immediately