Skip to content
Menu

Networking8 min read

XDP & tc-bpf

Programmable packet processing at line rate

XDP (eXpress Data Path)

XDP is the earliest programmable hook point in the Linux networking stack. It processes packets at the NIC driver level, before the kernel allocates an sk_buff structure. This makes XDP extraordinarily fast — packets that are dropped or redirected by XDP never touch the normal network stack at all.

Why XDP Is Fast

  • No sk_buff allocation — the kernel's sk_buff (socket buffer) is a heavyweight structure. XDP works with the minimal xdp_md struct, which is just pointers to the raw packet data.
  • No lock contention — runs per-CPU in the driver's NAPI context
  • No netfilter traversal — dropped/redirected packets skip all of iptables, conntrack, etc.
  • JIT-compiled — native machine code execution, not interpreted
  • Batching — processes packets in NAPI poll batches, amortizing overhead

Packet Processing: XDP vs. iptables Path

XDP Path (eBPF)
  1. NIC receives packet

  2. XDP eBPF program

    DROP/PASS/TX/REDIRECT

  3. sk_buff allocation

  4. Normal stack

Tip

Dropped packets never allocate sk_buff. At 10 Gbps, this saves millions of allocations per second.

iptables Path (Traditional)
  1. NIC receives packet

  2. sk_buff allocation

  3. tc ingress

  4. Netfilter PREROUTING

  5. Routing decision

  6. Netfilter INPUT/FORWARD

    O(n) chain walking per packet

  7. Application

Warning

Every packet traverses the full chain even if it will be dropped.

XDP Actions (Return Codes)

Every XDP program must return one of these action codes. The return value tells the NIC driver what to do with the packet.

Action Value Behavior Use Case
XDP_PASS 2 Continue to normal network stack (allocate sk_buff) Default — let the packet through
XDP_DROP 1 Drop the packet immediately at the driver level DDoS mitigation, blocklists, rate limiting
XDP_TX 3 Bounce the packet back out the same NIC it arrived on Load balancer reflection, hairpin NAT
XDP_REDIRECT 4 Forward to a different NIC, CPU, or AF_XDP socket Cross-NIC forwarding, software switches, userspace fast-path (AF_XDP)
XDP_ABORTED 0 Error path — drop and generate trace event Error handling, debugging (triggers xdp:xdp_exception tracepoint)
Packet Arrives
->
XDP Program
->
XDP_PASS (to stack)
XDP Program
->
XDP_DROP (discarded)
XDP Program
->
XDP_TX (back out same NIC)
XDP Program
->
XDP_REDIRECT (to other NIC/CPU/AF_XDP)

XDP Attachment Modes

Mode Where It Runs Performance Requirements
Native (driver mode) In the NIC driver's receive path Best — true pre-sk_buff processing NIC driver must have XDP support (most modern drivers: i40e, mlx5, ixgbe, virtio_net, etc.)
Offloaded On the NIC hardware itself (SmartNIC) Ultimate — zero CPU usage SmartNIC with eBPF offload support (Netronome/Corigine)
Generic (SKB mode) In the network stack (after sk_buff) Slowest — loses the main XDP advantage Works on any NIC. Useful for testing only.
console
console

    $ # Attach XDP program in native mode
    $ ip link set dev eth0 xdpdrv obj xdp_prog.o sec xdp
    

    $ # Attach in generic (SKB) mode — fallback
    $ ip link set dev eth0 xdpgeneric obj xdp_prog.o sec xdp
    

    $ # Verify attachment
    $ ip link show eth0
    2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 xdp qdisc mq state UP
        prog/xdp id 42 tag a1b2c3d4
    

    $ # Detach XDP program
    $ ip link set dev eth0 xdp off
  

Warning

Never use generic mode in production. It defeats the purpose of XDP — packets still go through sk_buff allocation. Always verify your NIC driver supports native XDP. Check with ethtool -i eth0 and verify driver support in the kernel source.

XDP in Production

DDoS Mitigation (Cloudflare)

Cloudflare processes >30 Mpps (million packets per second) of DDoS traffic per server using XDP. Malicious packets are dropped at the driver level before consuming any CPU on the normal stack.

  • IP blocklists stored in eBPF hash maps
  • Rate limiting with per-CPU counters
  • Pattern matching on packet headers
  • Latency: sub-microsecond drop decision

L4 Load Balancing (Katran / Facebook)

Katran is Facebook's open-source XDP-based L4 load balancer. It replaces traditional IPVS/LVS with an eBPF program that does consistent hashing and packet encapsulation at line rate.

  • Consistent hashing via eBPF maps
  • XDP_TX to reflect modified packets back to NIC
  • IPIP/GUE encapsulation for backend forwarding
  • Handles millions of connections per server

XDP Program Structure (C Pseudocode)

An XDP program receives an xdp_md context with pointers to the raw packet data. Here is a simplified firewall that drops packets from a blocked IP list.

c
// SPDX-License-Identifier: GPL-2.0
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <bpf/bpf_helpers.h>

// Map: blocked source IPs (key = __be32 IP, value = drop counter)
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 10000);
    __type(key, __be32);           // source IP
    __type(value, __u64);          // packet counter
} blocked_ips SEC(".maps");

SEC("xdp")
int xdp_firewall(struct xdp_md *ctx) {
    // Packet data boundaries
    void *data     = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;

    // Parse Ethernet header — MUST bounds-check for verifier
    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;

    // Only process IPv4
    if (eth->h_proto != __constant_htons(ETH_P_IP))
        return XDP_PASS;

    // Parse IP header — MUST bounds-check again
    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;

    // Lookup source IP in blocked list
    __u64 *counter = bpf_map_lookup_elem(&blocked_ips, &ip->saddr);
    if (counter) {
        // IP is in blocklist — increment counter and DROP
        __sync_fetch_and_add(counter, 1);
        return XDP_DROP;
    }

    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

Note

Bounds checking is mandatory. Every pointer dereference after packet data must be preceded by a bounds check against data_end. Without it, the verifier rejects the program. This is the most common source of verifier errors for XDP newcomers.

tc-bpf (Traffic Control eBPF)

tc-bpf hooks into the Linux traffic control layer, processing packets after XDP but before netfilter/iptables. Unlike XDP, tc-bpf works with the full sk_buff structure, giving access to richer metadata — at the cost of being slightly slower.

XDP (xdp_md)
  • Runs before sk_buff allocation
  • Ingress only (packet arrival)
  • Minimal context: raw packet pointers only
  • Cannot see: routing decisions, socket info, skb metadata
  • Fastest possible processing
  • Best for: drop/redirect decisions that need only packet headers
tc-bpf (__sk_buff)
  • Runs after sk_buff is created
  • Ingress and egress
  • Rich context: protocol, mark, priority, ifindex, hash, cgroup
  • Can modify: packet data, headers, skb metadata, marks
  • Slightly slower than XDP
  • Best for: classification, NAT, policy that needs metadata

tc-bpf Actions

Action Behavior Equivalent
TC_ACT_OK Continue processing (accept) Similar to XDP_PASS
TC_ACT_SHOT Drop the packet Similar to XDP_DROP
TC_ACT_REDIRECT Redirect to another interface Similar to XDP_REDIRECT
TC_ACT_PIPE Continue to next tc filter in chain No XDP equivalent
TC_ACT_STOLEN Packet consumed, don't free sk_buff No XDP equivalent

Where tc-bpf Fits in the Stack

  1. NIC

    Packet arrives

  2. XDP

    Pre-sk_buff hook

  3. sk_buff created

    Kernel allocates socket buffer

  4. tc ingress (eBPF)

    Classify, mangle, redirect

  5. Netfilter / iptables

    PREROUTING, INPUT...

  6. Application

    Socket recv()

  1. Application

    Socket send()

  2. Routing

    Output route lookup

  3. Netfilter OUTPUT

    POSTROUTING

  4. tc egress (eBPF)

    Shape, police, redirect

  5. NIC

    Packet transmitted

console
console

    $ # Attach tc-bpf to ingress
    $ tc qdisc add dev eth0 clsact
    $ tc filter add dev eth0 ingress bpf da obj tc_prog.o sec tc
    

    $ # Attach tc-bpf to egress
    $ tc filter add dev eth0 egress bpf da obj tc_prog.o sec tc_egress
    

    $ # List attached tc programs
    $ tc filter show dev eth0 ingress
    filter protocol all pref 49152 bpf chain 0
      handle 0x1 tc_prog.o:[tc] direct-action id 15 tag d4e5f6a7 jited
  

Tip

Cilium uses tc-bpf extensively. While XDP handles early-stage fast-path decisions, Cilium's per-endpoint policy enforcement and service load balancing use tc-bpf programs on veth pairs. This gives Cilium access to full sk_buff metadata for identity-based policy, NAT, and L7 inspection hooks.

eBPF/XDP vs. iptables: The Comparison

iptables (Legacy)
  • O(n) linear chain walking per packet — every rule evaluated sequentially
  • Kernel module-based — fixed functionality
  • Conntrack overhead even for stateless rules
  • Difficult to extend — requires writing kernel modules
  • Rules are text-based, not programmatic
  • Scales poorly: 5,000+ rules = measurable latency
  • Atomic updates require replacing the entire chain
  • No egress filtering before routing (nat table workaround)
eBPF/XDP (Modern)
  • O(1) hash map lookups — constant time regardless of rule count
  • Programmable — arbitrary logic in C/Rust
  • No conntrack unless you explicitly implement it
  • Extensible — load new programs without rebooting
  • Full programming language, not chain rules
  • Scales to millions of rules (map size only)
  • Atomic map updates — no traffic interruption
  • Native egress support with tc-bpf

Performance Numbers

Metric iptables XDP (native) Improvement
Packet drop rate ~2-5 Mpps ~24-26 Mpps (single core) ~5-10x
Latency per packet (drop) ~5-15 us ~0.1-0.5 us ~10-100x
Rule lookup (10K rules) O(n) — evaluates each rule O(1) — hash map lookup Constant vs. linear
CPU overhead per packet High (sk_buff + conntrack + chain) Minimal (no sk_buff, JIT-compiled) Significant
Memory per connection (conntrack) ~400 bytes Custom (eBPF map entry) Configurable

Note

Real-world impact at K8s scale: A cluster with 10,000 services generates tens of thousands of iptables rules via kube-proxy. Each packet traverses these rules linearly. Cilium replaces this with eBPF hash map lookups — the performance is independent of the number of services. This is why Cilium can replace kube-proxy.

Decision Guide: XDP vs. tc-bpf vs. iptables

Use Case Best Tool Why
DDoS mitigation / volumetric drops XDP Fastest drop path, before any kernel processing
L4 load balancing XDP Modify + TX/redirect at line rate (Katran pattern)
Per-pod network policy (K8s) tc-bpf Needs sk_buff metadata (cgroup, mark) for identity
NAT / DNAT for services tc-bpf Needs to modify sk_buff fields, conntrack integration
Egress filtering tc-bpf XDP is ingress-only; tc has egress hook
Simple host firewall (low scale) iptables/nftables Good enough, simpler tooling, widely understood
L7-aware decisions tc-bpf + Envoy/proxy eBPF alone can't fully parse L7; Cilium uses an L7 proxy for deep inspection

Tip

They compose, not replace. In a Cilium-based K8s cluster, XDP, tc-bpf, cgroup-bpf, and sometimes even iptables coexist. XDP handles DDoS protection. tc-bpf handles service load balancing and policy. cgroup-bpf handles socket-level operations. Use the right hook for the right job.

Bonus: AF_XDP (Kernel Bypass to Userspace)

AF_XDP: XDP + Userspace Fast Path

XDP_REDIRECT can forward packets to an AF_XDP socket — a special socket type that delivers raw frames directly to userspace, bypassing the entire kernel network stack. This is Linux's answer to DPDK, but without requiring a dedicated NIC or kernel bypass driver.

How It Works

  • XDP program calls bpf_redirect_map() pointing to an XSKMAP
  • Packets land in a shared UMEM ring buffer
  • Userspace application polls the ring — zero-copy path
  • Achieves millions of pps in userspace

Use Cases

  • Custom packet processing engines (IDS/IPS)
  • Userspace protocol implementations
  • Network function virtualization (NFV)
  • Replacing DPDK with a less invasive approach
Solidnines — solidnines.com