Skip to content
Book a call
Menu

Networking8 min read

Netfilter Internals

The kernel framework behind iptables, conntrack, and NAT

Netfilter Hooks: The 5 Hook Points

Netfilter is a kernel framework that provides hook points in the network stack where kernel modules can register callback functions. Every packet traversing the Linux network stack passes through a defined sequence of these hooks. iptables, nftables, conntrack, and NAT are all clients of netfilter.

The 5 Netfilter Hooks

Hook Constant When It Fires
PREROUTING NF_IP_PRE_ROUTING Packet just arrived on an interface, before the routing decision
INPUT NF_IP_LOCAL_IN Packet is destined for this host (after routing decided it's local)
FORWARD NF_IP_FORWARD Packet is being routed through this host to another destination
OUTPUT NF_IP_LOCAL_OUT Packet was generated locally by a process on this host
POSTROUTING NF_IP_POST_ROUTING Packet is about to leave the host (after routing, before NIC)

Packet Flow Through Netfilter Hooks

Incoming packet destined for this host

  1. NIC receives

    frame arrives

  2. PREROUTING

    DNAT, conntrack

  3. Routing Decision

    local or forward?

  4. INPUT

    filter, accept/drop

  5. Local Process

    application

Packet being forwarded (routed through this host)

  1. NIC receives

    frame arrives

  2. PREROUTING

    DNAT, conntrack

  3. Routing Decision

    not local → forward

  4. FORWARD

    filter, accept/drop

  5. POSTROUTING

    SNAT, MASQUERADE

  6. NIC sends

    out the wire

Locally generated outbound packet

  1. Local Process

    application

  2. OUTPUT

    filter, DNAT

  3. Routing Decision

    which interface?

  4. POSTROUTING

    SNAT, MASQUERADE

  5. NIC sends

    out the wire

How iptables/nftables Register at Hooks

iptables organizes rules into tables, each containing chains. Each built-in chain is registered at a specific netfilter hook with a defined priority.

Table Purpose Chains (Hook → Priority)
raw Bypass conntrack PREROUTING (-300), OUTPUT (-300)
mangle Alter packet headers (TTL, TOS, mark) All 5 hooks (-150)
nat Network Address Translation PREROUTING (-100), OUTPUT (-100), POSTROUTING (100)
filter Accept/drop/reject packets INPUT (0), FORWARD (0), OUTPUT (0)
security SELinux/AppArmor labeling INPUT (50), FORWARD (50), OUTPUT (50)

Note

Priority determines order: Lower number = earlier execution. At the PREROUTING hook, raw (-300) runs before mangle (-150) which runs before nat (-100). This is why raw can skip conntrack before NAT ever sees the packet.

Table/Chain Processing Order for an Incoming Forwarded Packet

  1. raw

    PREROUTING

  2. conntrack

    (if not skipped)

  3. mangle

    PREROUTING

  4. nat

    PREROUTING (DNAT)

  5. routing

    decision

  6. filter

    FORWARD

  7. nat

    POSTROUTING (SNAT)

Connection Tracking (conntrack) Internals

conntrack is a netfilter subsystem that tracks the state of every network flow passing through the host. It maintains a hash table keyed by the 5-tuple (source IP, dest IP, source port, dest port, protocol) and is essential for stateful firewalling and NAT.

Conntrack States

NEW
→
ESTABLISHED
→
DESTROYED
RELATED
  
INVALID
State Meaning Example
NEW First packet of a flow seen (SYN for TCP, first UDP packet) TCP SYN to 10.0.0.5:80
ESTABLISHED Reply packet seen — bidirectional traffic confirmed SYN-ACK received back
RELATED New connection associated with an existing ESTABLISHED flow FTP data channel, ICMP "port unreachable" for a UDP flow
INVALID Packet doesn't match any known flow or violates protocol ACK without prior SYN, malformed packet

Inspecting conntrack

console
console

    # List all tracked connections

    $ conntrack -L

    tcp      6 431999 ESTABLISHED src=10.0.0.2 dst=93.184.216.34 sport=54321 dport=443

                                  dst=10.0.0.2 src=93.184.216.34 sport=443 dport=54321 [ASSURED]

    udp      17 29 src=10.0.0.2 dst=8.8.8.8 sport=45678 dport=53

                                  dst=10.0.0.2 src=8.8.8.8 sport=53 dport=45678


    # Show conntrack stats

    $ conntrack -S

    cpu=0   found=1284 invalid=12 insert=0 insert_failed=0 drop=0 early_drop=0

    cpu=1   found=987  invalid=8  insert=0 insert_failed=0 drop=0 early_drop=0


    # Current table size and max

    $ sysctl net.netfilter.nf_conntrack_count

    net.netfilter.nf_conntrack_count = 2847

    $ sysctl net.netfilter.nf_conntrack_max

    net.netfilter.nf_conntrack_max = 262144

Warning

conntrack table exhaustion: When nf_conntrack_count reaches nf_conntrack_max, the kernel drops new connections with the infamous message: nf_conntrack: table full, dropping packet. This is a common issue on high-traffic load balancers, K8s nodes, and NAT gateways. Fix: increase nf_conntrack_max, decrease timeouts (nf_conntrack_tcp_timeout_established), or bypass conntrack for high-volume flows using the raw table's NOTRACK target.

conntrack Hash Table Internals

  • Hash table with nf_conntrack_buckets buckets (default: nf_conntrack_max / 4)
  • Each entry is ~300 bytes — 262,144 entries = ~75 MB memory
  • Keyed by 5-tuple: (src_ip, dst_ip, src_port, dst_port, protocol)
  • Entries expire based on protocol-specific timeouts (TCP ESTABLISHED: 5 days default, UDP: 30s)
  • Entries marked [ASSURED] survive table pressure (they have seen traffic in both directions)

NAT Implementation

NAT in Linux is implemented as a conntrack extension. The first packet of a flow triggers NAT translation, and conntrack records the mapping. All subsequent packets in the same flow are automatically translated using the stored mapping.

SNAT / MASQUERADE
  • Hooks at POSTROUTING
  • Changes the source IP/port of outgoing packets
  • SNAT: static mapping (--to-source 1.2.3.4)
  • MASQUERADE: dynamically uses outgoing interface's IP (for DHCP/dynamic IPs)
  • Reply packets are automatically reverse-translated by conntrack
console
# SNAT with static IP
iptables -t nat -A POSTROUTING \
  -s 10.0.0.0/24 -o eth0 \
  -j SNAT --to-source 203.0.113.1

# MASQUERADE (dynamic IP)
iptables -t nat -A POSTROUTING \
  -s 10.0.0.0/24 -o eth0 \
  -j MASQUERADE
DNAT
  • Hooks at PREROUTING
  • Changes the destination IP/port of incoming packets
  • Used for port forwarding, load balancing, transparent proxying
  • Reply packets are automatically reverse-translated by conntrack
  • Also available at OUTPUT (for locally-generated packets)
console
# DNAT: port forward 8080 → container:80
iptables -t nat -A PREROUTING \
  -p tcp --dport 8080 \
  -j DNAT --to-destination 172.17.0.2:80

# Redirect to local port (transparent proxy)
iptables -t nat -A PREROUTING \
  -p tcp --dport 80 \
  -j REDIRECT --to-port 3128

  1. Client → Host:8080

    original dst

  2. PREROUTING DNAT

    rewrite dst to 172.17.0.2:80

  3. FORWARD

    route to container

  4. Container replies

    conntrack reverses DNAT

  5. Client sees Host:8080

    transparent to client

How Docker Injects iptables Rules

When you run docker run -p 8080:80 nginx, Docker programmatically inserts iptables rules across multiple chains. Understanding these rules is essential for debugging container networking issues.

Docker's iptables Chains

Chain Table Purpose
DOCKER nat DNAT rules for port mappings (-p host:container)
DOCKER filter Allow incoming traffic to mapped ports
DOCKER-ISOLATION-STAGE-1 filter Prevent traffic between Docker networks (first check)
DOCKER-ISOLATION-STAGE-2 filter Drop traffic that crossed network boundaries
DOCKER-USER filter User-defined rules (processed before Docker's FORWARD rules)

Example: Docker NAT Rules

console
console

    $ iptables -t nat -L -n --line-numbers


    Chain PREROUTING (policy ACCEPT)

    1    DOCKER     all  --  0.0.0.0/0    0.0.0.0/0    ADDRTYPE match dst-type LOCAL


    Chain OUTPUT (policy ACCEPT)

    1    DOCKER     all  --  0.0.0.0/0   !127.0.0.0/8  ADDRTYPE match dst-type LOCAL


    Chain POSTROUTING (policy ACCEPT)

    1    MASQUERADE  all  --  172.17.0.0/16  0.0.0.0/0

    2    MASQUERADE  tcp  --  172.17.0.2     172.17.0.2    tcp dpt:80


    Chain DOCKER (2 references)

    1    RETURN     all  --  0.0.0.0/0    0.0.0.0/0

    2    DNAT       tcp  --  0.0.0.0/0    0.0.0.0/0     tcp dpt:8080 to:172.17.0.2:80

What happens when a request arrives at Host:8080

  1. Packet hits PREROUTING → jumps to DOCKER chain (dst-type LOCAL)
  2. DOCKER chain: DNAT rule rewrites destination from host:8080 to 172.17.0.2:80
  3. Routing decision: destination is on docker0 bridge → packet enters FORWARD chain
  4. DOCKER chain in filter table: ACCEPT for traffic to 172.17.0.2:80
  5. Packet forwarded through docker0 bridge to container's veth interface
  6. Container replies → conntrack reverse-translates (src becomes host:8080) → reply sent to client

Tip

DOCKER-USER chain: Docker inserts its rules automatically and re-adds them if you delete them. To add custom firewall rules that survive Docker restarts, use the DOCKER-USER chain: iptables -I DOCKER-USER -s 10.0.0.0/8 -j DROP. This chain is processed before Docker's own FORWARD rules.

How kube-proxy Injects iptables Rules

kube-proxy watches the Kubernetes API for Service and Endpoints changes, then programs iptables rules to implement ClusterIP, NodePort, and LoadBalancer service types. This is the iptables proxy mode (default in many clusters).

kube-proxy's Chain Structure

Chain Purpose
KUBE-SERVICES Entry point: match ClusterIP:port → jump to per-service chain
KUBE-SVC-<hash> Per-service: probability-based load balancing across endpoints
KUBE-SEP-<hash> Per-endpoint (Service EndPoint): DNAT to actual pod IP:port
KUBE-NODEPORTS Match NodePort traffic → jump to per-service chain
KUBE-MARK-MASQ Mark packets that need SNAT (external → service traffic)
KUBE-POSTROUTING Apply MASQUERADE to marked packets

Example: A Service with 3 Endpoints

console
console

    # Service: my-app (ClusterIP 10.96.100.50:80) → 3 pods


    Chain KUBE-SERVICES (2 references)

    KUBE-SVC-XYZABC  tcp  --  0.0.0.0/0  10.96.100.50  tcp dpt:80


    Chain KUBE-SVC-XYZABC (1 references)

    KUBE-SEP-AAA  all  --  0.0.0.0/0  0.0.0.0/0  statistic mode random probability 0.33333

    KUBE-SEP-BBB  all  --  0.0.0.0/0  0.0.0.0/0  statistic mode random probability 0.50000

    KUBE-SEP-CCC  all  --  0.0.0.0/0  0.0.0.0/0  /* last endpoint, no probability needed */


    Chain KUBE-SEP-AAA (1 references)

    KUBE-MARK-MASQ  all  --  10.244.1.5  0.0.0.0/0

    DNAT            tcp  --  0.0.0.0/0   0.0.0.0/0  tcp to:10.244.1.5:8080


    Chain KUBE-SEP-BBB (1 references)

    KUBE-MARK-MASQ  all  --  10.244.2.8  0.0.0.0/0

    DNAT            tcp  --  0.0.0.0/0   0.0.0.0/0  tcp to:10.244.2.8:8080


    Chain KUBE-SEP-CCC (1 references)

    KUBE-MARK-MASQ  all  --  10.244.3.2  0.0.0.0/0

    DNAT            tcp  --  0.0.0.0/0   0.0.0.0/0  tcp to:10.244.3.2:8080

Note

Probability math: With 3 endpoints, the first rule matches with 1/3 probability. If it doesn't match, the second rule matches with 1/2 probability (which is 1/2 of the remaining 2/3 = 1/3). The third catches everything left (also 1/3). This gives uniform distribution.

Packet flow: pod-to-Service

  1. Pod sends to

    10.96.100.50:80

  2. KUBE-SERVICES

    match ClusterIP

  3. KUBE-SVC-XYZ

    random select endpoint

  4. KUBE-SEP-BBB

    DNAT to 10.244.2.8:8080

  5. Pod receives

    at real IP:port

Why iptables Doesn't Scale

iptables processes rules linearly — every packet walks the chain from top to bottom until a match is found. This has significant performance implications at scale.

iptables — O(n) Linear Matching
  • Every packet walks rules sequentially
  • 5,000 services × 3 endpoints = ~15,000 rules in KUBE-SVC chains
  • Each Service adds: 1 KUBE-SERVICES entry + N KUBE-SEP entries + probability rules
  • Rule updates are not incremental — entire table is rewritten atomically
  • At 10K+ services, rule programming takes seconds and causes latency spikes
  • Latency per packet increases linearly with service count
Alternatives — O(1) Lookup
  • IPVS (kube-proxy ipvs mode): hash-table based — O(1) lookup regardless of service count
  • eBPF (Cilium): programmable kernel datapath, hash maps for service lookup, bypasses netfilter entirely
  • nftables: sets and maps for efficient matching, but still not as fast as IPVS/eBPF
  • IPVS supports more load balancing algorithms (round-robin, least connections, weighted)
  • eBPF can short-circuit: skip conntrack, skip full netfilter traversal

Warning

Real-world impact: In clusters with 5,000+ services, iptables-mode kube-proxy can add milliseconds of latency per connection just for rule matching. Service endpoint changes trigger full iptables-restore, which holds a lock and blocks all rule evaluation. This is why production clusters increasingly use IPVS mode or Cilium (eBPF).

Summary: Netfilter Architecture

The Full Picture

  • Netfilter = kernel framework providing 5 hook points in the packet path
  • iptables/nftables = user-facing tools that register rules at these hooks via tables and chains
  • conntrack = stateful connection tracking (hash table of 5-tuples) enabling stateful rules and NAT
  • NAT = conntrack extension that rewrites addresses; DNAT at PREROUTING, SNAT at POSTROUTING
  • Docker = injects DNAT (port mapping) + MASQUERADE (outbound NAT) + isolation rules
  • kube-proxy = injects DNAT chains with probability-based load balancing to implement Services
  • Scaling limit = iptables is O(n); alternatives like IPVS and eBPF provide O(1) lookups
Solidnines — solidnines.com