Skip to content
Book a call
Menu

Networking8 min read

Network Policies

Pod-level firewall rules and service mesh basics

NetworkPolicy: Kubernetes-Native Firewalling

By default, Kubernetes has a flat, open network — every Pod can talk to every other Pod, across all namespaces. There are no firewall rules. NetworkPolicy is the mechanism to restrict this.

How NetworkPolicy Works

  • A NetworkPolicy selects Pods using label selectors (the podSelector field)
  • It defines allowed ingress (incoming) and/or egress (outgoing) traffic
  • Once any NetworkPolicy selects a Pod, that Pod switches from default-allow to default-deny for the specified policyTypes
  • Traffic not explicitly allowed by any policy is dropped
  • Policies are additive — if multiple policies select a Pod, the union of all allowed traffic applies (you cannot deny what another policy allows)

Warning

Common misconception: NetworkPolicy does not have "deny" rules. You cannot write "deny traffic from namespace X." You can only write "allow traffic from Y" — everything else is implicitly denied once a policy selects the Pod. This is a whitelist-only model.

Default Deny Patterns

The first step in any network security posture: apply default deny to a namespace, then explicitly allow required traffic.

Deny All Ingress to a Namespace

console
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: production
spec:
  podSelector: {}           # Empty = selects ALL pods in namespace
  policyTypes:
    - Ingress                # Only affects ingress; egress remains open
  # No 'ingress' field = no traffic allowed in

Deny All Egress from a Namespace

console
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-egress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Egress
  # No 'egress' field = no traffic allowed out

Warning

DNS will break! Denying all egress blocks DNS resolution (UDP/TCP port 53). You almost always need a companion policy that allows egress to kube-dns:

yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

Deny All Ingress + Egress (Full Lockdown)

yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Policy Structure in Detail

A complete NetworkPolicy allowing specific ingress and egress:

console
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-server-policy
  namespace: production
spec:
  # Which pods this policy applies to
  podSelector:
    matchLabels:
      app: api-server

  policyTypes:
    - Ingress
    - Egress

  # INGRESS: who can talk TO these pods
  ingress:
    # Rule 1: Allow from frontend pods in same namespace
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

    # Rule 2: Allow from monitoring namespace
    - from:
        - namespaceSelector:
            matchLabels:
              purpose: monitoring
      ports:
        - protocol: TCP
          port: 9090

    # Rule 3: Allow from specific CIDR (e.g., corporate VPN)
    - from:
        - ipBlock:
            cidr: 10.0.0.0/8
            except:
              - 10.0.1.0/24     # except this subnet
      ports:
        - protocol: TCP
          port: 8080

  # EGRESS: where these pods can talk TO
  egress:
    # Allow to database pods
    - to:
        - podSelector:
            matchLabels:
              app: postgres
      ports:
        - protocol: TCP
          port: 5432

    # Allow DNS
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53

    # Allow HTTPS to external APIs
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
      ports:
        - protocol: TCP
          port: 443

Selector Types in from/to

Selector Matches Example
podSelector Pods by label (same namespace only) app: frontend
namespaceSelector All pods in namespaces matching labels purpose: monitoring
podSelector + namespaceSelector Specific pods in specific namespaces Pods labeled role: scraper in ns labeled purpose: monitoring
ipBlock IP CIDR ranges (with optional except) 10.0.0.0/8 except 10.0.1.0/24

Warning

AND vs OR gotcha: Multiple items in a single from entry are AND'd. Multiple from entries in the ingress array are OR'd. This is a common source of bugs:

console
# OR: allow from frontend pods OR from monitoring namespace
ingress:
  - from:
      - podSelector:
          matchLabels: { app: frontend }
  - from:
      - namespaceSelector:
          matchLabels: { purpose: monitoring }

# AND: allow from pods labeled frontend IN namespaces labeled monitoring
ingress:
  - from:
      - podSelector:
          matchLabels: { app: frontend }
        namespaceSelector:
          matchLabels: { purpose: monitoring }

CNI Enforcement Requirement

NetworkPolicy is a spec, not an implementation. The CNI plugin must actually enforce the rules. If your CNI doesn't support NetworkPolicy, the resources are silently accepted by the API server but have zero effect.

CNI Plugin NetworkPolicy Support Enforcement Mechanism Notes
Calico Full support iptables or eBPF Also supports Calico-specific policies (GlobalNetworkPolicy)
Cilium Full support + L7 eBPF CiliumNetworkPolicy CRD for L7 + FQDN policies
Weave Net Full support iptables Built-in enforcement
Antrea Full support OVS (Open vSwitch) Also supports tier-based policies (Antrea CRDs)
Flannel NO support None Common gotcha! Policies accepted but never enforced
AWS VPC CNI Partial (with add-on) eBPF (with Calico or Network Policy Controller) Requires installing an additional network policy agent

Warning

Flannel trap: Flannel is one of the most commonly deployed CNIs (it's the default in many k3s/RKE setups). It does not enforce NetworkPolicy. If you apply a default-deny policy with Flannel, nothing changes — all traffic still flows. This is a silent security gap. If you need NetworkPolicy, use Calico or Cilium.

Cilium: L7-Aware Network Policies

Standard Kubernetes NetworkPolicy operates at L3/L4 (IPs and ports). Cilium extends this to L7 using its CiliumNetworkPolicy CRD, enabling protocol-aware filtering.

HTTP-Level Filtering

console
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: api-http-policy
spec:
  endpointSelector:
    matchLabels:
      app: api-server
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: frontend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              # Allow GET on public endpoints
              - method: GET
                path: /api/v1/public.*
              # Allow POST on specific endpoint
              - method: POST
                path: /api/v1/orders
              # Everything else on :8080 is DENIED
              # (e.g., DELETE /api/v1/admin is blocked)

Kafka-Level Filtering

console
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: kafka-producer-policy
spec:
  endpointSelector:
    matchLabels:
      app: kafka-broker
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: order-service
      toPorts:
        - ports:
            - port: "9092"
              protocol: TCP
          rules:
            kafka:
              - apiKey: produce
                topic: orders
              # order-service can produce to "orders" topic only
              # Cannot consume, cannot access other topics

FQDN-Based Egress

console
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-external-apis
spec:
  endpointSelector:
    matchLabels:
      app: payment-service
  egress:
    - toFQDNs:
        - matchName: api.stripe.com
        - matchPattern: *.amazonaws.com
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
    # DNS egress required for FQDN resolution
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: ANY
          rules:
            dns:
              - matchName: api.stripe.com
              - matchPattern: *.amazonaws.com

Cilium L7 Capabilities Summary

Protocol What You Can Filter Example
HTTP Method, path, headers Allow GET /api/public but deny DELETE /api/admin
gRPC Service name, method Allow UserService/GetUser but deny AdminService/*
Kafka API key (produce/consume), topic Allow produce to orders but deny consume from payments
DNS Domain name, pattern Allow resolution of *.example.com only

Service Mesh: Brief Introduction

A service mesh injects a sidecar proxy (typically Envoy) into every Pod. All traffic flows through the proxy, giving you mTLS, traffic management, and observability without changing application code.

Architecture

  1. Pod A

    App container

  2. Envoy Sidecar

    Pod A's proxy

  3. mTLS Encrypted

    Mutual TLS on the wire

  4. Envoy Sidecar

    Pod B's proxy

  5. Pod B

    App container

Istio vs Linkerd

ISTIO
  • Proxy: Envoy (full-featured, heavy)
  • mTLS: Yes, auto-injected
  • Traffic management: Full — canary, retries, timeouts, circuit breaking, fault injection, mirroring
  • Observability: Distributed tracing, metrics, service graph (Kiali)
  • Complexity: High — large control plane, steep learning curve
  • Resources: ~100-200MB memory per sidecar
  • Policy: AuthorizationPolicy CRD for L7 access control
LINKERD
  • Proxy: linkerd2-proxy (Rust, ultra-lightweight)
  • mTLS: Yes, auto-injected
  • Traffic management: Basic — retries, timeouts, traffic splits
  • Observability: Golden metrics (latency, throughput, success rate), service graph
  • Complexity: Low — simple to install, minimal config
  • Resources: ~20-30MB memory per sidecar
  • Policy: Server + ServerAuthorization CRDs

What Service Meshes Add

mTLS (Mutual TLS)

  • Automatic certificate provisioning and rotation
  • All service-to-service traffic encrypted
  • Both sides verified (mutual authentication)
  • No application code changes needed
  • Identity-based access control (SPIFFE IDs)

Traffic Management

  • Canary: Route 5% of traffic to v2
  • Retries: Automatic retry on 503, with backoff
  • Timeouts: Per-route timeout enforcement
  • Circuit breaking: Stop sending to failing backends
  • Fault injection: Inject delays/errors for chaos testing

Observability

  • Request metrics: Latency (p50/p99), RPS, error rate
  • Distributed tracing: Trace requests across services
  • Service graph: Visual map of service dependencies
  • All without instrumenting application code

Cilium: Sidecar-Free Approach

  • Uses eBPF in the kernel instead of sidecar proxies
  • Lower latency (no userspace proxy hop)
  • Lower resource overhead (no sidecar per pod)
  • mTLS via WireGuard or IPsec at the node level
  • L7 visibility via eBPF-based protocol parsing

When to Adopt a Service Mesh

Decision Framework

  1. Start with NetworkPolicy + Cilium. This covers 80% of use cases: L3/L4 firewalling, L7 policy, FQDN egress, basic observability. No sidecar overhead.
  2. Add a mesh when you actually need it. Signals: 20+ services, regulatory mTLS requirement, complex traffic management (canary/circuit breaking), need for distributed tracing.
  3. Do NOT start with a mesh. The operational overhead is significant: sidecar injection, certificate management, debugging proxy-layer issues, resource overhead. Premature service mesh adoption is a common anti-pattern.

Tip

Practical progression: NetworkPolicy (K8s native) -> CiliumNetworkPolicy (L7 + FQDN) -> Cilium with mTLS (WireGuard) -> Full service mesh (only if justified). Each step adds complexity; only move forward when the previous layer is insufficient.

Key Takeaways

NetworkPolicy

  • Default: all pods can talk to all pods
  • Whitelist-only model (no deny rules)
  • Always start with default-deny, then allow
  • CNI must support enforcement (Flannel does NOT)
  • AND vs OR semantics in selectors are a common trap

Beyond L3/L4

  • Cilium: L7 policies (HTTP, gRPC, Kafka, DNS)
  • FQDN-based egress control
  • Service mesh: mTLS, traffic management, observability
  • Istio (full-featured) vs Linkerd (lightweight)
  • Don't adopt a mesh prematurely — start with NetworkPolicy
Solidnines — solidnines.com