Skip to content
Menu

Networking9 min read

VPNs

Encrypted tunnels — IPsec, OpenVPN, and WireGuard

VPN Architectures

A VPN creates an encrypted tunnel between endpoints, making it appear as if they share a private network even when communicating over the public internet. The architecture determines how many tunnels exist and who connects to whom.

Site-to-Site

Connects two entire networks over an encrypted tunnel. All traffic between the networks is automatically routed through the VPN. Common for connecting offices, data centers, or cloud VPCs.

💻
Office Hosts
10.1.0.0/16
🔒
VPN Gateway
Office A
🔒
VPN Gateway
Office B
💻
DC Hosts
10.2.0.0/16

Note

Cloud equivalent: AWS VPN Gateway, GCP Cloud VPN, Azure VPN Gateway. These create IPsec tunnels between your VPC and on-prem network. Also: AWS Transit Gateway for hub-and-spoke multi-VPC connectivity.

Remote Access

A single device (laptop, phone) connects to a network. The VPN client creates a tunnel to the VPN server, and the device gets an IP on the remote network.

💻
Remote Laptop
Coffee shop WiFi
🔒
VPN Server
Corporate network
🖥
Internal Services
10.0.0.0/8

Note

The traditional corporate VPN model: OpenVPN, Cisco AnyConnect, GlobalProtect. The problem: all traffic routes through a central VPN server, creating a bottleneck. This is the "hub-and-spoke" anti-pattern that mesh VPNs solve.

Full Mesh

Every node connects directly to every other node. No central server bottleneck — traffic takes the shortest path. This is the Tailscale/Nebula model.

💻
Laptop
Home
🖥
Server
AWS
💻
Workstation
Office

All nodes have direct encrypted tunnels to each other — no central hub

Tip

Tailscale's model: A coordination server (control plane) distributes public keys and endpoint info. Data plane is fully mesh — WireGuard tunnels directly between nodes. The coordination server never sees your traffic. If direct connections fail (NAT), DERP relay servers are used as a fallback.

IPsec

IPsec is the original VPN standard, operating at Layer 3 (network layer). It's built into every major OS kernel and is the default for site-to-site VPNs in enterprise and cloud environments. It's powerful but notoriously complex to configure and debug.

IKE (Internet Key Exchange)

Before IPsec can encrypt traffic, peers must authenticate and agree on encryption parameters. This negotiation happens in two phases via IKE (usually IKEv2).

Peer A (Initiator)
Peer B (Responder)
IKE_SA_INIT (DH exchange, proposals)
IKE_SA_INIT response (DH, chosen proposal)
--- Phase 1 Complete: IKE SA established (encrypted channel) ---
IKE_AUTH (identity, auth, traffic selectors)
IKE_AUTH response (identity, auth, SA)
--- Phase 2 Complete: IPsec SA established (data can flow) ---

IKE Phase Summary

PhasePurposeResult
Phase 1 (IKE SA) Authenticate peers, Diffie-Hellman key exchange, negotiate encryption algorithms Secure channel for Phase 2 negotiation
Phase 2 (IPsec SA) Negotiate IPsec parameters, define which traffic to protect (traffic selectors) IPsec Security Associations — ready to encrypt data

ESP (Encapsulating Security Payload)

ESP is the protocol that actually encrypts and authenticates the data. It's IP protocol 50 (not TCP or UDP — it's its own protocol at the IP layer).

Tunnel Mode vs Transport Mode

Tunnel Mode

Entire original IP packet is encrypted and wrapped in a new IP header.

New IP Header (gateway IPs)
ESP Header
Original IP Header + Payload (encrypted)
  • Original source/destination IPs are hidden
  • Used for site-to-site VPNs (gateway-to-gateway)
  • New IP header uses the VPN gateway addresses
  • This is the default and most common mode
Transport Mode

Only the payload is encrypted. Original IP header is preserved.

Original IP Header (preserved!)
ESP Header
Payload only (encrypted)
  • Original source/destination IPs are visible
  • Used for host-to-host encryption
  • Less overhead (no extra IP header)
  • Rare in practice — mostly for L2TP/IPsec

Warning

IPsec complexity is real. Configuring IPsec involves: IKE version, encryption algorithm, hash algorithm, DH group, authentication method (PSK vs certificates), SA lifetime, PFS (Perfect Forward Secrecy), DPD (Dead Peer Detection), traffic selectors, NAT-T (NAT Traversal on UDP 4500)... and both sides must match exactly. A single mismatch = silent failure. Debugging means reading IKE logs line by line. This is why WireGuard was created.

OpenVPN

OpenVPN is a TLS-based VPN that runs in userspace (not kernel). It's been the de facto standard for remote access VPNs for over a decade — mature, well-understood, and widely deployed.

OpenVPN Architecture

AspectDetails
Transport UDP (preferred) or TCP (fallback for restrictive firewalls)
Encryption TLS for control channel, AES-256-GCM or ChaCha20-Poly1305 for data channel
Authentication Certificates (PKI), pre-shared keys, or username/password + certificates
Interface type tun (Layer 3 — IP packets) or tap (Layer 2 — Ethernet frames)
Runs in Userspace (not kernel) — processes packets via /dev/net/tun
Codebase ~100,000+ lines of C — large attack surface for auditing
Default port UDP 1194
Strengths
  • Battle-tested over 20+ years
  • Extremely flexible configuration
  • Works over TCP (can traverse HTTP proxies)
  • Rich PKI support for enterprise cert management
  • Runs on everything (Linux, macOS, Windows, Android, iOS, routers)
Weaknesses
  • Userspace processing = lower throughput than kernel VPNs
  • Large codebase = harder to audit, larger attack surface
  • Complex configuration (server.conf can be 50+ lines)
  • TCP mode suffers from "TCP meltdown" (TCP over TCP)
  • No built-in roaming support — connection drops on network change

Note

tun vs tap: tun creates a virtual network interface that handles IP packets (Layer 3). tap creates one that handles Ethernet frames (Layer 2) — needed if you want the VPN peers to be on the same broadcast domain (bridging). Almost all modern deployments use tun.

WireGuard

WireGuard is a modern VPN protocol designed to be simpler, faster, and more secure than IPsec and OpenVPN. It's built into the Linux kernel since 5.6 (2020) and is the foundation for Tailscale, Mullvad, and many other modern VPN products.

Design Philosophy

Minimal

~4,000 lines of kernel code. IPsec is ~400,000 lines. OpenVPN is ~100,000 lines. Less code = fewer bugs = auditable by a single person.

Opinionated

No cipher negotiation. One fixed set of modern primitives. If a vulnerability is found, the entire protocol version is replaced — not patched with options.

Silent

Does not respond to unauthenticated packets. From the network's perspective, a WireGuard interface is invisible — no handshake until you send a valid packet.

Cryptographic Primitives

FunctionAlgorithmNotes
Key exchange Curve25519 (ECDH) Fast, safe elliptic curve. Used in Noise protocol handshake.
Symmetric encryption ChaCha20-Poly1305 AEAD cipher. Fast in software (no AES-NI needed). Same as used in TLS 1.3.
Hashing BLAKE2s Faster than SHA-256, cryptographically strong.
Key derivation HKDF Derives session keys from the Noise handshake output.
Handshake framework Noise_IK Formally verified key exchange pattern. 1-RTT handshake.

Tip

No cipher negotiation: Unlike IPsec/TLS where peers negotiate which algorithms to use (creating downgrade attack opportunities), WireGuard uses exactly these algorithms. If they're ever broken, WireGuard will ship a new protocol version with new algorithms — simple version bump, no backward compatibility mess.

Cryptokey Routing

This is WireGuard's core innovation. The public key → allowed IPs mapping IS the routing table. There's no separate routing configuration — the tunnel configuration IS the routing.

How Cryptokey Routing Works

  1. Sending: Kernel looks up destination IP in the AllowedIPs of each peer. Finds matching peer's public key. Encrypts packet with that peer's session key. Sends to that peer's endpoint.
  2. Receiving: Decrypts packet. Checks that the decrypted source IP is within that peer's AllowedIPs. If not, the packet is dropped — this is the firewall.

Peer's Public Key aB3d...xY9z= → AllowedIPs 10.0.0.2/32, 192.168.1.0/24

This mapping says: packets destined for 10.0.0.2 or 192.168.1.0/24 should be encrypted with this peer's key. Packets received from this peer must have source IPs in these ranges.

Stateless and Roaming

Traditional VPN (OpenVPN/IPsec)
  • Connection state: sessions, handshakes, keepalives
  • Network change (WiFi → cellular) = reconnect
  • Timeout/renegotiation required
  • Client must know server IP upfront
WireGuard
  • No connection state — just send encrypted UDP packets
  • Network change = next packet comes from new IP, endpoint updated
  • Roaming is automatic — peer's endpoint updates on every authenticated packet
  • Works like "encrypted IP" — stateless at the tunnel level

Note

How roaming works: WireGuard stores the last known endpoint (IP:port) for each peer. When it receives an authenticated packet from a peer at a new IP:port, it simply updates the endpoint. No renegotiation, no reconnection. Your phone switches from WiFi to cellular — the next packet arrives from the new IP, WireGuard updates, and traffic continues seamlessly.

WireGuard Configuration Example

Server (10.0.0.1)
console
# /etc/wireguard/wg0.conf

[Interface]
Address = 10.0.0.1/24
ListenPort = 51820
PrivateKey = SERVER_PRIVATE_KEY

# Enable IP forwarding and NAT
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; \
         iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; \
           iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

[Peer]
# Laptop
PublicKey = LAPTOP_PUBLIC_KEY
AllowedIPs = 10.0.0.2/32

[Peer]
# Phone
PublicKey = PHONE_PUBLIC_KEY
AllowedIPs = 10.0.0.3/32
Client / Laptop (10.0.0.2)
console
# /etc/wireguard/wg0.conf

[Interface]
Address = 10.0.0.2/24
PrivateKey = LAPTOP_PRIVATE_KEY
DNS = 1.1.1.1

[Peer]
# Server
PublicKey = SERVER_PUBLIC_KEY
Endpoint = 203.0.113.5:51820
AllowedIPs = 0.0.0.0/0
# ↑ Route ALL traffic through VPN
# Use 10.0.0.0/24 for split tunnel

PersistentKeepalive = 25
# ↑ Send keepalive every 25s
# Needed when client is behind NAT
# Keeps the NAT mapping alive
console
console

    # Generate key pair

    $ wg genkey | tee privatekey | wg pubkey > publickey


    # Bring up the interface

    $ wg-quick up wg0


    # Check status

    $ wg show

    interface: wg0

      public key: aB3d...xY9z=

      private key: (hidden)

      listening port: 51820


    peer: Kf5x...bN2w=

      endpoint: 198.51.100.42:38291

      allowed ips: 10.0.0.2/32

      latest handshake: 42 seconds ago

      transfer: 1.48 MiB received, 3.21 MiB sent
  

IPsec vs OpenVPN vs WireGuard

Feature IPsec OpenVPN WireGuard
Layer L3 (kernel) L3/L2 (userspace) L3 (kernel)
Codebase ~400K lines ~100K lines ~4K lines
Handshake IKEv1/v2 (complex multi-phase) TLS handshake Noise_IK (1-RTT)
Transport IP protocol 50 (ESP) + UDP 500/4500 UDP 1194 or TCP UDP only (custom port)
Cipher negotiation Yes (many options) Yes (TLS cipher suites) No (fixed primitives)
Performance High (kernel, but complex path) Moderate (userspace) Highest (kernel, minimal overhead)
Roaming No (requires renegotiation) No (reconnection needed) Yes (automatic endpoint update)
NAT traversal NAT-T (UDP encap on port 4500) Native (runs over UDP/TCP) Native (runs over UDP)
Stealth IKE packets are identifiable Can be disguised as HTTPS (TCP mode) Silent to unauthenticated traffic
Configuration Complex (many knobs, both sides must match) Complex (.conf files, PKI setup) Simple (~10 lines per peer)
Auditability Difficult (massive codebase) Difficult (large codebase) Realistic (single person can audit)
Best for Site-to-site, cloud VPN gateways Remote access, legacy compatibility Everything modern (remote access, mesh, site-to-site)
Used by AWS VPN, Cisco, strongSwan Many corporate VPNs Tailscale, Mullvad, Cloudflare WARP

Warning

When WireGuard is NOT the answer: If you need TCP transport (to traverse aggressive firewalls/proxies that block non-HTTPS UDP), WireGuard can't do it natively. OpenVPN over TCP or tunneling WireGuard inside a WebSocket/HTTPS wrapper is needed. Also, WireGuard doesn't natively support dynamic peer authentication (like RADIUS/LDAP) — that's what Tailscale's coordination layer adds.

Tip

The modern stack: WireGuard (encryption + tunneling) + Tailscale/Headscale (coordination, auth, NAT traversal, ACLs) = what corporate VPNs should be. No more "connect to VPN, then everything routes through HQ." Instead: mesh connectivity, zero-trust ACLs, SSO integration, and direct peer-to-peer connections.

Solidnines — solidnines.com