Web Servers

What they do, how they handle connections, and how they fit in the stack

What a Web Server Actually Does

A web server is a process that listens on a TCP socket (typically port 80/443), accepts incoming connections, parses HTTP requests, and sends back HTTP responses. That's the core loop — everything else is configuration around it.

1. Bind & Listen

   socket() → bind(:80) → listen()

2. Accept Connection

   TCP handshake completes, fd returned

3. TLS Handshake

   If HTTPS — negotiate cipher, exchange certs

4. Read Request

   Parse method, path, headers, body

5. Route & Handle

   Static file? Proxy to backend? Redirect?

6. Send Response

   Status line, headers, body bytes

Connections — What They Are and How They're Managed

An HTTP connection is a TCP connection (or QUIC session for HTTP/3) between the client and the server. Understanding connections is fundamental because they determine performance, resource usage, and concurrency.

Anatomy of a Connection

Every connection is identified by the 5-tuple: (protocol, client IP, client port, server IP, server port). Each connection is a file descriptor in the server process — it consumes memory (kernel buffers, TLS state, application buffers) and a slot in the OS's fd table.

Keep-Alive and Connection Reuse

In HTTP/1.1, connections are persistent by default (Connection: keep-alive). After the server sends a response, the connection stays open for the next request. The server closes it after a configurable idle timeout.

Connection Limits and Concurrency

A web server can handle thousands to millions of concurrent connections, depending on its architecture. Key limits:

Connection Handling Models

The fundamental question for any server: how do you handle many connections at once? The answer has evolved over decades.

1. Process-per-Connection (Fork)

2. Thread-per-Connection

3. Event-Driven / Non-Blocking I/O

4. Hybrid: Event Loop + Worker Threads

// Pseudocode
while (true) {
  conn = accept(listen_fd)
  spawn_thread(() => {
    request = read(conn)    // blocks this thread
    response = handle(request)
    write(conn, response)  // blocks this thread
    close(conn)
  })
}

// Pseudocode
epoll_fd = epoll_create()
epoll_add(epoll_fd, listen_fd)

while (true) {
  events = epoll_wait(epoll_fd) // blocks until ANY fd ready
  for (ev in events) {
    if (ev.fd == listen_fd)
      accept_and_register(ev.fd)
    else
      handle_io(ev.fd) // non-blocking read/write
  }
}

Worker Architecture in Practice

Modern web servers don't run a single event loop — they run multiple worker processes, each with its own event loop, to use all CPU cores. The master process manages workers, handles config reloads, and binds to privileged ports.

1. Master Process

   Binds ports, manages workers, reads config, handles signals

2. Worker 1

   Event loop + epoll. Handles connections independently.

3. Worker 2

   Event loop + epoll. Handles connections independently.

4. Worker N

   Event loop + epoll. One per CPU core (typically).

All workers share the same listening socket(s). The kernel distributes incoming connections across workers via SO_REUSEPORT (since Linux 3.9) or the accept mutex.

nginx Example

# /etc/nginx/nginx.conf
worker_processes auto;          # one per CPU core
worker_connections 4096;       # max connections per worker
                                    # total capacity = workers × worker_connections

events {
    use epoll;                   # Linux event multiplexer
    multi_accept on;             # accept multiple connections at once
}

http {
    keepalive_timeout 65;        # close idle connections after 65s
    keepalive_requests 1000;    # max requests per keep-alive connection

    server {
        listen 443 ssl http2;
        server_name example.com;

        ssl_certificate     /etc/ssl/certs/example.pem;
        ssl_certificate_key /etc/ssl/private/example.key;

        # Serve static files directly
        location /static/ {
            root /var/www;
            expires 30d;
            add_header Cache-Control "public, immutable";
        }

        # Reverse proxy to application server
        location /api/ {
            proxy_pass http://backend:8000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

Reverse Proxy — The Most Important Role

In modern deployments, the web server's primary role isn't serving files — it's acting as a reverse proxy in front of application servers. The web server faces the internet; the app server stays internal.

Why Not Expose the App Server Directly?

Upstream Connection Pooling

The web server maintains a pool of persistent connections to each backend (upstream). Instead of opening a new TCP connection for every proxied request, it reuses existing ones — eliminating the TCP + TLS handshake overhead on the backend path.

# nginx upstream with connection pooling
upstream backend {
    server 10.0.1.10:8000;
    server 10.0.1.11:8000;
    server 10.0.1.12:8000;
    keepalive 32;              # keep 32 idle connections per worker
}

server {
    location /api/ {
        proxy_pass http://backend;
        proxy_http_version 1.1;                # required for keepalive to upstreams
        proxy_set_header Connection "";        # clear Connection: close
    }
}

Load Balancing

When proxying to multiple backends, the web server distributes requests across them. This is L7 load balancing — the server understands HTTP and can make routing decisions based on URL path, headers, cookies, etc.

Load Balancing Algorithms

Virtual Hosting — Multiple Sites on One Server

A single web server on one IP can serve different websites based on the Host header (HTTP/1.1) or SNI (TLS). This is how shared hosting works, and how a reverse proxy routes to different backends per domain.

How It Works

1. Client connects to the server's IP on port 443
2. In the TLS ClientHello, the client includes SNI (Server Name Indication): example.com
3. The server selects the correct certificate and completes the TLS handshake
4. The HTTP request includes Host: example.com — the server matches this to a server block
5. The matched server block determines how to handle the request (serve files, proxy, redirect)

# nginx: two sites on the same IP
server {
    listen 443 ssl;
    server_name app.example.com;
    ssl_certificate /etc/ssl/app.pem;
    location / { proxy_pass http://app-backend:3000; }
}

server {
    listen 443 ssl;
    server_name api.example.com;
    ssl_certificate /etc/ssl/api.pem;
    location / { proxy_pass http://api-backend:8000; }
}

Request Routing

Once the server block is matched (by Host/server_name), the server evaluates the request URI against location blocks to decide how to handle it. This is the core routing logic.

Common Routing Patterns

Traefik Example — Dynamic, Label-Based Routing

Traefik takes a different approach from nginx: instead of static config files, it discovers routes dynamically from infrastructure (Docker labels, K8s Ingress, Consul). The architecture is based on entrypoints, routers, middlewares, and services.

# traefik.yml — static configuration
entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
  websecure:
    address: ":443"

providers:
  docker:
    exposedByDefault: false        # only route explicitly labeled containers

certificatesResolvers:
  letsencrypt:
    acme:
      email: admin@example.com
      storage: /data/acme.json
      httpChallenge:
        entryPoint: web

# docker-compose.yml — routes discovered from labels
services:
  app:
    image: myapp:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.app.rule=Host(`app.example.com`)"
      - "traefik.http.routers.app.tls.certresolver=letsencrypt"
      - "traefik.http.services.app.loadbalancer.server.port=3000"

  api:
    image: myapi:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`api.example.com`)"
      - "traefik.http.routers.api.tls.certresolver=letsencrypt"
      - "traefik.http.routers.api.middlewares=ratelimit"
      - "traefik.http.middlewares.ratelimit.ratelimit.average=100"
      - "traefik.http.services.api.loadbalancer.server.port=8000"

HTTP Version Handling

A web server must negotiate which HTTP version to use with the client, and separately which version to use with backends. These are often different.

Client-Facing (Frontend)

Backend-Facing (Upstream)

The web server typically speaks HTTP/1.1 to backends, even if it speaks HTTP/2 or HTTP/3 to clients. Reasons:

• App servers (gunicorn, uvicorn, Node) already handle HTTP/1.1 well
• HTTP/2's multiplexing benefit is on the internet (high latency) — on a local network with sub-millisecond RTT, there's little advantage
• Connection pooling with HTTP/1.1 keepalive is sufficient for backend traffic
• Simpler to debug — you can curl or tcpdump the backend directly

# nginx: HTTP/2 to clients, HTTP/1.1 to backends
server {
    listen 443 ssl http2;     # client-facing: HTTP/2
    listen 443 quic;           # client-facing: HTTP/3
    add_header Alt-Svc 'h3=":443"; ma=86400';  # advertise HTTP/3

    location / {
        proxy_pass http://backend:8000;     # backend: plain HTTP/1.1
        proxy_http_version 1.1;              # keepalive requires 1.1
    }
}

Response Caching

The web server can cache backend responses and serve them directly for subsequent requests, avoiding a round-trip to the app server entirely.

1. Request arrives

   GET /api/products

2. Cache lookup

   Key = method + URI + Vary headers

3. HIT

   Return cached response immediately

4. MISS

   Proxy to backend, cache response, return

Headers the Web Server Adds

When reverse proxying, the web server should pass along client information that the backend needs. These headers are not part of the original client request — the proxy adds them.

Access Logging

Every request processed by the web server can be logged. These access logs are the primary source of truth for traffic analysis, debugging, and security monitoring.

Common Log Fields

# nginx combined log format
log_format combined
    '$remote_addr - $remote_user [$time_local] '
    '"$request" $status $body_bytes_sent '
    '"$http_referer" "$http_user_agent"';

# Example output:
203.0.113.50 - - [31/Mar/2026:14:23:01 +0000] "GET /api/users HTTP/2.0" 200 1234 "-" "curl/8.5.0"

For observability, add timing fields:

# nginx with upstream timing
log_format timed
    '$remote_addr [$time_local] "$request" $status '
    'rt=$request_time uct=$upstream_connect_time '
    'uht=$upstream_header_time urt=$upstream_response_time';

# Example:
203.0.113.50 [31/Mar/2026:14:23:01] "GET /api/users HTTP/2.0" 200 rt=0.045 uct=0.001 uht=0.040 urt=0.040

Security Headers

The web server is the right place to enforce security headers — they apply globally and don't depend on the backend implementing them.

# nginx: add security headers globally
server {
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-Frame-Options "DENY" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
}

Where the Web Server Fits

Zooming out — the web server sits at a specific point in the request lifecycle, and different deployment models place it differently.

Traditional: Dedicated Reverse Proxy

Kubernetes: Ingress Controller

CDN Edge: Cloud-Managed

Sidecar Proxy: Service Mesh