Networking7 min read

Ports & Sockets

How the OS multiplexes network connections

Port Numbers

A port is a 16-bit number (0-65535) that identifies a specific process or service on a host. Ports exist at Layer 4 (Transport) — both TCP and UDP use them, and they are independent namespaces (TCP port 53 and UDP port 53 are different).

Port Ranges

Range	Name	Description
`0 - 1023`	Well-Known Ports	Assigned by IANA. Require root/CAP_NET_BIND_SERVICE to bind on Linux.
`1024 - 49151`	Registered Ports	Assigned by IANA for specific services. No special privileges required.
`49152 - 65535`	Ephemeral (Dynamic) Ports	Automatically assigned by the OS for client-side connections.

Note

Linux ephemeral range: Linux actually defaults to 32768-60999 (configurable via net.ipv4.ip_local_port_range), which is wider than the IANA recommendation. This gives ~28,000 ephemeral ports per destination IP.

Common Well-Known Ports

Web & Email

Port	Protocol	Service
`80`	TCP	HTTP
`443`	TCP/UDP	HTTPS / HTTP/3 (QUIC)
`25`	TCP	SMTP (mail relay)
`465`	TCP	SMTPS (SMTP over TLS)
`587`	TCP	SMTP submission
`143`	TCP	IMAP
`993`	TCP	IMAPS

Infrastructure

Port	Protocol	Service
`22`	TCP	SSH
`53`	TCP/UDP	DNS
`67/68`	UDP	DHCP (server/client)
`123`	UDP	NTP
`161/162`	UDP	SNMP / SNMP Trap
`514`	UDP	Syslog
`179`	TCP	BGP

Common Registered Ports (DevOps Essentials)

Port	Protocol	Service
`5432`	TCP	PostgreSQL
`3306`	TCP	MySQL
`6379`	TCP	Redis
`27017`	TCP	MongoDB
`9092`	TCP	Kafka
`2379/2380`	TCP	etcd (client/peer)
`6443`	TCP	Kubernetes API server
`10250`	TCP	Kubelet API
`8080`	TCP	HTTP alternate (common for app servers)
`8443`	TCP	HTTPS alternate
`9090`	TCP	Prometheus
`3000`	TCP	Grafana
`51820`	UDP	WireGuard

What a Socket Really Is

A "socket" in networking has two meanings that are often conflated. Understanding the distinction is important.

The Network Definition (5-Tuple)

A unique connection is identified by a 5-tuple:

Protocol (TCP or UDP)
Source IP
Source Port
Destination IP
Destination Port

Any difference in any field = different connection.

The OS Definition (File Descriptor)

In Unix/Linux, a socket is a file descriptor — an integer that represents an open I/O channel.

Created by the socket() syscall
Readable/writable with read() / write()
Lives in /proc/PID/fd/
Inherits the "everything is a file" Unix philosophy

How Multiplexing Works

A common misconception: "a server on port 443 can only handle one connection." In reality, a server can handle millions of concurrent connections on a single port because each connection has a unique 5-tuple.

Example: Nginx on Port 443

Protocol	Source IP	Source Port	Dest IP	Dest Port
TCP	203.0.113.1	52341	10.0.0.1	443
TCP	203.0.113.1	52342	10.0.0.1	443
TCP	198.51.100.5	44801	10.0.0.1	443
TCP	198.51.100.5	44802	10.0.0.1	443

All four connections share destination port 443. Each is unique because the source IP:port differs.

Note

Theoretical limit: Per destination IP:port, the number of concurrent connections from a single source IP is limited by the ephemeral port range (~28,000 on Linux). Across different source IPs, there's no practical limit from the port namespace — the real limits are file descriptors, memory, and CPU.

Socket API Lifecycle

The Berkeley sockets API (BSD sockets) is the universal interface for network programming. Every language wraps it, but the underlying syscalls are the same.

TCP Server Lifecycle

socket()
Create a socket fd
(AF_INET, SOCK_STREAM)
bind()
Assign local IP:port
(0.0.0.0:443)
listen()
Mark socket as passive
(set backlog size)
accept()
Block until client connects
(returns NEW fd)
read/write
Exchange data on
the connection fd
close()
Teardown connection
(FIN exchange)

TCP Client Lifecycle

socket()
Create a socket fd
connect()
Initiate 3-way handshake
(OS auto-binds ephemeral port)
read/write
Exchange data
close()
Teardown connection

Key Insight: Two Types of Server Sockets

Listening socket: Created by socket() + bind() + listen(). It never carries data — it only accepts new connections. There is exactly ONE per port.
Connected socket: Returned by accept(). This is a new file descriptor representing one specific connection (one 5-tuple). Data flows on these. There are MANY per port.

SO_REUSEADDR

Without SO_REUSEADDR, if a server crashes and restarts, bind() fails because the old socket is still in TIME_WAIT. You'll see the dreaded Address already in use error.

Without SO_REUSEADDR

bash

// Server crashes, restarts...
bind(fd, addr, len);
// ERROR: EADDRINUSE
// Old socket still in TIME_WAIT
// Must wait 60s (2xMSL) to restart!

With SO_REUSEADDR

int opt = 1;
setsockopt(fd, SOL_SOCKET,
           SO_REUSEADDR,
           &opt, sizeof(opt));
bind(fd, addr, len);
// SUCCESS - bind to TIME_WAIT addr

Tip

Every server should set SO_REUSEADDR. This is not optional for production services. Without it, server restarts during deployments will fail for up to 60 seconds. Every web server, database, and application framework sets this by default.

SO_REUSEPORT

SO_REUSEPORT (Linux 3.9+) takes things further: it allows multiple processes/threads to bind to the same IP:port simultaneously. The kernel distributes incoming connections across all listeners.

How SO_REUSEPORT Works

Multiple processes call bind() + listen() on the same port
The kernel load-balances incoming connections across all listening sockets (consistent hashing by source IP:port since Linux 4.6)
Each process has its own accept queue — no thundering herd problem
If a process dies, only its connections are affected; others continue serving

Traditional: Single Listener + Worker Threads

bash

// One process binds + listens
// accept() returns fd
// Hand fd to worker thread/process
//
// Problem: accept() is a bottleneck
// Thundering herd with fork()

Modern: SO_REUSEPORT

bash

// N processes all bind + listen
// on the same port
// Kernel distributes connections
//
// No bottleneck, no thundering herd
// Used by Nginx, Envoy, HAProxy

Note

In Kubernetes: When you run multiple replicas of a pod and they bind to the same port, each gets its own network namespace — so SO_REUSEPORT isn't needed for inter-pod distribution (that's kube-proxy/iptables/eBPF). But within a single pod, multi-process servers (like Nginx with worker_processes auto) use SO_REUSEPORT to distribute across workers.

Listen Backlog & Connection Queues

When a server calls listen(fd, backlog), the backlog parameter controls the maximum number of pending connections. But the kernel actually uses a two-queue model.

The Two-Queue Model

SYN Queue
Half-open connections
(SYN received, SYN-ACK sent,
waiting for final ACK)
Accept Queue
Fully established connections
waiting for application
to call accept()
Application
Connection handed
to the application
as a new fd

Queue Parameters

Queue	Controlled By	Overflow Behavior
SYN Queue	`tcp_max_syn_backlog` (default: 128-1024)	Drop the SYN (client retries) or use SYN cookies if enabled
Accept Queue	`min(backlog, somaxconn)` where `backlog` is the `listen()` argument and `somaxconn` is the sysctl	Drop the final ACK (connection silently fails) or send RST

console


    # Check and tune queue limits

    $ sysctl net.core.somaxconn

    net.core.somaxconn = 4096

    $ sysctl net.ipv4.tcp_max_syn_backlog

    net.ipv4.tcp_max_syn_backlog = 4096


    # Check current queue state for a listening socket

    $ ss -tlnp

    State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port  Process

    LISTEN  0       4096    0.0.0.0:443         0.0.0.0:*          users:(("nginx",pid=1234,fd=6))

    LISTEN  0       4096    0.0.0.0:80          0.0.0.0:*          users:(("nginx",pid=1234,fd=7))

    LISTEN  0       128     127.0.0.1:5432      0.0.0.0:*          users:(("postgres",pid=5678,fd=3))

Note

Reading ss -tln output for LISTEN sockets:
Recv-Q = current number of connections waiting in the accept queue
Send-Q = maximum accept queue size (the backlog)
If Recv-Q approaches Send-Q, the application is not calling accept() fast enough.

Warning

Common misconfiguration: Many applications pass listen(fd, 128) as the backlog. On a busy server, 128 is far too low. But even if the application passes 65535, the kernel caps it at somaxconn. Both must be tuned. Modern defaults (kernel 5.4+) set somaxconn=4096, but older kernels default to 128.

Inspecting Sockets in Practice

console


    # List all listening TCP sockets with process info

    $ ss -tlnp

    State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port  Process

    LISTEN  0       4096    0.0.0.0:443         0.0.0.0:*          users:(("envoy",pid=1001,fd=25))

    LISTEN  0       4096    0.0.0.0:80          0.0.0.0:*          users:(("envoy",pid=1001,fd=24))

    LISTEN  0       4096    127.0.0.1:15000     0.0.0.0:*          users:(("envoy",pid=1001,fd=18))

    LISTEN  0       128     0.0.0.0:22          0.0.0.0:*          users:(("sshd",pid=842,fd=3))

    LISTEN  0       511     127.0.0.1:8080      0.0.0.0:*          users:(("myapp",pid=2345,fd=5))


    # List all UDP listening sockets

    $ ss -ulnp

    State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port  Process

    UNCONN  0       0       0.0.0.0:51820       0.0.0.0:*          users:(("wireguard",pid=901))

    UNCONN  0       0       127.0.0.53:53       0.0.0.0:*          users:(("systemd-resolve",pid=610,fd=13))


    # Find which process is using a specific port

    $ ss -tlnp 'sport = :8080'

    State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port  Process

    LISTEN  0       511     127.0.0.1:8080      0.0.0.0:*          users:(("myapp",pid=2345,fd=5))


    # Count connections per state to a destination

    $ ss -tan dst 10.0.1.5 | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rn

       4201 ESTAB

        312 TIME-WAIT

          3 CLOSE-WAIT

Ephemeral Port Exhaustion

If a client opens many connections to the same destination IP:port, it can run out of ephemeral ports. Each connection needs a unique source port from the ephemeral range.

Symptom: connect() fails with EADDRNOTAVAIL ("Cannot assign requested address")
Check: ss -tan dst TARGET:PORT | wc -l — if it's close to 28,000, you're exhausted
Mitigations:
- Widen the range: sysctl -w net.ipv4.ip_local_port_range="1024 65535" (gives ~64,000 ports)
- Connection pooling (reuse connections instead of creating new ones)
- Use multiple destination IPs (load balancer VIPs)
- Enable tcp_tw_reuse to reclaim TIME_WAIT ports

console


    # Check ephemeral port range

    $ sysctl net.ipv4.ip_local_port_range

    net.ipv4.ip_local_port_range = 32768	60999


    # Check file descriptor limits (also constrains max connections)

    $ ulimit -n

    1048576

    $ sysctl fs.file-max

    fs.file-max = 9223372036854775807

Tip

K8s troubleshooting: In Kubernetes, ephemeral port exhaustion often manifests when a sidecar proxy (Envoy/Istio) makes many outbound connections to the same service. The symptoms are intermittent connection failures under load. Check with ss -s inside the pod and look at the TIME_WAIT count.

Port Numbers

Port Ranges

Common Well-Known Ports

Common Registered Ports (DevOps Essentials)

What a Socket Really Is

How Multiplexing Works

Socket API Lifecycle

TCP Server Lifecycle

socket()

bind()

listen()

accept()

read/write

close()

TCP Client Lifecycle

socket()

connect()

read/write

close()

SO_REUSEADDR

SO_REUSEPORT

Listen Backlog & Connection Queues

The Two-Queue Model

SYN Queue

Accept Queue

Application

Inspecting Sockets in Practice