Servers

A server is not a magic box. It is one program, on one machine, taking turns. Watch the kernel accept a connection, hand it to a worker, and try to keep up. Then crank the load until something snaps.

Scenario

Client browser
Kernel Linux TCP stack
Server process ./myserver
Ready
Blocking server · one at a time 0 / 0 steps
Pick a scenario and click Play flow. Every step is a real kernel syscall or a real user-space action. Click any step to see the exact call and its arguments.

Syscall / event

What just happened

Nerd tip

The accept queue is a real queue with a real depth. listen(fd, backlog) asks the kernel for a queue of completed TCP handshakes waiting to be accept()ed. The kernel clamps your requested backlog to net.core.somaxconn (historically 128, raised to 4096 in Linux 5.4). When the queue fills, new SYN-ACKs are answered but the final ACK is silently dropped and the connection is ignored, or with net.ipv4.tcp_abort_on_overflow=1 you RST instead. Your application log shows nothing. ss -lnt's Recv-Q column is the only tell. Every senior SRE has spent one bad night learning this.

Nerd tip

C10k, and why event loops won. Dan Kegel's 1999 essay framed the problem: how do you serve 10,000 concurrent clients on one machine. A pthread per connection costs ~8 MB of stack each. At 10k that is 80 GB of virtual memory and a scheduler meltdown. epoll_wait (Linux 2.5.44, 2002) and kqueue (FreeBSD 4.1, 2000) returned ready events in O(1) instead of O(N). That one change made nginx, HAProxy, Node, and every modern reverse proxy possible. When someone says "it is async" they mean this.

Nerd tip

Thundering herd and the SO_REUSEPORT fix. In the classic prefork model, N workers all accept() on the same listening socket. A single new connection wakes all N (pre-Linux 4.5), 999 of them find nothing and go back to sleep. Nginx shipped accept_mutex as a userland workaround. SO_REUSEPORT (Linux 3.9, 2013) was the real fix: each worker opens its own listening socket on the same port, and the kernel hashes incoming 4-tuples across them. Load-balanced accepts, no herd. The kernel team added EPOLLEXCLUSIVE in 4.5 (2016) for the epoll-based case. Three kernel releases, one nightmare avoided.

Nerd tip

File descriptors are the ceiling you will hit first. Every socket, every open file, every pipe is an fd. Default Linux soft limit is 1024 per process. ulimit -n shows it. Hit it and every subsequent accept, open, or socket returns EMFILE. The fix is /etc/security/limits.conf or the unit file's LimitNOFILE=. At the kernel level there is also /proc/sys/fs/file-max, the system-wide cap. Run any serious service on stock defaults long enough and one of those three numbers bites you.