Python Select and Polling — Deep Dive

The C10K Problem and Why It Matters

In the early 2000s, handling 10,000 concurrent connections (C10K) was a major engineering challenge. The bottleneck wasn’t bandwidth or CPU — it was the system call overhead of select() and per-connection threading models.

select() copies the entire file descriptor set from userspace to kernel space on every call. With 10,000 FDs, that’s ~1.2KB copied twice per iteration. The kernel then linearly scans every FD to check readiness. At high connection counts, the process spent more time in select() than actually handling data.

epoll (Linux 2.6, 2003) and kqueue (FreeBSD 4.1, 2000) solved this by maintaining kernel-side state:

• Registration is separate from waiting. epoll_ctl() registers interest once. epoll_wait() only returns ready FDs.
• O(ready) not O(total). The kernel maintains a ready list — FDs are added to it when events occur (via interrupt handlers), so epoll_wait() just drains the list.

epoll Deep Dive

The Three System Calls

import select

# 1. Create an epoll instance (kernel allocates an internal red-black tree + ready list)
ep = select.epoll()

# 2. Register interest in file descriptors
ep.register(fd, select.EPOLLIN | select.EPOLLOUT)
# Kernel adds fd to the red-black tree with the specified event mask

# 3. Wait for events
events = ep.poll(timeout=1.0)
# Returns: [(fd, event_mask), ...]
# Kernel returns only FDs with pending events

Kernel Data Structures

epoll instance:
├── Red-black tree: all registered FDs (O(log n) insert/delete)
├── Ready list: linked list of FDs with pending events (O(1) append)
└── Wait queue: threads blocked in epoll_wait()

When data arrives on a socket:
1. NIC interrupt → kernel network stack processes packet
2. Socket buffer receives data → socket becomes readable
3. Kernel walks epoll's interest tree for this FD
4. If EPOLLIN is registered → add entry to ready list
5. If any thread is in epoll_wait() → wake it up

Level-Triggered vs Edge-Triggered

# Level-triggered (default): reports FD as ready whenever data is available
ep.register(fd, select.EPOLLIN)

# Edge-triggered: reports FD only when NEW data arrives
ep.register(fd, select.EPOLLIN | select.EPOLLET)

Level-triggered (LT): epoll_wait() returns this FD every time you call it, as long as there’s unread data. Safe but potentially more syscalls.

Edge-triggered (ET): epoll_wait() returns this FD only once per new event. You must drain all available data (read in a loop until EAGAIN), or you’ll miss it:

def handle_et_read(fd):
    chunks = []
    while True:
        try:
            data = fd.recv(4096)
            if not data:
                # Connection closed
                return None
            chunks.append(data)
        except BlockingIOError:
            break  # No more data right now
    return b"".join(chunks)

Edge-triggered is more efficient (fewer spurious wakeups) but harder to get right. asyncio uses level-triggered by default. High-performance servers like nginx use edge-triggered.

kqueue Deep Dive (macOS/BSD)

kqueue uses a different abstraction: kevents (kernel events). Instead of registering file descriptors with event masks, you register arbitrary events:

import select

kq = select.kqueue()

# Create a kevent for read readiness on fd
event = select.kevent(
    fd,
    filter=select.KQ_FILTER_READ,
    flags=select.KQ_EV_ADD | select.KQ_EV_ENABLE
)

# Register and wait
kq.control([event], 0)  # register only
events = kq.control([], 10, 1.0)  # wait for up to 10 events, 1s timeout

kqueue is more flexible than epoll — it can monitor file system changes, process exits, signals, and timers, not just socket I/O. This makes it the foundation for macOS’s FSEvents and various monitoring tools.

Building a Minimal Event Loop

Understanding select/poll deeply enough to build a toy event loop:

import selectors
import socket
from collections import defaultdict

class MiniEventLoop:
    def __init__(self):
        self._selector = selectors.DefaultSelector()
        self._callbacks = {}
        self._timers = []
    
    def add_reader(self, fd, callback):
        self._selector.register(fd, selectors.EVENT_READ)
        self._callbacks[fd] = callback
    
    def remove_reader(self, fd):
        self._selector.unregister(fd)
        del self._callbacks[fd]
    
    def call_later(self, delay, callback):
        import time
        deadline = time.monotonic() + delay
        self._timers.append((deadline, callback))
        self._timers.sort()
    
    def run_forever(self):
        import time
        while True:
            # Calculate timeout
            if self._timers:
                timeout = max(0, self._timers[0][0] - time.monotonic())
            else:
                timeout = 1.0
            
            # Wait for I/O
            events = self._selector.select(timeout=timeout)
            
            # Handle I/O events
            for key, mask in events:
                callback = self._callbacks[key.fileobj]
                callback(key.fileobj)
            
            # Handle timers
            now = time.monotonic()
            while self._timers and self._timers[0][0] <= now:
                _, callback = self._timers.pop(0)
                callback()

# Usage
loop = MiniEventLoop()
server = socket.socket()
server.bind(("0.0.0.0", 9000))
server.listen()
server.setblocking(False)

def on_connect(server_sock):
    client, addr = server_sock.accept()
    client.setblocking(False)
    loop.add_reader(client, on_data)

def on_data(client_sock):
    data = client_sock.recv(4096)
    if data:
        client_sock.sendall(data)
    else:
        loop.remove_reader(client_sock)
        client_sock.close()

loop.add_reader(server, on_connect)
loop.run_forever()

This is essentially what asyncio’s event loop does, plus coroutine scheduling and Future/Task management.

Production Patterns

Connection Limiting

import selectors
import socket

MAX_CONNECTIONS = 10000
current_connections = 0

def accept(server_sock):
    global current_connections
    client, addr = server_sock.accept()
    if current_connections >= MAX_CONNECTIONS:
        client.close()
        return
    client.setblocking(False)
    sel.register(client, selectors.EVENT_READ)
    current_connections += 1

Write Buffering

Writes can block too. Register for write events only when you have data to send:

class Connection:
    def __init__(self, sock):
        self.sock = sock
        self.write_buffer = bytearray()
    
    def want_write(self):
        if self.write_buffer:
            sel.modify(self.sock, selectors.EVENT_READ | selectors.EVENT_WRITE)
        else:
            sel.modify(self.sock, selectors.EVENT_READ)
    
    def on_write_ready(self):
        sent = self.sock.send(self.write_buffer)
        self.write_buffer = self.write_buffer[sent:]
        self.want_write()

Thundering Herd Mitigation

With SO_REUSEPORT (Linux 3.9+), multiple processes can bind to the same port. The kernel distributes connections across them:

server = socket.socket()
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
server.bind(("0.0.0.0", 8080))
server.listen()

This eliminates the thundering herd problem where multiple processes compete for the same listen socket. Combined with epoll, this is how high-performance Python servers scale to hundreds of thousands of connections.

Benchmarking I/O Multiplexing

A simple benchmark comparing approaches with 1,000 idle connections and periodic activity on 10:

Method	Time per iteration	Scales to
select	~500µs	~1,000 FDs
poll	~400µs	~10,000 FDs
epoll (level)	~15µs	~1,000,000 FDs
epoll (edge)	~10µs	~1,000,000 FDs

The gap widens dramatically as total connections increase but active connections stay small.

One thing to remember: select and poll scan all file descriptors on every call — epoll and kqueue maintain kernel-side state to report only ready FDs in O(ready) time. This is the fundamental mechanism that enables single-threaded servers to handle hundreds of thousands of connections, and it’s what asyncio uses under the hood.