Python Event Loop Internals — Deep Dive

Dissecting _run_once

The _run_once method in BaseEventLoop (found in Lib/asyncio/base_events.py) is the single most important function in asyncio. Here’s the actual control flow, simplified from CPython source:

def _run_once(self):
    # 1. Compute poll timeout
    #    - If _ready queue is non-empty: timeout = 0 (don't block)
    #    - Elif scheduled heap has entries: timeout = time until first
    #    - Else: timeout = None (block indefinitely)
    timeout = self._calculate_timeout()

    # 2. Poll for I/O events
    event_list = self._selector.select(timeout)
    self._process_events(event_list)

    # 3. Move expired scheduled callbacks to _ready
    now = self.time()
    while self._scheduled:
        handle = self._scheduled[0]
        if handle._when > now:
            break
        handle = heapq.heappop(self._scheduled)
        if not handle._cancelled:
            self._ready.append(handle)

    # 4. Execute all ready callbacks
    ntodo = len(self._ready)
    for _ in range(ntodo):
        handle = self._ready.popleft()
        if not handle._cancelled:
            handle._run()

Key observations:

• The ready queue (_ready) is a collections.deque — O(1) append and popleft.
• The scheduled heap (_scheduled) is a min-heap sorted by _when timestamps.
• Only ntodo callbacks run per iteration — callbacks added during execution wait for the next cycle. This prevents starvation.

Handle Objects

Every scheduled callback is wrapped in a Handle or TimerHandle object:

class Handle:
    __slots__ = ('_callback', '_args', '_cancelled', '_loop',
                 '_source_traceback', '_repr', '__weakref__')

    def cancel(self):
        self._cancelled = True

    def _run(self):
        self._callback(*self._args)

TimerHandle extends this with a _when attribute and comparison methods so it can sit in the heap. The cancelled flag is checked at execution time — cancelled handles stay in the heap but get skipped. This lazy deletion avoids expensive heap removal operations.

Selector Mechanics in Detail

epoll (Linux)

The EpollSelector wraps the epoll_ctl() and epoll_wait() system calls:

# Registration
selector.register(fd, selectors.EVENT_READ, callback_data)
# Internally calls: epoll_ctl(epfd, EPOLL_CTL_ADD, fd, events)

# Polling
events = selector.select(timeout)
# Internally calls: epoll_wait(epfd, maxevents, timeout_ms)

epoll operates in level-triggered mode by default in asyncio. This means if a socket has data and you don’t read it all, it will show up as ready again on the next poll. Edge-triggered mode (report only new events) is more efficient but harder to get right.

kqueue (macOS/BSD)

macOS uses kqueue, which works differently — events are registered as individual kevent structs, and the poll returns a list of fired events. The selectors module abstracts this difference away from asyncio.

IOCP (Windows)

Windows uses ProactorEventLoop with I/O Completion Ports — a fundamentally different model where the OS completes the I/O operation and notifies you, rather than telling you a descriptor is ready. This is why Windows has a separate event loop class.

Clock Source: loop.time()

The event loop’s clock uses time.monotonic() — a clock that never goes backward, even if the system clock is adjusted. All timer scheduling (call_later, call_at, asyncio.sleep) uses this monotonic clock.

# These are equivalent:
loop.call_later(5, callback)
loop.call_at(loop.time() + 5, callback)

The resolution is platform-dependent but typically microsecond-level on modern systems.

Task Scheduling Internals

When you call asyncio.create_task(coro), the chain is:

1. Task.__init__ wraps the coroutine and calls loop.call_soon(self.__step)
2. __step calls coro.send(None) to advance the coroutine
3. The coroutine runs until it hits an await, which returns a Future
4. __step calls future.add_done_callback(self.__wakeup)
5. When the future resolves, __wakeup calls loop.call_soon(self.__step)
6. Back to step 2 — the coroutine resumes

This is the core machinery that makes async/await work. Every await point is a potential suspension where control returns to _run_once.

# Simplified from CPython's tasks.py
class Task(Future):
    def __step(self, exc=None):
        try:
            if exc is None:
                result = self._coro.send(None)
            else:
                result = self._coro.throw(exc)
        except StopIteration as e:
            self.set_result(e.value)
        except Exception as e:
            self.set_exception(e)
        else:
            # result is a Future — wait for it
            result.add_done_callback(self.__wakeup)

    def __wakeup(self, future):
        try:
            future.result()
        except Exception as exc:
            self.__step(exc)
        else:
            self.__step()

Thread Safety and call_soon_threadsafe

The event loop is not thread-safe by default. call_soon() appends to _ready without locking. If you need to schedule work from another thread:

loop.call_soon_threadsafe(callback, *args)

This acquires a lock, appends to _ready, and writes a byte to a self-pipe. The self-pipe trick is essential — the main thread might be blocked in selector.select(). Writing to the self-pipe wakes it up so it processes the new callback immediately.

def call_soon_threadsafe(self, callback, *args):
    handle = self._call_soon(callback, args)
    self._write_to_self()  # Wake the selector
    return handle

Writing a Custom Event Loop

You can replace the default loop by implementing asyncio.AbstractEventLoop. At minimum, you need:

• _run_once() — the main iteration
• run_forever() / run_until_complete() — entry points
• call_soon(), call_later(), call_at() — scheduling
• add_reader() / add_writer() — I/O registration
• create_future() / create_task() — object factories

uvloop is the most prominent custom loop — it replaces Python’s selector with libuv (the same library behind Node.js), achieving 2-4x throughput for many workloads:

import uvloop

async def main():
    # Your async code here
    pass

uvloop.run(main())  # Python 3.12+

Performance Characteristics

Metric	SelectorEventLoop	uvloop	ProactorEventLoop
Socket echo throughput	~25k req/s	~75k req/s	~15k req/s
Timer resolution	~1ms	~1ms	~15ms
Max watched FDs	OS limit (~1M)	OS limit	Unlimited (IOCP)
Implementation	Pure Python + C selector	Cython + libuv	Pure Python + IOCP

Debugging the Loop Internals

Beyond debug=True, you can instrument the loop directly:

import asyncio

async def monitor_loop():
    loop = asyncio.get_running_loop()
    while True:
        # Number of pending callbacks
        ready = len(loop._ready)
        scheduled = len(loop._scheduled)
        print(f"Ready: {ready}, Scheduled: {scheduled}")
        await asyncio.sleep(1)

For production monitoring, hook into loop.set_exception_handler() to catch unhandled exceptions in callbacks, and use loop.slow_callback_duration to tune the warning threshold.

One thing to remember: The event loop is fundamentally a while True that computes a timeout, polls the OS selector, promotes expired timers, and drains the ready queue — every advanced async behavior emerges from this simple cycle.