Python aiofiles Async I/O — Core Concepts

The Problem: Files Block the Event Loop

Network I/O in asyncio is truly non-blocking — the OS notifies the event loop when data arrives. But file I/O on most operating systems is synchronous. When you call open('file.txt').read(), the calling thread blocks until the disk returns data.

In an async program, this is a problem. The event loop runs on a single thread. If that thread blocks for a file read, everything stops — other coroutines, incoming network requests, timer callbacks. A 10ms disk read during a busy web server can stall hundreds of requests.

How aiofiles Works

aiofiles solves this by running file operations in a thread pool:

1. Your async code calls await f.read()
2. aiofiles submits the blocking f.read() to a thread pool executor
3. The event loop continues running other coroutines
4. When the thread completes the file read, the result is returned to your coroutine

import aiofiles

async def read_config():
    async with aiofiles.open('config.json', mode='r') as f:
        contents = await f.read()
    return json.loads(contents)

The API mirrors Python’s built-in open():

# Writing
async with aiofiles.open('output.txt', mode='w') as f:
    await f.write('Hello, world!\n')

# Appending
async with aiofiles.open('log.txt', mode='a') as f:
    await f.write(f'{timestamp}: {event}\n')

# Binary mode
async with aiofiles.open('image.png', mode='rb') as f:
    data = await f.read()

# Line by line
async with aiofiles.open('data.csv') as f:
    async for line in f:
        process(line)

File System Operations

aiofiles also wraps os and os.path operations:

from aiofiles import os as aios

# Check existence
exists = await aios.path.exists('/tmp/data.txt')

# File stats
stat = await aios.stat('/tmp/data.txt')
print(stat.st_size)

# Directory listing
entries = await aios.listdir('/tmp')

# Rename
await aios.rename('old.txt', 'new.txt')

# Remove
await aios.remove('temp.txt')

# Create directory
await aios.makedirs('/tmp/new/path', exist_ok=True)

When You Need aiofiles

Definitely use it:

• Web servers that serve or process uploaded files
• Async applications that write logs or cache files
• Any async code path that reads configuration or data files during request handling

Probably don’t need it:

• Startup/shutdown code (reading config at boot time doesn’t need to be async)
• Scripts that aren’t async to begin with
• Tiny files in low-concurrency scenarios (the overhead of threading isn’t worth it)

Performance Consideration

aiofiles adds overhead compared to direct file access — each operation creates a thread pool future and context-switches between threads. For a single file read, open().read() is faster.

The benefit appears under concurrency. If your async server handles 100 concurrent requests and each one reads a file, aiofiles prevents those 100 reads from serializing on the event loop thread.

Alternatives

asyncio.to_thread() (Python 3.9+) — you can wrap any blocking call manually:

import asyncio

async def read_file(path):
    return await asyncio.to_thread(lambda: open(path).read())

This works but is more verbose and doesn’t provide the familiar file handle API.

anyio.open_file() — AnyIO provides its own async file wrapper that works on both asyncio and Trio.

Common Misconception

“aiofiles makes file I/O faster.” It doesn’t. Individual file operations may be slightly slower due to thread pool overhead. What aiofiles does is prevent file I/O from blocking other tasks. It’s about responsiveness under concurrency, not raw throughput.

One thing to remember: aiofiles wraps blocking file operations in a thread pool so they don’t stall your async event loop. Use it in any async code path that touches the filesystem during request handling — but understand it’s about non-blocking behavior, not speed.