Python Async Resource Management — Core Concepts

Why Async Resources Need Special Handling

In synchronous code, cleanup is straightforward — __exit__ runs immediately. In async code, cleanup often involves I/O: closing a connection sends a FIN packet, releasing a lock might wake another coroutine, shutting down a pool drains pending queries. These operations need await, which regular __exit__ can’t do.

Async Context Managers

The async with statement calls __aenter__ and __aexit__, both of which are coroutines:

class DatabaseConnection:
    async def __aenter__(self):
        self.conn = await asyncpg.connect("postgresql://localhost/mydb")
        return self.conn

    async def __aexit__(self, exc_type, exc_val, exc_tb):
        await self.conn.close()
        return False  # Don't suppress exceptions

The contextlib module provides a decorator shortcut:

from contextlib import asynccontextmanager

@asynccontextmanager
async def db_connection():
    conn = await asyncpg.connect("postgresql://localhost/mydb")
    try:
        yield conn
    finally:
        await conn.close()

Connection Pools

Creating a new connection for every operation is expensive. Pools maintain a set of reusable connections:

async def main():
    pool = await asyncpg.create_pool(
        "postgresql://localhost/mydb",
        min_size=5,
        max_size=20
    )

    async with pool.acquire() as conn:
        # Borrows a connection from the pool
        result = await conn.fetch("SELECT * FROM users")
    # Connection returns to the pool (not closed)

    await pool.close()  # Closes all connections on shutdown

The pool’s acquire() is an async context manager that checks out a connection and returns it on exit — even if your code raises an exception.

The Cancellation Cleanup Problem

When a task is cancelled, __aexit__ still runs — but the cleanup itself might be cancelled too:

async def risky():
    async with db_connection() as conn:
        await long_query(conn)  # Cancelled here
    # __aexit__ runs, calls conn.close()
    # But if close() is also an await, it might get cancelled too!

The solution: shield critical cleanup operations:

@asynccontextmanager
async def safe_connection():
    conn = await asyncpg.connect(DSN)
    try:
        yield conn
    finally:
        await asyncio.shield(conn.close())

Resource Stacks

When you need to manage multiple resources with proper LIFO cleanup:

from contextlib import AsyncExitStack

async def setup_services():
    async with AsyncExitStack() as stack:
        db = await stack.enter_async_context(db_pool())
        cache = await stack.enter_async_context(redis_pool())
        broker = await stack.enter_async_context(message_broker())
        # All three are available here
    # Cleaned up in reverse order: broker → cache → db

AsyncExitStack is essential when the number of resources is dynamic or determined at runtime.

Shared Resources Across Tasks

When multiple tasks share a resource, you need careful coordination:

async def main():
    async with db_pool() as pool:
        async with asyncio.TaskGroup() as tg:
            for url in urls:
                tg.create_task(process(url, pool))
        # TaskGroup ensures all tasks finish before pool closes

The nesting order matters: the resource (pool) must outlive all tasks that use it. TaskGroup inside the resource context manager guarantees this.

Common Misconception: “finally Always Runs in Async Code”

finally does run during cancellation, but if the finally block contains an await, that await can itself receive a CancelledError. This is different from synchronous code where finally always completes. For critical cleanup, use asyncio.shield() or keep cleanup synchronous when possible.

Patterns Summary

Pattern	Use Case
async with	Single resource, single task
AsyncExitStack	Multiple or dynamic resources
Connection pool	High-throughput database/cache access
Shield in __aexit__	Cleanup that must complete despite cancellation
TaskGroup inside resource scope	Shared resource across concurrent tasks

One thing to remember: Always nest resources outside task groups — the resource must outlive every task that uses it, and async with guarantees cleanup even when cancellation strikes.