TaskGroup and Structured Concurrency in Python — Core Concepts

Why TaskGroup matters

Concurrent programming is full of invisible bugs: tasks that fail silently, exceptions swallowed by callbacks, background work that outlives the function that started it. Python 3.11 introduced asyncio.TaskGroup to address these problems by enforcing structured concurrency — the idea that concurrent tasks should have a clear lifetime tied to a code block.

The problem with gather

Before TaskGroup, the standard tool was asyncio.gather():

results = await asyncio.gather(fetch_users(), fetch_orders(), fetch_inventory())

This works fine when everything succeeds. But when something fails:

• With return_exceptions=False (default), one failure cancels nothing — other tasks keep running, and you get one exception while losing track of the rest.
• With return_exceptions=True, failures mix with results, forcing you to manually check each return value.

Neither option gives you clean cancellation or proper error reporting for multiple failures.

How TaskGroup works

import asyncio

async def main():
    async with asyncio.TaskGroup() as tg:
        task1 = tg.create_task(fetch_users())
        task2 = tg.create_task(fetch_orders())
        task3 = tg.create_task(fetch_inventory())

    # All three tasks are guaranteed done here
    users = task1.result()
    orders = task2.result()
    inventory = task3.result()

The async with block defines the task lifetime:

1. Create tasks inside the block using tg.create_task()
2. Wait — the block doesn’t exit until all tasks complete
3. Cancel on failure — if any task raises, the others are cancelled
4. Collect errors — all exceptions are bundled into an ExceptionGroup

ExceptionGroup: the companion feature

When multiple tasks fail, TaskGroup raises an ExceptionGroup containing all the exceptions:

try:
    async with asyncio.TaskGroup() as tg:
        tg.create_task(might_fail_1())
        tg.create_task(might_fail_2())
except* ValueError as eg:
    print(f"Value errors: {eg.exceptions}")
except* ConnectionError as eg:
    print(f"Connection errors: {eg.exceptions}")

The except* syntax (also new in Python 3.11) catches specific exception types from within the group without losing the others. This is a fundamental shift from Python’s traditional one-exception-at-a-time model.

Structured concurrency explained

The core principle: every task has an owner, and the owner waits for the task to finish. No fire-and-forget. No orphan tasks.

Traditional async code often looks like this:

# Unstructured: task may outlive the function
asyncio.create_task(background_work())

With structured concurrency:

# Structured: task lifetime is bound to this block
async with asyncio.TaskGroup() as tg:
    tg.create_task(background_work())
# background_work is guaranteed done here

This makes reasoning about code easier — you always know exactly when concurrent work starts and stops.

Common misconception

“TaskGroup is just a cleaner syntax for gather.”

TaskGroup changes the error-handling semantics fundamentally. With gather, you get partial results and swallowed exceptions. With TaskGroup, you get automatic cancellation and complete error reporting via ExceptionGroup. It’s a different concurrency model, not a syntax upgrade.

When to use TaskGroup vs gather

Scenario	Use TaskGroup	Use gather
All tasks must succeed	✅	⚠️
Partial results are acceptable	❌	✅
Need clean cancellation on failure	✅	❌
Python 3.10 or earlier	❌	✅
Dynamic number of tasks	✅	✅

Use gather with return_exceptions=True when you want best-effort results. Use TaskGroup when task failures should stop everything.

Practical tips

• Don’t create tasks outside the group. If you need a long-running background task that outlives the group, that’s a design signal — reconsider whether structured concurrency fits that part of your system.
• Handle ExceptionGroup properly. Don’t bare except Exception — it won’t catch ExceptionGroup in Python 3.11+ since ExceptionGroup inherits from BaseException.
• Nest TaskGroups for hierarchical concurrency. Inner groups can fail and be caught without cancelling the outer group’s other tasks.

The one thing to remember: TaskGroup ties task lifetimes to code blocks — when the block exits, all tasks are done, all errors are reported, and nothing is left running in the shadows.