Structured Logging with structlog in Python — Core Concepts

Traditional Python logging with the logging module outputs unstructured text. Parsing “ERROR: Failed to process order 789 for user 42” requires regular expressions and guesswork. Structured logging outputs key-value pairs or JSON, making logs machine-readable while staying human-readable during development.

structlog is the most popular structured logging library for Python. It wraps or replaces the standard logging module and adds a processor pipeline that transforms log entries before output.

Why structured logging matters

In a system with 20 services each producing thousands of log lines per minute, you need to answer questions like:

• “Show me all errors for user 42 in the last hour”
• “How many payment timeouts happened today?”
• “What was the request ID for this failed order?”

With JSON logs, these become simple queries in Elasticsearch, Loki, or CloudWatch. With text logs, they require brittle regex patterns that break when someone changes the log message wording.

Basic setup

import structlog

structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.dev.ConsoleRenderer()  # Pretty output for development
    ],
    wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
)

log = structlog.get_logger()
log.info("order_created", order_id=789, user_id=42, total=99.50)

Development output:

2026-03-27 14:30:00 [info] order_created  order_id=789 user_id=42 total=99.5

For production, swap ConsoleRenderer for JSONRenderer:

structlog.processors.JSONRenderer()

Output:

{"event": "order_created", "order_id": 789, "user_id": 42, "total": 99.5, "level": "info", "timestamp": "2026-03-27T14:30:00Z"}

The processor pipeline

structlog’s power comes from its processor chain. Each processor is a function that receives the log event dictionary and returns a modified version. Processors run in order.

Common processors:

• add_log_level: Adds "level": "info" to the event dict.
• TimeStamper: Adds a timestamp.
• merge_contextvars: Merges context variables bound earlier (see below).
• StackInfoRenderer: Adds stack traces for exceptions.
• JSONRenderer or ConsoleRenderer: Final output formatting.

You can write custom processors:

def add_service_name(logger, method_name, event_dict):
    event_dict["service"] = "order-service"
    return event_dict

structlog.configure(
    processors=[
        add_service_name,
        structlog.processors.add_log_level,
        structlog.processors.JSONRenderer(),
    ]
)

Context binding

structlog lets you bind context that appears in all subsequent log entries. This eliminates passing request IDs through every function.

Logger-level binding

log = structlog.get_logger()
log = log.bind(request_id="abc-123", user_id=42)

log.info("processing_started")  # Includes request_id and user_id
log.info("step_completed", step="validation")  # Also includes them

Context variables (thread/async safe)

For web applications, bind context per-request using contextvars:

from structlog.contextvars import bind_contextvars, clear_contextvars

# In middleware
def before_request():
    clear_contextvars()
    bind_contextvars(request_id=request.headers.get("X-Request-ID"))

# Anywhere in the request lifecycle
log.info("payment_processed", amount=42.50)
# Output includes request_id automatically

This works correctly with asyncio — each coroutine chain gets its own context.

Integration with standard logging

Many third-party libraries use Python’s standard logging module. structlog can capture these logs and process them through the same pipeline:

import logging
import structlog

structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.stdlib.add_logger_name,
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
    logger_factory=structlog.stdlib.LoggerFactory(),
)

# Configure standard logging to use structlog
logging.basicConfig(format="%(message)s", handlers=[logging.StreamHandler()])

Now both structlog.get_logger() and logging.getLogger() produce consistent JSON output.

Common misconception

“Structured logging is just logging in JSON format.” JSON output is the end result, but the real value is in the processor pipeline and context binding. structlog lets you automatically enrich every log entry with request IDs, user context, and service metadata without cluttering your application code with logging boilerplate.

When to adopt structlog

If you are starting a new Python service, use structlog from day one. The cost is minimal and the debugging benefits are immediate. For existing services, migrate incrementally — structlog’s stdlib integration means you can adopt it without rewriting every logging.getLogger() call.

One thing to remember: structlog turns Python logging into a data pipeline — processors enrich events automatically, context binding adds request metadata everywhere, and JSON output makes every log entry queryable.