Python Loguru Modern Logging — Deep Dive

Loguru’s simple API conceals a flexible architecture. Understanding its internals lets you build logging pipelines that match production requirements without losing the developer experience that makes Loguru attractive.

Architecture overview

Loguru uses a single global logger instance. Under the hood, this object maintains a list of sinks (handlers). When you call logger.info(msg), the library:

1. Creates a Record dict with timestamp, level, message, caller info, and bound extras.
2. Iterates registered sinks.
3. For each sink, checks the level filter and any custom filter function.
4. Formats the record using the sink’s format string.
5. Calls the sink’s write function (synchronous or async).

There is no logger hierarchy like the standard library’s dotted-name tree. Filtering happens per-sink, not per-logger.

Custom sink implementations

A sink can be any callable accepting a formatted string:

from loguru import logger

def datadog_sink(message):
    """Ship logs to Datadog via HTTP."""
    record = message.record
    payload = {
        "message": record["message"],
        "level": record["level"].name,
        "service": "order-api",
        "timestamp": record["time"].isoformat(),
        **record["extra"]
    }
    # batch in practice — this is simplified
    httpx.post("https://http-intake.logs.datadoghq.com/v1/input",
               json=payload,
               headers={"DD-API-KEY": DD_KEY})

logger.add(datadog_sink, level="WARNING", serialize=False)

Key detail: the message parameter is a str subclass with a .record attribute containing the raw Record dict. This dual interface lets sinks access both formatted text and structured data.

Async sinks

For non-blocking log shipping, Loguru supports coroutine sinks:

import httpx
from loguru import logger

async def async_webhook(message):
    async with httpx.AsyncClient() as client:
        await client.post(WEBHOOK_URL, json={"text": str(message)})

logger.add(async_webhook, level="ERROR")

Loguru runs async sinks in a dedicated event loop on a background thread. This means they work even in synchronous applications. However, each async sink gets its own thread — avoid adding dozens of async sinks.

Enqueue for thread safety

For multi-threaded applications, enqueue=True routes all log calls through a multiprocessing.SimpleQueue:

logger.add("app.log", enqueue=True, rotation="100 MB")

This guarantees:

• No thread-lock contention on the file.
• Log lines are written in order.
• The calling thread returns immediately after enqueue.

Critical caveat: If the process crashes (SIGKILL, OOM), queued records that haven’t been written are lost. For crash-safe logging, pair enqueue=True with a short flush interval or use a watchdog process.

Rotation, retention, and compression

Loguru’s file sink supports rich lifecycle management:

logger.add(
    "logs/app_{time:YYYY-MM-DD}.log",
    rotation="00:00",        # new file at midnight
    retention="30 days",     # delete older files
    compression="gz",        # gzip rotated files
    encoding="utf-8",
    enqueue=True
)

Rotation triggers:

• Time strings: "00:00", "1 week", "Sunday"
• Size strings: "500 MB", "1 GB"
• Callable: rotation=lambda msg, file: file.stat().st_size > 1e8

Retention accepts time durations or a callable that receives a list of rotated files and can delete selectively.

Complete exception diagnostics

Loguru’s traceback enhancement deserves a detailed look:

@logger.catch(reraise=True)
def divide(a, b):
    c = a / b
    return c

divide(10, 0)

Output includes:

> File "app.py", line 4, in divide
    c = a / b
        │   └ 0
        └ 10
ZeroDivisionError: division by zero

The variable annotations are generated by inspecting each frame’s f_locals. This works for:

• Local variables
• Function arguments
• Comprehension variables
• Nested attribute access (with depth limits)

For production, you may want to limit this to avoid leaking sensitive data:

logger.add("prod.log", diagnose=False)  # disable variable inspection

Standard library interception — production pattern

The intercept handler pattern needs care in production:

import logging
import sys
from loguru import logger

class InterceptHandler(logging.Handler):
    def emit(self, record: logging.LogRecord):
        # Find the Loguru level matching the standard level
        try:
            level = logger.level(record.levelname).name
        except ValueError:
            level = record.levelno

        # Find caller from where the log call originated
        frame, depth = logging.currentframe(), 2
        while frame and frame.f_code.co_filename == logging.__file__:
            frame = frame.f_back
            depth += 1

        logger.opt(depth=depth, exception=record.exc_info).log(
            level, record.getMessage()
        )

def setup_logging(level="INFO"):
    # Remove default Loguru handler
    logger.remove()

    # Add production sinks
    logger.add(sys.stderr, level=level)
    logger.add("app.log", rotation="100 MB", retention="7 days",
               serialize=True, enqueue=True)

    # Intercept standard library logging
    logging.basicConfig(handlers=[InterceptHandler()], level=0, force=True)

    # Quiet noisy third-party loggers
    for name in ["uvicorn", "sqlalchemy.engine"]:
        logging.getLogger(name).setLevel(logging.WARNING)

The depth calculation ensures Loguru reports the correct file and line number from the original caller, not from the intercept handler.

Filtering patterns

Per-module filtering

# Only accept records from the "payments" module
logger.add("payments.log", filter="payments")

# Custom filter function
def no_health_checks(record):
    return "/health" not in record["message"]

logger.add("api.log", filter=no_health_checks)

Dynamic level control

level_gate = {"current": "INFO"}

def level_filter(record):
    return record["level"].no >= logger.level(level_gate["current"]).no

sink_id = logger.add(sys.stderr, filter=level_filter, level=0)

# At runtime, switch to DEBUG without restarting
level_gate["current"] = "DEBUG"

This enables runtime log level changes via admin endpoints — useful for debugging production issues without redeployment.

Contextualize for request tracing

In web frameworks, bind request-scoped data:

from loguru import logger
from starlette.middleware.base import BaseHTTPMiddleware
import uuid

class LogContextMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        request_id = request.headers.get("X-Request-ID", str(uuid.uuid4()))
        with logger.contextualize(request_id=request_id, path=request.url.path):
            logger.info("Request started")
            response = await call_next(request)
            logger.info("Request completed", status=response.status_code)
            return response

Every log line emitted within the context manager includes request_id and path — including lines from deeply nested service calls.

Performance considerations

Benchmark comparison (Python 3.12, 100k log lines, single thread):

Configuration	Time (s)	Notes
Standard logging + StreamHandler	0.21	Baseline
Loguru default (stderr)	0.35	Color formatting overhead
Loguru + serialize=True	0.48	JSON serialization
Loguru + enqueue=True	0.12	Only measures enqueue time
Loguru + file + rotation	0.38	100 MB rotation threshold

Loguru is 40-60% slower than the standard library for simple cases because of richer default formatting. With enqueue=True, the calling thread is faster because I/O happens off-thread. For most applications, the difference is negligible — you’ll bottleneck on network I/O or database queries long before logging matters.

Migration strategy

Moving from standard logging to Loguru in an existing codebase:

1. Install Loguru and set up the intercept handler — all existing code works unchanged.
2. New code uses from loguru import logger directly.
3. Gradually replace logging.getLogger() calls in modules you touch.
4. Remove old handler configuration once all modules are migrated.

This is a rolling migration, not a flag day. The intercept handler ensures both systems coexist.

When to stick with standard logging

Loguru isn’t always the right choice:

• Libraries published to PyPI should use logging.getLogger(__name__) — adding a Loguru dependency to a library forces it on all consumers.
• Environments with strict dependency policies (government, regulated industries) may not approve third-party logging.
• Complex handler topologies (per-module syslog routing, custom protocol handlers) are easier with the standard library’s explicit hierarchy.

One thing to remember: Loguru’s value is removing accidental complexity from logging configuration. Use it for applications where developer velocity matters more than handler flexibility, and pair it with the intercept handler to preserve compatibility with the standard library ecosystem.