Python sys.settrace Hooks — Core Concepts

Why sys.settrace matters

Every Python debugger, coverage tool, and most profilers rely on the tracing mechanism exposed by sys.settrace(). Understanding it reveals how these essential tools work and when (and why) they slow your code down.

How the trace function works

When you call sys.settrace(my_trace_func), Python will call my_trace_func at specific points during code execution. The trace function receives three arguments:

• frame — the current execution frame, containing local variables, the code object, and the line number
• event — a string describing what happened: "call", "line", "return", or "exception"
• arg — extra data depending on the event (return value for "return", exception info tuple for "exception", None for others)

The trace function must return either itself (to keep tracing the current scope), a different function (to use a local trace function), or None (to stop tracing that scope).

The event types

call

Fired when a function is about to execute. The frame’s f_code tells you which function. Returning a trace function here enables line-by-line tracing inside that function. Returning None skips tracing that function entirely — a critical optimization.

line

Fired before each line of source code executes. This is the heartbeat of debuggers. By checking the current filename and line number against a set of breakpoints, a debugger knows when to pause.

return

Fired when a function is about to return. The arg parameter contains the return value. Useful for tracing data flow.

exception

Fired when an exception is raised. The arg is a tuple of (exception_class, exception_value, traceback). This fires even for exceptions that are caught by except blocks.

Global vs local trace functions

sys.settrace() sets the global trace function. It gets called on every "call" event. The function it returns becomes the local trace function for that scope, receiving "line", "return", and "exception" events.

This two-tier system is the key optimization. If your global trace function returns None for library code and only returns a local tracer for your application code, you avoid the per-line overhead for most of the execution.

Relationship with sys.setprofile

sys.setprofile() is a lighter-weight alternative that only fires on "call", "return", "c_call", "c_return", and "c_exception" events — no "line" events. Python’s built-in cProfile module uses this. If you don’t need line-level granularity, setprofile is significantly faster.

Performance impact

Tracing adds overhead at every traced event. In CPython, each event requires a Python function call from the C evaluation loop. Benchmarks typically show:

• No tracing: baseline
• setprofile only: 5–15% overhead
• settrace (global only, selective local): 2–5x slower
• settrace (tracing all lines): 5–10x slower

This is why coverage tools like coverage.py have invested heavily in C extensions and, more recently, use sys.monitoring (Python 3.12+) which was designed to be lower-overhead.

Common misconception

People often think sys.settrace hooks into the bytecode interpreter at a low level. It does — but only through a checking mechanism. The interpreter checks a per-frame flag on every line execution. The overhead comes not from the check itself (it’s a simple pointer comparison) but from calling your Python trace function through the C-to-Python function call boundary, which is relatively expensive.

Thread considerations

sys.settrace() only affects the current thread. To trace all threads, you need to call threading.settrace() as well, which sets the trace function for any newly created threads. Existing threads that were started before threading.settrace() was called remain untraced.

The one thing to remember: sys.settrace is a two-tier callback system — a global function catches calls, and the local function it returns handles line-by-line events — and understanding this architecture explains why debugging tools work and why they cost performance.