Macro Systems — Core Concepts

What Are Macros?

In languages like Lisp, Rust, and Julia, macros are first-class features that transform code before it executes. They operate on the code’s structure (its syntax tree), not its runtime values. Python lacks built-in macros, but its dynamic features enable equivalent patterns.

Python’s “macro-like” approaches fall into three categories:

1. Decorator-based macros — transform functions and classes at definition time
2. AST transformation — rewrite the syntax tree before compilation
3. Import-time hooks — intercept and modify source code during import

Decorator-Based Macros

Decorators are Python’s most accessible macro pattern. They receive a function or class and return a modified version:

import functools
import time

def retry(max_attempts=3, delay=1.0):
    """Macro-like decorator that adds retry logic to any function."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            last_exception = None
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    last_exception = e
                    if attempt < max_attempts - 1:
                        time.sleep(delay * (2 ** attempt))
            raise last_exception
        return wrapper
    return decorator

@retry(max_attempts=5, delay=0.5)
def fetch_data(url):
    # This function now automatically retries on failure
    pass

Class decorators can add entire methods:

def auto_repr(cls):
    """Add __repr__ based on __init__ parameters."""
    import inspect
    params = list(inspect.signature(cls.__init__).parameters)[1:]  # skip self

    def __repr__(self):
        attrs = ', '.join(f'{p}={getattr(self, p)!r}' for p in params)
        return f'{cls.__name__}({attrs})'

    cls.__repr__ = __repr__
    return cls

@auto_repr
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

print(Point(1, 2))  # Point(x=1, y=2)

AST Transformation

For true macro-like power, you can transform Python’s Abstract Syntax Tree before code runs:

import ast
import inspect
import textwrap

def debug_expressions(func):
    """Transform function so every expression also prints its value."""
    source = textwrap.dedent(inspect.getsource(func))
    tree = ast.parse(source)

    class DebugTransformer(ast.NodeTransformer):
        def visit_Expr(self, node):
            # Wrap standalone expressions with a print
            if isinstance(node.value, ast.Call):
                return node  # don't wrap function calls
            debug_call = ast.Expr(
                value=ast.Call(
                    func=ast.Name(id='print', ctx=ast.Load()),
                    args=[
                        ast.Constant(value=f'DEBUG: '),
                        node.value,
                    ],
                    keywords=[],
                )
            )
            return [debug_call, node]

    tree = DebugTransformer().visit(tree)
    ast.fix_missing_locations(tree)

    code = compile(tree, inspect.getfile(func), 'exec')
    namespace = {}
    exec(code, func.__globals__, namespace)
    return namespace[func.__name__]

This approach is how tools like pytest transform assert statements into detailed failure messages — they rewrite the AST to capture intermediate values.

Import-Time Transformation

The most powerful macro approach intercepts modules during import and transforms them:

import sys
import ast
import importlib.abc
import importlib.util

class MacroFinder(importlib.abc.MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        # Only handle modules in our macro-enabled package
        if not fullname.startswith('macro_enabled.'):
            return None

        # Find the actual file
        for finder in sys.meta_path:
            if finder is self:
                continue
            spec = getattr(finder, 'find_spec', lambda *a: None)(fullname, path, target)
            if spec:
                spec.loader = MacroLoader(spec.loader)
                return spec
        return None

class MacroLoader(importlib.abc.Loader):
    def __init__(self, original_loader):
        self.original = original_loader

    def exec_module(self, module):
        # Read source, transform, then execute
        source = self.original.get_data(module.__spec__.origin)
        tree = ast.parse(source)
        tree = apply_macros(tree)  # your AST transformations
        code = compile(tree, module.__spec__.origin, 'exec')
        exec(code, module.__dict__)

sys.meta_path.insert(0, MacroFinder())

The MacroPy Legacy

MacroPy was a pioneering library that brought Lisp-style macros to Python. Though no longer maintained, its ideas influenced the ecosystem:

# MacroPy-style (historical example)
# from macropy.macros.pattern import macros, switch

# result = switch(value):
#     1 >> "one"
#     2 >> "two"
#     _ >> "other"

MacroPy used import hooks to transform code at import time, proving that Python’s import system is flexible enough to support full macro expansion.

Common Misconception

People often think decorators and macros are the same thing. Decorators operate on runtime objects (the function or class after it is defined). True macros operate on source code structure (the AST before compilation). A decorator cannot change the syntax inside a function body — it can only wrap or replace the function. AST-based macros can rewrite anything inside the function.

When to Use Each Approach

Approach	Power	Complexity	Debugging
Decorators	Function/class wrapping	Low	Good
__init_subclass__	Class body transformation	Medium	Good
AST rewriting	Any code transformation	High	Poor
Import hooks	Module-wide transformation	Very high	Poor

One thing to remember: Python’s macro-like capabilities range from simple decorators (which wrap functions) to full AST transformers (which can rewrite any code) — choose the simplest approach that solves your problem, since more powerful transformation techniques make code harder to debug.