Python 3.12 New Features — Deep Dive

Technical overview

Python 3.12 (October 2023) was a pivotal transitional release. On the surface, it simplified type annotations and f-strings. Under the hood, it restructured CPython’s interpreter to support per-interpreter GILs — the most significant architectural change since the introduction of the GIL itself.

PEP 695 — type parameter syntax internals

Scoping model

The new [T] syntax introduces an annotation scope — a new scope type that sits between function local and module global:

class Container[T]:
    # T is visible here
    items: list[T]

    def add(self, item: T) -> None:
        # T is visible here too
        self.items.append(item)

# T is NOT visible here

This is different from the old TypeVar approach where T = TypeVar("T") was module-level and polluted the namespace.

TypeAliasType objects

The type statement creates TypeAliasType instances:

type Vector[T] = list[T]

# Equivalent to (but not identical to):
# Vector = TypeAliasType("Vector", list[T], type_params=(T,))

print(type(Vector))        # <class 'typing.TypeAliasType'>
print(Vector.__value__)    # list[T] — lazily evaluated
print(Vector.__type_params__)  # (T,)

Lazy evaluation means forward references work naturally:

type Tree[T] = Node[T] | Leaf[T]  # Node and Leaf can be defined later

No need for from __future__ import annotations or string quotes.

Variance inference

With PEP 695, type checkers infer variance automatically:

class Box[T]:          # T is invariant (used in both read and write positions)
    value: T

class Producer[T]:     # T is covariant (only in return positions)
    def get(self) -> T: ...

class Consumer[T]:     # T is contravariant (only in parameter positions)
    def accept(self, item: T) -> None: ...

You can still be explicit: class Producer[T: covariant].

Bound and constraint syntax

# Upper bound
class NumberBox[T: (int, float)]:  # T must be int or float
    value: T

# With bound
class Comparable[T: SupportsLessThan]:  # T must implement SupportsLessThan
    pass

F-string PEG parser integration

Before 3.12: custom tokenizer

F-strings were handled by a hand-coded tokenizer that:

• Tracked quote nesting depth (max 3 levels)
• Disallowed backslashes in expressions
• Couldn’t handle comments in multi-line expressions
• Required same-quote-type matching at each nesting level

After 3.12: full PEG grammar

F-strings are now parsed by the same PEG parser that handles all Python code. The grammar addition:

fstring: FSTRING_START fstring_middle* FSTRING_END
fstring_middle: FSTRING_MIDDLE | '{' annotated_rhs '='? ['!' NAME] [':' fstring_format_spec] '}'

The tokenizer emits new token types: FSTRING_START, FSTRING_MIDDLE, FSTRING_END. The parser handles the rest, including arbitrary nesting.

Performance impact

F-string parsing is slightly faster in 3.12 because the unified parser avoids the overhead of switching between the main parser and the custom f-string tokenizer. Benchmark: ~3% faster for f-string-heavy code.

Per-interpreter GIL (PEP 684)

Architecture

Before 3.12:

Process → one GIL → all threads → all interpreters

After 3.12:

Process → Interpreter 1 → GIL 1 → threads of interpreter 1
        → Interpreter 2 → GIL 2 → threads of interpreter 2

Implementation details

1. Global state migration — hundreds of global variables were moved to PyInterpreterState. This multi-year effort touched every subsystem.
2. Module state isolation — extension modules must opt in to per-interpreter isolation via Py_mod_multiple_interpreters. Modules that use global state are restricted to the main interpreter.
3. GIL ownership — each interpreter has its own _gil_runtime_state. Thread switching only affects threads in the same interpreter.

C API changes

// Create an isolated interpreter with its own GIL
PyInterpreterConfig config = {
    .use_main_obmalloc = 0,
    .allow_fork = 0,
    .allow_exec = 0,
    .allow_threads = 1,
    .allow_daemon_threads = 0,
    .check_multi_interp_extensions = 1,
    .gil = PyInterpreterConfig_OWN_GIL,  // Key flag
};
PyStatus status = Py_NewInterpreterFromConfig(&interp, &config);

Limitations in 3.12

• No convenient Python-level API yet (the _interpreters module is private)
• Many C extensions haven’t opted into multi-interpreter support
• Data sharing between interpreters requires serialisation (no shared objects)
• numpy, pandas, and most C extensions still require the main interpreter

Comprehension inlining (PEP 709)

List, set, and dict comprehensions no longer create a separate function object:

# Before 3.12: comprehension creates an inner function
result = [x * 2 for x in range(10)]
# Internally: _listcomp = lambda .0: [x * 2 for x in .0]; result = _listcomp(range(10))

# After 3.12: inlined into the enclosing scope
# No function call overhead, ~2x faster for small comprehensions

Benchmark impact:

Comprehension type	Speedup
[x for x in range(100)]	1.8×
{k: v for k, v in items}	1.6×
{x for x in range(100)}	1.9×

The semantic change: comprehension variables can now leak into the enclosing scope. In practice, this rarely matters because comprehension variable names are typically short-lived, but it’s a breaking change from the formal semantics.

distutils removal — migration guide

distutils was deprecated in 3.10 and removed in 3.12. Migration paths:

distutils feature	Replacement
distutils.core.setup()	setuptools.setup()
distutils.core.Extension	setuptools.Extension
distutils.command.*	setuptools.command.*
distutils.sysconfig	sysconfig (stdlib)
distutils.version	packaging.version
distutils.util.strtobool	Write your own (it’s 5 lines)

For new projects, skip setuptools entirely and use pyproject.toml with hatchling, flit-core, or pdm-backend.

Performance summary

Beyond comprehension inlining and f-string improvements:

• asyncio 5-10% faster due to internal restructuring
• Startup time ~15% faster — fewer modules imported at startup
• Memory reduction — wstr removal saves 8 bytes per string object

Migration strategy

1. Search for distutils imports — replace immediately, this is a hard removal
2. Adopt type statement for new type aliases; no rush for existing TypeAlias
3. Simplify f-strings — remove workarounds for backslash and nesting limitations
4. Test with --check-multi-interp-extensions — prepare for per-interpreter isolation
5. Pin setuptools >= 67 in build requirements for projects that still use it

The one thing to remember: Python 3.12 was where CPython’s architecture pivoted — per-interpreter GILs proved the concept, and the cleaner type syntax signalled that Python’s type system had graduated from bolted-on library to first-class language feature.