Python Template Strings — Deep Dive

Python’s string.Template class is intentionally minimal, but its design decisions reveal important lessons about balancing power with safety. This deep dive examines the implementation, extension points, security properties, and practical patterns for production use.

Implementation Under the Hood

Template is a class in the string module. Its core mechanism is a compiled regular expression stored as a class attribute called pattern.

from string import Template
import re

# The default pattern (simplified view)
print(Template.pattern.pattern)

The default regex matches four alternatives, tried in order:

1. Escaped: $$ → literal $
2. Named: $identifier → lookup by name
3. Braced: ${identifier} → lookup by name (with braces)
4. Invalid: bare $ followed by something unexpected → error

# The actual regex groups
for m in Template.pattern.finditer("$name said $$5 for ${item}!"):
    print(m.groupdict())
# {'named': 'name', 'braced': None, 'escaped': None, 'invalid': None}
# {'named': None, 'braced': None, 'escaped': '$', 'invalid': None}
# {'named': None, 'braced': 'item', 'escaped': None, 'invalid': None}

The substitute Method

substitute() uses re.sub() with a callback function. For each match, it:

1. Checks if it’s an escaped $$ → returns literal $
2. Checks named or braced group → looks up value in the mapping
3. If invalid → raises ValueError
4. If key not found → raises KeyError

# Equivalent pseudo-implementation
def _substitute(self, mapping):
    def convert(mo):
        named = mo.group('named') or mo.group('braced')
        if named is not None:
            return str(mapping[named])
        if mo.group('escaped') is not None:
            return self.delimiter
        raise ValueError(f"Unrecognized group in pattern")
    return self.pattern.sub(convert, self.template)

Custom Template Classes

Subclassing Template lets you override three class attributes:

Custom Delimiter

from string import Template

class HashTemplate(Template):
    delimiter = '#'

t = HashTemplate("Hello #name, balance: $500")
t.substitute(name="Alice")
# "Hello Alice, balance: $500"

Dollar signs become normal characters when you change the delimiter.

Custom Identifier Pattern

The idpattern attribute controls what constitutes a valid identifier:

class DottedTemplate(Template):
    idpattern = r'[a-z][a-z0-9]*(?:\.[a-z][a-z0-9]*)*'

t = DottedTemplate("Server: $server.host, Port: $server.port")
t.substitute({"server.host": "localhost", "server.port": "8080"})
# "Server: localhost, Port: 8080"

This enables dotted keys common in configuration files.

Fully Custom Pattern

For maximum control, override the pattern attribute directly:

class AngleTemplate(Template):
    delimiter = '<'
    pattern = re.compile(r"""
        <(?:
            (?P<escaped><)               |  # << for literal <
            (?P<named>[a-zA-Z_]\w*)>     |  # <name>
            \{(?P<braced>[a-zA-Z_]\w*)}>  |  # <{name}>
            (?P<invalid>)                    # anything else
        )
    """, re.VERBOSE)

t = AngleTemplate("Hello <name>, use << for literal")
t.substitute(name="World")
# "Hello World, use < for literal"

Security Analysis

Why .format() Is Dangerous with Untrusted Input

Python’s .format() allows attribute access and indexing:

# An attacker can traverse object attributes
class User:
    def __init__(self, name):
        self.name = name

user = User("Alice")

# Innocent looking:
"{0.name}".format(user)  # "Alice"

# Malicious:
"{0.__class__.__init__.__globals__}".format(user)
# Dumps global variables including imported modules!

This is a real vulnerability (CVE-2019-8341 in Jinja2, similar patterns in Django). The attacker crafts a format string that traverses the object graph to access sensitive data.

Why Template Strings Are Safe

Template substitution only performs dictionary key lookup. There is no:

• Attribute access (.attr)
• Index access ([0])
• Method calls (())
• Expression evaluation
• Type conversion

The replacement value is always str(mapping[key]) — a flat key lookup followed by string conversion.

# Attacker's template string
t = Template("$user.__class__.__init__.__globals__")
try:
    t.substitute(user="Alice")
except KeyError:
    # Raises KeyError for 'user' — the dots are not part of the key
    # (unless you customized idpattern to allow dots)
    pass

Security Checklist for Template Usage

1. Use Template (not .format()) when template source is untrusted
2. Validate placeholder names if using custom idpattern
3. Use safe_substitute() if missing keys should not crash
4. Sanitize substitution values if they’ll be rendered in HTML
5. Never use eval() or exec() on template output

Production Patterns

Email Templates from Database

from string import Template

def render_email(template_text: str, context: dict) -> str:
    """Safely render an email template stored in the database."""
    t = Template(template_text)
    # safe_substitute won't crash on missing keys
    return t.safe_substitute(context)

# Template stored by admin in database:
# "Dear $customer_name, your order #$order_id ships on $ship_date."

Multi-Pass Templating

When templates reference other templates or need staged processing:

from string import Template

def multi_pass_render(template_text: str, *contexts) -> str:
    """Apply substitution contexts in order, leaving unknown placeholders."""
    result = template_text
    for ctx in contexts:
        result = Template(result).safe_substitute(ctx)
    return result

# Pass 1: system variables
# Pass 2: user variables
result = multi_pass_render(
    "Hello $name, server: $host:$port",
    {"host": "prod.example.com", "port": "443"},  # System
    {"name": "Alice"},  # User
)
# "Hello Alice, server: prod.example.com:443"

Configuration File Processing

from string import Template
from pathlib import Path
import os

class EnvTemplate(Template):
    """Template that resolves from environment variables."""
    idpattern = r'[A-Z][A-Z0-9_]*'

def load_config(path: str) -> str:
    raw = Path(path).read_text()
    return EnvTemplate(raw).safe_substitute(os.environ)

# config.ini content:
# database_url = postgresql://$DB_USER:$DB_PASS@$DB_HOST/mydb

Internationalization (i18n) Templates

Template strings work well for localization where translators provide message patterns:

translations = {
    "en": Template("Welcome, $name! You have $count items."),
    "es": Template("Bienvenido, $name! Tienes $count artículos."),
    "de": Template("Willkommen, $name! Du hast $count Artikel."),
}

def localize(lang: str, **kwargs) -> str:
    return translations.get(lang, translations["en"]).substitute(kwargs)

Performance Considerations

Template strings are the slowest Python formatting method because they:

1. Compile a regex pattern (cached per class, not per instance)
2. Run re.sub() with a Python callback for each placeholder
3. Call str() on each value

For a template with 5 placeholders on a short string, rough relative timings:

Method	Relative Speed
f-string	1x (baseline)
.format()	1.5-2x slower
%-formatting	1.2-1.5x slower
Template.substitute()	5-10x slower

The overhead is negligible for email generation, config loading, or any I/O-bound task. It only matters in tight inner loops processing millions of strings — which is not the typical use case for Template.

Comparison with Jinja2

For complex templating needs (loops, conditionals, inheritance), consider Jinja2:

from jinja2 import SandboxedEnvironment

env = SandboxedEnvironment()  # Restricts dangerous operations
template = env.from_string("Hello {{ name }}!")
template.render(name="Alice")

Jinja2’s SandboxedEnvironment provides more features than string.Template while still blocking dangerous attribute access. Choose based on complexity:

• Simple substitution → string.Template
• Loops, conditionals, filters → Jinja2 with sandbox
• Full logic in templates → Jinja2 (trusted templates only)

One Thing to Remember

string.Template is deliberately underpowered — that’s the feature. When templates come from users, databases, or config files, the inability to evaluate expressions is not a limitation but a security boundary that .format() and f-strings cannot provide.