Custom Import Hooks — Core Concepts

How Python Imports Work

When you write import mymodule, Python follows a well-defined search protocol:

1. Check sys.modules — if the module was already imported, return the cached version
2. Search sys.meta_path — ask each finder on this list if it can locate the module
3. Search sys.path_hooks — for path-based imports, check if a path hook can handle the directory
4. Raise ModuleNotFoundError if nothing works

The two extension points — sys.meta_path and sys.path_hooks — are where custom import hooks plug in.

Finders and Loaders

Python’s import system is built on two abstractions:

Finders answer the question “can you find this module?” They implement a find_module() or find_spec() method that returns module metadata (a ModuleSpec) or None.

Loaders answer “can you load this module?” They implement create_module() and exec_module() to actually build the module object and execute its code.

Modern Python (3.4+) combines these into importers that handle both finding and loading.

Meta Path Hooks

A meta path hook is an object placed on sys.meta_path. It gets consulted for every import before Python searches the file system:

import sys
import importlib.abc
import importlib.util
import types

class DatabaseFinder(importlib.abc.MetaPathFinder):
    def __init__(self, db_connection):
        self.db = db_connection

    def find_spec(self, fullname, path, target=None):
        # Check if module exists in database
        row = self.db.get_module(fullname)
        if row:
            loader = DatabaseLoader(row['source'])
            return importlib.util.spec_from_loader(fullname, loader)
        return None  # let other finders try

class DatabaseLoader(importlib.abc.Loader):
    def __init__(self, source_code):
        self.source = source_code

    def create_module(self, spec):
        return None  # use default module creation

    def exec_module(self, module):
        exec(compile(self.source, module.__name__, 'exec'), module.__dict__)

# Install the hook
sys.meta_path.insert(0, DatabaseFinder(my_database))

# Now this works if "my_plugin" exists in the database
import my_plugin

The find_spec method returns a ModuleSpec object that tells Python the module’s name, loader, and origin. Returning None passes the search to the next finder.

Path Hooks

Path hooks are different — they handle specific entries on sys.path. When Python encounters a directory on sys.path, it checks if any path hook can handle that path:

class ZipImporter:
    def __init__(self, path):
        if not path.endswith('.myzip'):
            raise ImportError  # decline this path
        self.archive = open_archive(path)

    def find_spec(self, fullname, target=None):
        if fullname in self.archive:
            loader = self  # this object is also the loader
            return importlib.util.spec_from_loader(fullname, loader)
        return None

    def exec_module(self, module):
        source = self.archive.read(module.__name__)
        exec(compile(source, module.__name__, 'exec'), module.__dict__)

sys.path_hooks.append(ZipImporter)
sys.path.append('/path/to/plugins.myzip')

import some_plugin  # loaded from the archive

Python’s built-in zipimporter works exactly this way — it is how Python can import from .zip files and .egg files.

Common Use Cases

Plugin systems: Discover and load plugins from a registry, database, or network location without users needing to install packages.

Hot reloading: Watch for file changes and reload modules during development. Tools like importlib.reload() handle the mechanics, but custom hooks can trigger reloads automatically.

Source transformation: Load non-Python files (YAML configs, custom DSLs) as if they were Python modules by transforming them during the load step.

Security sandboxing: Control which modules can be imported by filtering through a custom finder.

Common Misconception

Developers often confuse importlib.reload() with import hooks. reload() re-executes an existing module’s code but does not change how modules are found or loaded. Import hooks modify the finding and loading process itself — they determine where code comes from and how it gets executed.

The Import Order

When multiple hooks are installed, order matters:

1. sys.modules cache (always checked first)
2. sys.meta_path finders (checked in order — first match wins)
3. sys.path entries, each checked against sys.path_hooks

Inserting your hook at position 0 in sys.meta_path gives it priority over all built-in finders. Appending it makes it a fallback.

One thing to remember: Python’s import system is a chain of finders and loaders — custom hooks let you intercept this chain to load code from any source, in any format, with any transformation you need.