Python PEX Executables — Deep Dive

PEX bootstrap internals

When you execute a .pex file, Python’s zipimport mechanism kicks in, but PEX adds several layers on top:

1. Shebang dispatch — the OS reads #!/usr/bin/env python3 and launches the appropriate interpreter.
2. __main__.py bootstrap — Python executes the archive’s __main__.py, which is PEX’s bootstrap code.
3. Interpreter validation — the bootstrap checks the running Python version against constraints baked into the PEX at build time.
4. Dependency unpacking — compiled wheels (.so/.pyd files) cannot be imported directly from a zip. PEX extracts them to a cache directory (~/.pex/ by default).
5. sys.path injection — pure-Python packages are loaded from the zip; compiled packages point to the cache.
6. Entry point execution — your main() function runs.

This sequence typically takes 50-200ms on a warm cache (compiled dependencies already extracted) and 1-5 seconds on a cold cache for large dependency sets.

# Inspect PEX internals at runtime
import sys
print(sys.path)           # Shows zip paths and cache paths
print(sys._MEIPASS)       # Not set — that's PyInstaller
print(__file__)           # Points to the .pex file itself

The PEX cache

PEX maintains a local cache at ~/.pex/ with this structure:

~/.pex/
├── installed_wheels/     # Extracted compiled wheels
│   └── numpy-1.26.4-cp311-cp311-linux_x86_64/
├── packed_wheels/        # Downloaded wheel files
└── interpreters/         # Cached interpreter metadata

Cache management matters in production:

# Set a custom cache location
PEX_ROOT=/opt/pex-cache ./myapp.pex

# Disable caching entirely (slower, but no disk writes)
PEX_ROOT=/dev/null ./myapp.pex

# Pre-warm the cache during deployment
./myapp.pex --version  # triggers extraction without running the app

In containerized environments, mount PEX_ROOT as a volume to persist the cache across container restarts.

Building multi-platform PEX files

For heterogeneous server fleets, you need PEX files that work across multiple platforms:

# Build for multiple platforms
pex numpy==1.26.4 \
    --platform linux_x86_64-cp-3.11-cp311 \
    --platform linux_aarch64-cp-3.11-cp311 \
    --platform macosx_11_0_arm64-cp-3.11-cp311 \
    -e myapp:main \
    -o myapp.pex

PEX downloads platform-specific wheels for each target and bundles them all. At runtime, the bootstrap selects the correct wheel for the current platform. This creates larger files but enables truly portable deployments.

For this to work, pre-built wheels must exist for every target platform. If a package only distributes source distributions, you need to cross-compile or build on each platform separately.

Pants build system integration

PEX was created alongside the Pants build system, and they integrate tightly:

# BUILD file in a Pants project
python_sources(name="lib", sources=["**/*.py"])

pex_binary(
    name="myapp",
    entry_point="myapp/main.py:main",
    dependencies=[":lib"],
    execution_mode="venv",       # Modern venv-based execution
    include_tools=True,          # Include pip/setuptools in the PEX
    resolve="default",
)

Pants handles dependency resolution, lockfile generation, and multi-platform builds automatically:

# Build the PEX
pants package src/myapp:myapp

# Output: dist/src.myapp/myapp.pex

The Pants + PEX combination gives you reproducible builds with content-based caching — Pants only rebuilds the PEX when source code or dependencies actually change.

Execution modes

PEX supports three execution modes that affect performance and compatibility:

Zipapp mode (default)

Imports pure-Python modules directly from the zip. Compiled extensions are extracted to the PEX cache. Fastest startup for pure-Python apps.

Venv mode

pex --venv requests flask -e myapp:main -o myapp.pex

At first run, PEX creates a full virtual environment from the bundled wheels. Subsequent runs reuse the venv. This mode is most compatible — some packages (looking at you, pkg_resources) struggle with zip imports.

Unzip mode

pex --unzip requests flask -e myapp:main -o myapp.pex

Fully extracts the PEX to disk on first run. Maximum compatibility at the cost of first-run latency and disk space.

Interpreter constraints

Lock down which Python versions can run your PEX:

pex --python-shebang='#!/usr/bin/env python3.11' \
    --interpreter-constraint='CPython>=3.11,<3.13' \
    -r requirements.txt \
    -e myapp:main \
    -o myapp.pex

The interpreter constraint is checked at startup. If the system Python does not match, PEX exits with a clear error instead of crashing mid-execution with import failures.

Production deployment patterns

Pattern 1: CI/CD artifact

# GitHub Actions
- name: Build PEX
  run: |
    pip install pex
    pex -r requirements.lock -e myapp:main -o myapp.pex
    
- name: Upload artifact
  uses: actions/upload-artifact@v4
  with:
    name: myapp-${{ github.sha }}
    path: myapp.pex

Pattern 2: Docker with PEX

FROM python:3.11-slim
COPY myapp.pex /app/myapp.pex
ENV PEX_ROOT=/app/.pex-cache
RUN /app/myapp.pex --version  # Pre-warm cache
CMD ["/app/myapp.pex"]

Using PEX inside Docker may seem redundant, but it eliminates pip install during image builds, making layers thinner and builds faster.

Pattern 3: Fleet deployment with rsync

# Build once, distribute everywhere
pex -r requirements.lock -e myapp:main -o myapp.pex
chmod +x myapp.pex

# Deploy to 100 servers
for host in $(cat servers.txt); do
    rsync -az myapp.pex "$host:/opt/myapp/myapp.pex"
    ssh "$host" 'systemctl restart myapp'
done

Size optimization

PEX files can grow large with heavy dependencies:

# Check what's inside
unzip -l myapp.pex | tail -20

# Exclude unnecessary packages
pex -r requirements.txt \
    --exclude boto3 \
    --exclude botocore \
    -e myapp:main -o myapp.pex

# Use --strip-pex-env to reduce bootstrap overhead
pex --strip-pex-env -r requirements.txt -e myapp:main -o myapp.pex

For the most aggressive size reduction, audit your requirements and split large applications into multiple PEX files — one per service or CLI tool.

Debugging PEX issues

# Verbose mode — see what PEX is doing
PEX_VERBOSE=3 ./myapp.pex

# Drop into the PEX environment interactively
PEX_INTERPRETER=1 ./myapp.pex
>>> import sys; print(sys.path)
>>> import numpy; print(numpy.__file__)

# Check what's bundled
unzip -l myapp.pex | grep '.whl'

The PEX_INTERPRETER=1 trick is invaluable for debugging import errors — it gives you a Python shell with the exact same environment your app runs in.

PEX vs Shiv comparison

Both PEX and Shiv produce Python zip applications, but they differ in philosophy:

Aspect	PEX	Shiv
Origin	Twitter/Pants ecosystem	LinkedIn
Resolution	Built-in resolver + pip	pip only
Multi-platform	Yes (bundled wheels)	No (build platform only)
Execution model	Zip + cache	Extract to site-packages
Startup latency	Lower for pure-Python	Lower for compiled deps
Complexity	Higher (more features)	Lower (simpler model)

Choose PEX for complex, multi-platform server deployments. Choose Shiv for simpler use cases where you want zipapp packaging with minimal configuration.

One thing to remember: PEX’s power lies in its bootstrap layer — understanding the cache, execution modes, and interpreter constraints turns it from a simple packaging tool into a reliable deployment mechanism for production Python at any scale.