Python Dependency Management Strategies — Core Concepts

The Dependency Problem

Every non-trivial Python project depends on external libraries, and those libraries depend on others. Managing this tree of transitive dependencies — ensuring compatible versions, reproducible installs, and security updates — is what dependency management solves.

Core Workflow

Regardless of the tool, the workflow follows three steps:

1. Declare — list direct dependencies with flexible version constraints (e.g., requests>=2.28,<3)
2. Resolve — the tool computes compatible versions for all transitive dependencies
3. Lock — save exact resolved versions to a lock file that guarantees reproducibility

Tool Comparison

pip + requirements.txt

The original approach. You write requirements.txt by hand or generate it with pip freeze. It works but has gaps: no automatic resolution of conflicts, no separation between direct and transitive dependencies, and pip freeze captures your entire environment including dev tools.

Best for: Quick scripts, tutorials, environments where adding new tools is restricted.

pip-tools

A thin layer over pip. You write requirements.in with direct dependencies and run pip-compile to generate a fully pinned requirements.txt. It solves the resolution gap while staying close to pip.

Best for: Teams that want reproducibility without learning a new ecosystem.

Poetry

An all-in-one tool: dependency resolution, virtual environment management, building, and publishing. Uses pyproject.toml for declarations and poetry.lock for pins. Its resolver is thorough but historically slower than newer alternatives.

Best for: Library authors who publish to PyPI and want one tool for the full lifecycle.

PDM

Similar to Poetry but follows PEP standards more strictly. Uses the official pyproject.toml metadata format and supports PEP 582 (local package directories without virtualenvs). Its resolver is fast and standards-compliant.

Best for: Teams that prioritize PEP compliance and want Poetry-like ergonomics.

uv

Built in Rust by Astral, uv is a drop-in pip replacement that also handles resolution and locking. It is 10-100x faster than pip for installs and resolves. Supports workspaces for monorepos and can replace pip, pip-tools, and virtualenv in one binary.

Best for: Teams that want the fastest possible installs and a modern, unified tool.

Version Constraints

Writing good constraints prevents most dependency headaches:

• >=2.0,<3 — compatible with any 2.x release (common for libraries following semver)
• ~=2.5 — equivalent to >=2.5,<3.0 (PEP 440 compatible release)
• ==2.5.3 — exact pin in the constraint file (usually only in lock files)

Avoid pinning exact versions in pyproject.toml for libraries — it forces consumers into conflicts. Pin exact versions only in lock files and application deployments.

Lock Files: The Non-Negotiable

A lock file records every package, every version, and every hash. It answers the question: “What exact set of packages made our tests pass?”

Without a lock file, pip install -r requirements.txt can resolve to different versions on different days, because new releases appear on PyPI constantly. This makes debugging nightmarish — “it worked yesterday” has no forensic trail.

Always commit lock files for applications. For libraries, lock files are optional (they pin development dependencies but should not restrict consumers).

Handling Conflicts

When two dependencies require incompatible versions of a shared sub-dependency, the resolver fails. Strategies:

• Upgrade the offending package — often the conflict exists only in older versions
• Find an alternative — a different library may have looser requirements
• Use dependency overrides — Poetry and uv allow forcing a version, at your own risk

Common Misconception

“Pinning dependencies makes your project more secure.” Pinning prevents unexpected upgrades but also prevents automatic security patches. The real security strategy is pinning in a lock file and running automated dependency update tools like Dependabot or Renovate that create PRs for new versions.

Practical Checklist

• Declare dependencies in pyproject.toml, not scattered text files
• Generate and commit a lock file
• Use separate dependency groups for dev, test, and docs
• Run pip audit or safety check in CI to catch known vulnerabilities
• Automate version bump PRs with Dependabot or Renovate

The one thing to remember: Declare loosely in project metadata, lock tightly for deployments, and automate updates — that trio prevents most dependency disasters.