Test Coverage Strategies — Core Concepts

What coverage actually measures

Coverage tools instrument your code, tracking which lines execute when tests run. The standard tool in Python is coverage.py, typically paired with pytest via the pytest-cov plugin.

There are several coverage metrics, each revealing different things:

• Line coverage — Did this line execute? The simplest metric.
• Branch coverage — Did every if/else branch get taken? Catches untested conditional paths.
• Path coverage — Were all possible execution paths through a function exercised? Exponentially harder but far more thorough.

Most teams track line and branch coverage. Path coverage is usually impractical for large codebases.

How it works in practice

Running pytest --cov=mypackage --cov-branch generates a report showing which lines and branches were hit. The output looks like:

Name                   Stmts   Miss Branch BrPart  Cover
mypackage/core.py         80      5     20      3    91%
mypackage/utils.py        45      0     10      0   100%

BrPart stands for partial branches — places where only one side of a conditional was tested. These are the most dangerous gaps because they mean an entire decision path was never exercised.

The coverage trap

A common misconception: high coverage equals high quality. In reality, coverage is a necessary but insufficient signal. Consider a function that divides two numbers. A test that only passes (10, 2) achieves full line coverage but never checks division by zero.

Teams that mandate 100% coverage often end up with tests like assert True or tests that call a function without checking the result — just to light up the coverage report. This is worse than lower coverage because it creates false confidence.

Strategies that work

1. Coverage ratchet — Never let coverage drop below the current level. Each pull request must maintain or increase coverage. This prevents regression without demanding perfection upfront.

2. Critical-path-first — Identify the code that handles money, authentication, data mutations, or external API calls. Cover those thoroughly (branch + edge cases) before worrying about utility helpers.

3. Mutation-aware coverage — Combine coverage with mutation testing. Mutation testing changes your code slightly and checks if tests catch the change. High coverage with low mutation score means your tests are superficial.

4. Diff coverage — Only enforce coverage on changed lines. Tools like diff-cover compare your coverage report against a git diff, ensuring new code comes with tests while leaving legacy code alone.

What good coverage reports tell you

The most valuable insight is not the number — it’s the uncovered lines. When you see a gap, ask: “Is this an error handler? A rare edge case? Dead code?” Each answer leads to a different action: write a test, remove the dead code, or accept the gap with a documented reason.

Coverage trends over time matter more than snapshots. A project that holds 85% consistently is healthier than one that swings between 70% and 95% depending on who’s committing.

The one thing to remember: Use branch coverage to find untested decisions, enforce a ratchet to prevent backsliding, and never chase the number at the expense of test quality.