Performance Testing Patterns — Core Concepts

The performance testing spectrum

Performance testing isn’t one thing — it’s a spectrum of techniques that answer different questions:

Benchmarking measures how fast a specific function or code path runs. This is micro-level: “How many milliseconds does this sorting function take for 10,000 items?”

Load testing simulates realistic user traffic against your full application. This is system-level: “Can our API handle 500 concurrent users with sub-second response times?”

Profiling identifies where time is spent within your code. This is diagnostic: “Which function consumes 80% of the request processing time?”

Each technique has its place. Benchmarking catches performance regressions in critical algorithms. Load testing validates your infrastructure can handle expected traffic. Profiling guides optimization when something is too slow.

Benchmarking with pytest-benchmark

pytest-benchmark integrates directly into your test suite, making performance measurement part of your regular test workflow:

def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

def test_fibonacci_performance(benchmark):
    result = benchmark(fibonacci, 20)
    assert result == 6765

The plugin runs your function many times, collects statistics (min, max, mean, standard deviation), and detects regressions by comparing against saved baselines. When a pull request makes a function 20% slower, the benchmark fails.

Load testing with Locust

Locust is Python’s most popular load testing framework. You write user behavior as Python classes:

from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
    wait_time = between(1, 3)

    @task(3)
    def browse_products(self):
        self.client.get("/api/products")

    @task(1)
    def place_order(self):
        self.client.post("/api/orders", json={
            "product_id": 42,
            "quantity": 1,
        })

The @task decorator weights indicate relative frequency — this user browses products three times for every order placed. The wait_time simulates realistic pauses between actions.

Locust provides a web dashboard showing requests per second, response times (median and percentiles), and failure rates in real-time. You can scale to thousands of simulated users across multiple worker machines.

Key metrics to track

Not all metrics matter equally:

• P50 (median) response time — the typical user experience
• P95 and P99 response time — the experience for your slowest users (often where problems hide)
• Throughput — requests per second your system handles
• Error rate — percentage of failed requests under load
• Saturation — CPU, memory, database connection pool usage

A system with great average response times but terrible P99 is hiding a problem. The 1% of users hitting the slow path will complain loudly.

Common misconception

Many teams only test performance before launches. This catches obvious issues but misses gradual degradation. A feature that adds 5ms per request seems harmless, but after 20 such features, your response time has doubled. Continuous performance testing — running benchmarks in CI on every merge — catches death-by-a-thousand-cuts performance regressions.

Anti-patterns to avoid

Testing on developer laptops — Your laptop has different hardware, no network latency, and no competing processes. Performance numbers from local testing are unreliable. Test in an environment that matches production.

Ignoring warmup — Python applications (especially those using JIT compilation with PyPy, or connection pooling) behave differently on first requests versus warmed-up requests. Always include a warmup phase before measuring.

Testing without realistic data — A database with 100 rows performs very differently from one with 10 million rows. Use production-scale data or realistic synthetic data.

The one thing to remember: Performance testing combines benchmarking for code-level speed, load testing for system-level capacity, and profiling for diagnosing bottlenecks — do all three, continuously.