N+1 Query Problem — ELI5
Imagine you’re ordering pizza for a party with 20 friends. You call the pizza place and ask, “What pizzas do you have?” They tell you 20 options. Great.
Now, for each pizza, you call them again and ask, “What toppings are on pizza number 1?” Then you hang up. Then you call back: “What toppings on pizza number 2?” Hang up. Call back. “Pizza number 3?”
That’s 21 phone calls for something you could have asked in one: “Tell me all the pizzas and their toppings.”
This is the N+1 query problem. Your program makes 1 call to get a list of things (the “1”), then makes N additional calls — one for each thing — to get the details. If you have 20 items, that’s 21 trips to the database. If you have 1,000 items, that’s 1,001 trips.
Each trip to the database takes time — not just for the work, but for the travel back and forth (like dialing the phone, waiting for an answer, and hanging up). Those round trips add up fast. A page that should load in 50 milliseconds suddenly takes 3 seconds because it’s making 500 tiny database trips instead of 2 big ones.
The sneaky part is that your code often looks perfectly fine. You wrote a loop that goes through items and asks for related data on each one. It’s clean, readable code. But under the hood, it’s making the pizza place very annoyed.
The fix is to ask for everything you need upfront: “Give me all the pizzas with their toppings in one go.”
One thing to remember: The N+1 problem happens when your code fetches a list, then loops through it and makes a separate database call for each item. The fix is always: fetch the related data together, not one at a time.
See Also
- Python Aioredis Understand Aioredis through a practical analogy so your Python decisions become faster and clearer.
- Python Alembic Understand Alembic through a practical analogy so your Python decisions become faster and clearer.
- Python Asyncpg Database asyncpg is the fastest way for Python to talk to PostgreSQL without making your program sit around waiting.
- Python Asyncpg Understand Asyncpg through a practical analogy so your Python decisions become faster and clearer.
- Python Cassandra Python Understand Cassandra Python through a practical analogy so your Python decisions become faster and clearer.