Bias-Variance Tradeoff — Explain Like I'm 5

The Archer Who Always Misses Left

Imagine two archers.

Archer A always shoots to the left. Every single arrow — 5 centimeters left of center. It’s consistent, predictable, wrong in the same direction every time. This is bias — a systematic error, always in the same direction.

Archer B is different. Their arrows land all over the place — sometimes left, sometimes right, sometimes high, sometimes low. No pattern, just randomness. This is variance — inconsistent, scattered.

In machine learning, your model is the archer. Getting predictions right means minimizing both types of error.

Two Ways to Get Things Wrong

High bias (underfitting): The model is too simple. It doesn’t capture the real patterns in the data. Imagine using a single straight line to fit data that clearly curves. The line consistently misses in systematic ways.

High variance (overfitting): The model is too complex. It memorizes the training data, including the random noise. When shown new data, it fails because the noise doesn’t repeat. A very wiggly curve that passes through every single training point would fit training data perfectly but fail on new data.

The Tension

Making a model more complex usually:

• Reduces bias (it can capture more real patterns)
• Increases variance (it starts memorizing noise)

Making a model simpler usually:

• Increases bias (it can’t capture complex patterns)
• Reduces variance (it doesn’t memorize noise)

This is the tradeoff. Finding the right complexity — complex enough to capture real patterns, simple enough not to memorize noise — is one of machine learning’s most fundamental challenges.

The Good News

Modern machine learning (deep neural networks) has somewhat broken this classical tradeoff for large models. Very large neural networks trained on enormous datasets seem to get both lower bias AND lower variance through a phenomenon called “double descent” — they’re so large that they effectively don’t memorize in the harmful way. But the underlying tension still appears in smaller models and limited data settings.

One thing to remember: Bias is being consistently wrong; variance is being inconsistently wrong. Good models minimize both — by being complex enough to see real patterns but simple enough not to chase noise.