Differential Privacy in Python — Core Concepts

The formal guarantee

Differential privacy provides a mathematical promise: whether or not any single person’s data is included in the dataset, the output of the analysis looks essentially the same. An attacker observing the results can’t confidently determine if any specific individual participated.

More precisely, a mechanism M is ε-differentially private if for any two datasets D1 and D2 that differ in exactly one record, and for any possible output S:

P(M(D1) ∈ S) ≤ e^ε × P(M(D2) ∈ S)

This means adding or removing one person’s data changes the probability of any outcome by at most a factor of e^ε.

Understanding epsilon (ε)

Epsilon is the privacy budget — it quantifies how much privacy you’re spending. A smaller epsilon means stronger privacy but noisier results.

Practical epsilon ranges vary by context:

• ε < 1: Strong privacy. Apple uses ε values between 1 and 8 for different data types.
• ε = 1 to 3: Moderate privacy. Good balance for many analytics use cases.
• ε > 10: Weak privacy. The noise is so small that individual records may be distinguishable.

There’s no universally “correct” epsilon. It depends on the sensitivity of the data, the size of the dataset, the potential harm of disclosure, and your organization’s risk tolerance.

Noise mechanisms

Two noise distributions are used in most differential privacy implementations:

Laplace mechanism: Adds noise drawn from a Laplace distribution. Used for numerical queries (counts, sums, averages). The noise scale is calibrated as sensitivity/ε, where sensitivity is the maximum impact one person’s data can have on the query result. For a counting query, sensitivity is 1 (one person changes the count by at most 1).

Gaussian mechanism: Adds noise from a Gaussian (normal) distribution. Provides (ε, δ)-differential privacy, a slightly relaxed guarantee where δ represents a small probability of exceeding the privacy bound. Often preferred because Gaussian noise is mathematically convenient when composing multiple queries.

Sensitivity

Sensitivity measures how much a single person can influence a query result. It determines how much noise is needed.

Counting queries (how many users visited today?) have sensitivity 1. One person being present or absent changes the count by exactly 1.

Sum queries (total revenue from users) have sensitivity equal to the maximum possible contribution — if users can spend up to $1000, sensitivity is 1000. Higher sensitivity requires more noise.

Average queries are trickier because they combine a sum and a count. A common approach is to compute the noisy sum and noisy count separately, then divide.

Composition: the budget runs out

Every time you run a differentially private query, you spend some of your privacy budget. After many queries, the total privacy guarantee degrades. This is called composition.

Sequential composition: Running k queries with budgets ε₁, ε₂, …, εₖ on the same dataset yields a total privacy cost of ε₁ + ε₂ + … + εₖ. Privacy costs add up linearly.

Advanced composition: More sophisticated analysis shows that privacy loss grows proportionally to the square root of the number of queries rather than linearly. This allows more queries for the same total budget.

Parallel composition: Queries on disjoint subsets of data don’t compound. If you query the count of users in each state, and each user is in exactly one state, the total privacy cost is max(ε₁, ε₂, …) rather than the sum.

Budget tracking is crucial. Once your epsilon budget is exhausted, you can’t run more queries without degrading the privacy guarantee.

Local vs. global differential privacy

Global (central) model: A trusted curator holds the raw data and adds noise to query results before releasing them. This provides better accuracy because noise is added once to aggregated results. Used by organizations that must hold data centrally (census bureaus, hospitals).

Local model: Each individual adds noise to their own data before sending it to the collector. The collector never sees raw data. Less accurate (more noise needed) but doesn’t require trusting the data collector. This is what Apple and Google use — your device adds noise before the data leaves your phone.

Common misconception: differential privacy makes data useless

With a large enough dataset, differential privacy produces surprisingly accurate results. Adding Laplace noise with scale 1/ε to a count of 10 million introduces noise that’s typically less than 0.0001% of the true value. The technique struggles with small datasets or highly specific queries, but for the aggregate analytics most organizations need, it works remarkably well.

The one thing to remember: Differential privacy provides a mathematical guarantee — controlled by the epsilon parameter — that analysis results look nearly identical whether any single person’s data is included or not, enabling useful statistics without individual exposure.