Python Slowly Changing Dimensions — ELI5
Imagine your school keeps a file for every student. Your file says your name, your grade, and your home address. One day, your family moves to a new house.
Now the school has a choice:
- Erase the old address and write the new one. Simple, but now nobody remembers where you used to live.
- Keep the old address and add the new one below it, with dates. More work, but now the school knows you lived at Oak Street from 2022 to 2025 and Maple Avenue from 2025 onward.
- Add a new column called “previous address.” Quick, but only works if you move once—what about a second move?
These three choices are exactly what data engineers face when facts about a person, product, or company change over time. The fancy name is “slowly changing dimensions” because these details do not change every second—they change slowly, like an address or a job title.
Python helps teams handle this by reading the current records, comparing them to new information, and deciding which strategy to use. It can automatically stamp dates on old records and insert new ones so the history is preserved.
Why does history matter? Because a report about last year’s sales needs last year’s addresses, not today’s. If you just overwrite, your historical reports become wrong.
One thing to remember: slowly changing dimensions are just ways to decide whether to overwrite old facts, keep both old and new with dates, or add a column for the previous value.
See Also
- Python Adaptive Learning Systems How Python builds learning apps that adjust to each student like a personal tutor who knows exactly what you need next.
- Python Airflow Learn Airflow as a timetable manager that makes sure data tasks run in the right order every day.
- Python Altair Learn Altair through the idea of drawing charts by describing rules, not by hand-placing every visual element.
- Python Automated Grading How Python grades homework and exams automatically, from simple answer keys to understanding written essays.
- Python Batch Vs Stream Processing Batch processing is like doing laundry once a week; stream processing is like a self-cleaning shirt that cleans itself constantly.