Python Memory-Mapped Files — ELI5

Imagine you need to look something up in a huge encyclopedia — 30 volumes sitting on a library shelf. You have two choices:

Option A: Carry all 30 volumes home, stack them on your desk, then search through them. That’s a lot of heavy lifting, and your desk might not be big enough.

Option B: Go to the library, open just the volume you need, read the page, and put it back. You never carry more than one volume at a time.

Memory-mapped files work like Option B. When Python “memory-maps” a file, it doesn’t actually load the whole file into your computer’s memory (RAM). Instead, it creates a window into the file. When your program reads a particular section, the operating system fetches just that section from disk.

This is incredibly useful for big files. Imagine a 10-gigabyte log file. Loading it all into memory would need 10 GB of RAM — which many computers don’t have free. But with memory mapping, Python can work with that file as if it were fully loaded, while the operating system quietly manages which parts are actually in memory at any moment.

The neat trick is that your code looks almost identical to regular file handling. You can slice into the file, search for patterns, and jump to any position — all without worrying about how much memory you’re using.

The one thing to remember: Memory-mapped files let Python work with files much larger than your available memory by only loading the parts you actually touch — like reading a book at the library instead of hauling the entire collection home.