Python Reader-Writer Locks — Core Concepts
The problem reader-writer locks solve
A standard threading.Lock allows only one thread at a time, regardless of whether the thread is reading or writing. For a configuration cache read by 100 threads and updated once a minute, this means 99 threads are blocked needlessly during concurrent reads.
Reader-writer locks (RWLocks) distinguish between shared access (reading) and exclusive access (writing):
- Multiple readers can hold the lock simultaneously
- A writer gets exclusive access — no readers or other writers allowed
- Writers wait for all current readers to finish before proceeding
Python’s standard library gap
Python doesn’t include a reader-writer lock in the threading module. You can build one from existing primitives or use third-party packages like readerwriterlock from PyPI.
Reader-preference vs writer-preference
Reader-preference: if readers keep arriving, a waiting writer may never get access (writer starvation).
Writer-preference: once a writer is waiting, new readers are blocked until the writer finishes. This prevents starvation but reduces read throughput.
Fair (FIFO): requests are served in arrival order. Balanced but may have lower throughput than either preference.
| Policy | Read throughput | Write latency | Starvation risk |
|---|---|---|---|
| Reader-preference | Highest | Unbounded | Writers starve |
| Writer-preference | Lower | Bounded | Readers may wait |
| Fair | Moderate | Moderate | None |
Most production systems use writer-preference to ensure updates eventually happen.
When to use an RWLock
The decision depends on your read-to-write ratio:
- >10:1 reads to writes: RWLock likely wins
- ~1:1 reads to writes: Regular lock is simpler and performs similarly
- Writes dominate: Regular lock is better (RWLock adds overhead with no benefit)
Also consider the read hold time. If reads are very fast (a dictionary lookup), the overhead of acquiring an RWLock may exceed the time saved by allowing concurrent reads. RWLocks shine when reads take meaningful time — file scanning, complex computation on cached data, tree traversal.
Common misconception
“RWLocks are always better than regular locks for read-heavy workloads.” The overhead of managing reader counts makes RWLocks slower for very fast operations. If your critical section is a single dictionary lookup (nanoseconds), the lock acquisition cost dominates and a simple Lock wins. Profile before switching.
The GIL factor
In CPython, the GIL already serializes bytecode execution. An RWLock in a threading context helps when reads involve I/O (the GIL is released) or C extensions. For pure Python dictionary reads, the GIL ensures safety and a regular lock just adds contention management. RWLocks become genuinely valuable in multiprocessing with shared memory and in the emerging no-GIL Python builds.
The one thing to remember: reader-writer locks optimize for the common case where reads vastly outnumber writes. They allow concurrent reads while ensuring exclusive write access. Choose writer-preference to avoid starvation, and only use them when read operations are expensive enough to justify the overhead.
See Also
- Python Actor Model Why treating each piece of your program like a person with their own mailbox makes concurrency way less scary.
- Python Aiocache Caching aiocache remembers expensive answers so your async Python app doesn't waste time asking the same question twice.
- Python Aiofiles Async Io aiofiles lets your async Python program read and write files without freezing — because normal file operations secretly block everything.
- Python Aiohttp Understand Aiohttp through an everyday analogy so Python behavior feels intuitive, not random.
- Python Anyio Portability AnyIO lets your async Python code work with any async library — write once, run on asyncio or Trio without changes.