Python Zero-Copy Buffers — Core Concepts

The Cost of Copying

In typical Python code, many operations create copies without you realizing:

data = b"x" * 10_000_000  # 10 MB of bytes

chunk = data[1000:2000]   # Creates a NEW 1000-byte bytes object
another = data[:5000000]  # Creates a NEW 5 MB bytes object

Each slice creates a completely new object with its own memory allocation. For a program that repeatedly slices large buffers — network servers processing packets, video processors extracting frames — these copies become the dominant cost.

memoryview: Python’s Zero-Copy Tool

A memoryview creates a reference to existing data without copying it:

data = bytearray(10_000_000)  # 10 MB

view = memoryview(data)
chunk = view[1000:2000]   # Zero-copy slice — shares the same memory
another = view[:5000000]  # Zero-copy — still no duplication

The chunk and another views point into the same underlying bytearray. Modifying the bytearray through any view modifies the same data:

chunk[0] = 255
print(data[1000])  # 255 — same memory

The Buffer Protocol

memoryview works because of Python’s buffer protocol — a C-level interface that lets objects expose their internal memory. Objects that support the buffer protocol include:

• bytes and bytearray
• array.array
• NumPy ndarray
• mmap objects
• ctypes arrays

Any object implementing the buffer protocol can be wrapped in a memoryview:

import array

arr = array.array('d', [1.0, 2.0, 3.0, 4.0])
view = memoryview(arr)

print(view.format)    # 'd' (double)
print(view.itemsize)  # 8 bytes
print(view.nbytes)    # 32 bytes total

Practical Patterns

Network Data Processing

When parsing network packets, zero-copy slicing avoids allocating for each field:

def parse_packet(raw: bytes):
    view = memoryview(raw)
    header = view[:12]      # Zero-copy
    payload = view[12:-4]   # Zero-copy
    checksum = view[-4:]    # Zero-copy
    return header, payload, checksum

Socket Send Without Copying

Python sockets accept memoryview objects, enabling zero-copy sends:

data = bytearray(1_000_000)
view = memoryview(data)
sent = 0
while sent < len(data):
    n = sock.send(view[sent:])  # Sends from original buffer
    sent += n

Without memoryview, each data[sent:] slice would create a new bytes object.

Chunked File Processing

def process_file_chunks(filepath, chunk_size=65536):
    with open(filepath, 'rb') as f:
        buf = bytearray(chunk_size)
        view = memoryview(buf)
        while True:
            n = f.readinto(buf)  # Reads directly into existing buffer
            if not n:
                break
            process(view[:n])    # Zero-copy slice of what was read

Using readinto() instead of read() avoids creating a new bytes object for each chunk.

Performance Impact

Benchmarking 1 million slices of a 10 MB buffer:

Method	Time	Memory Allocated
data[start:end] (bytes slicing)	420 ms	~1 GB total (copies)
memoryview(data)[start:end]	180 ms	~80 MB (view objects only)

The memoryview approach is 2.3x faster and allocates 12x less memory. The remaining allocations are the tiny memoryview objects themselves (~56 bytes each), not data copies.

Limitations

Immutable sources produce read-only views. memoryview(b"hello") creates a read-only view because bytes is immutable. Use bytearray when you need writable views.

Not all operations are zero-copy. Converting a memoryview to bytes (bytes(view)) creates a copy. Passing to functions that don’t support the buffer protocol may force a copy.

Keeping views alive keeps the source alive. A tiny slice view holds a reference to the entire source buffer, preventing garbage collection of the large buffer:

data = bytearray(100_000_000)  # 100 MB
tiny_view = memoryview(data)[0:10]  # 10 bytes view
del data  # 100 MB NOT freed — tiny_view holds a reference

Release views when done to allow the source to be collected.

Common Misconception

Developers often think bytes slicing is already efficient because bytes are “simple.” In reality, every bytes slice is a full allocation + copy operation. In hot loops processing megabytes of data, switching to memoryview can cut both CPU time and memory pressure significantly. The buffer protocol exists precisely because copying is expensive enough to warrant a zero-copy alternative.

The one thing to remember: memoryview and the buffer protocol let you slice, share, and pass large data buffers without copying — use them whenever you’re processing chunks of binary data in performance-sensitive code.