Python IMAP Reading Emails — Deep Dive

System-level framing

Reading email programmatically involves three distinct layers: the IMAP connection (network protocol), the message fetch (data retrieval), and the MIME parse (content extraction). Each layer has its own failure modes. Production systems need to handle all three gracefully, especially when processing thousands of messages across unreliable network connections.

Connection patterns

Basic SSL connection

import imaplib

mail = imaplib.IMAP4_SSL('imap.gmail.com', 993)
mail.login('user@gmail.com', 'app-password-here')

# List all mailboxes
status, mailboxes = mail.list()
for mb in mailboxes:
    print(mb.decode())

# Select inbox
mail.select('INBOX')

Context manager for safety

imaplib does not natively support with statements, so wrap it:

from contextlib import contextmanager

@contextmanager
def imap_connection(host, user, password):
    conn = imaplib.IMAP4_SSL(host)
    conn.login(user, password)
    try:
        yield conn
    finally:
        try:
            conn.close()  # Closes selected mailbox
        except Exception:
            pass
        conn.logout()

with imap_connection('imap.gmail.com', user, password) as mail:
    mail.select('INBOX')
    # ... work with messages

Search and fetch strategies

UID-based fetching

# Search for unread messages
status, data = mail.uid('search', None, 'UNSEEN')
uid_list = data[0].split()

for uid in uid_list:
    status, msg_data = mail.uid('fetch', uid, '(RFC822)')
    raw_email = msg_data[0][1]

Always use mail.uid() instead of bare mail.search() and mail.fetch(). UIDs persist across sessions, while sequence numbers shift whenever messages are added or deleted. This prevents the classic bug where your script processes the wrong message after an inbox change.

Partial fetch for performance

Fetching RFC822 downloads the entire message including attachments. For large mailboxes, fetch only what you need:

# Fetch just headers (fast — no body or attachments)
status, data = mail.uid('fetch', uid, '(BODY[HEADER])')

# Fetch just the text body (skip attachments)
status, data = mail.uid('fetch', uid, '(BODY[TEXT])')

# Fetch envelope metadata (fastest)
status, data = mail.uid('fetch', uid, '(ENVELOPE)')

For a triage pipeline that only needs sender and subject, ENVELOPE is orders of magnitude faster than RFC822.

MIME parsing in depth

import email
from email import policy

raw_email = msg_data[0][1]
msg = email.message_from_bytes(raw_email, policy=policy.default)

# Structured access
subject = msg['subject']
sender = msg['from']
date = msg['date']

# Get plain text body
body = msg.get_body(preferencelist=('plain',))
if body:
    text = body.get_content()

# Get HTML body
html_body = msg.get_body(preferencelist=('html',))
if html_body:
    html = html_body.get_content()

Use policy.default (or policy.SMTP) for modern parsing. The legacy parser (no policy argument) returns older Message objects that handle encodings poorly and make attachment extraction harder.

Attachment extraction

from pathlib import Path

def save_attachments(msg, output_dir: Path):
    for attachment in msg.iter_attachments():
        filename = attachment.get_filename()
        if filename:
            filepath = output_dir / filename
            filepath.write_bytes(attachment.get_content())
            yield filepath

Watch out for:

• Duplicate filenames — Multiple attachments can share a name. Add a counter or hash.
• Path traversal — A malicious filename like ../../etc/passwd can escape your output directory. Always sanitize: filename = Path(filename).name.
• Encoding issues — Some clients encode filenames in RFC 2047 format. The modern policy.default parser handles this automatically.

IDLE: push-based notifications

Polling on a timer (every 60 seconds) wastes resources and adds latency. The IMAP IDLE extension lets the server push notifications when new mail arrives:

# imaplib doesn't support IDLE natively
# Use the imapclient library instead
from imapclient import IMAPClient

with IMAPClient('imap.gmail.com', ssl=True) as client:
    client.login('user@gmail.com', 'app-password-here')
    client.select_folder('INBOX')
    
    # Start IDLE mode
    client.idle()
    
    # Block until the server sends a notification (up to 29 min)
    responses = client.idle_check(timeout=300)
    
    # Process new mail
    client.idle_done()
    
    if responses:
        messages = client.search(['UNSEEN'])
        for uid, data in client.fetch(messages, ['RFC822']).items():
            raw = data[b'RFC822']
            # Parse and process...

IDLE connections typically time out after 29 minutes (RFC 2177 recommendation). Your production code needs a loop that re-enters IDLE after each timeout.

Production error handling

import time
import imaplib

class EmailProcessor:
    def __init__(self, host, user, password):
        self.host = host
        self.user = user
        self.password = password
        self.processed_uids = set()  # Persist this to a database
    
    def connect(self):
        self.mail = imaplib.IMAP4_SSL(self.host)
        self.mail.login(self.user, self.password)
        self.mail.select('INBOX')
    
    def process_new_messages(self):
        status, data = self.mail.uid('search', None, 'UNSEEN')
        if status != 'OK':
            raise RuntimeError(f"Search failed: {status}")
        
        uids = data[0].split()
        for uid in uids:
            uid_str = uid.decode()
            if uid_str in self.processed_uids:
                continue
            
            try:
                self._process_one(uid)
                self.processed_uids.add(uid_str)
            except Exception as e:
                log.error(f"Failed to process UID {uid_str}: {e}")
    
    def run_forever(self, interval=60):
        while True:
            try:
                self.connect()
                self.process_new_messages()
                self.mail.logout()
            except (imaplib.IMAP4.abort, 
                    imaplib.IMAP4.error,
                    ConnectionResetError,
                    TimeoutError) as e:
                log.warning(f"Connection error: {e}, retrying...")
                time.sleep(10)
            
            time.sleep(interval)

Key reliability patterns:

• Idempotent processing — Track processed UIDs in a database, not just IMAP flags.
• Connection recycling — IMAP connections go stale. Reconnect on each poll cycle or after errors.
• Graceful degradation — If parsing fails for one message, log it and continue with the next.

OAuth2 authentication

For Google Workspace and Microsoft 365, basic password authentication is deprecated. Use OAuth2:

import imaplib

# After obtaining an OAuth2 access token
auth_string = f'user={email}\x01auth=Bearer {access_token}\x01\x01'
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.authenticate('XOAUTH2', lambda x: auth_string.encode())

This requires registering an OAuth application and implementing the token refresh flow. Libraries like google-auth and msal (Microsoft) handle the token lifecycle.

Tradeoffs: imaplib vs third-party libraries

Factor	imaplib (stdlib)	imapclient	exchangelib
Dependencies	None	One package	One package
IDLE support	No	Yes	N/A (EWS)
API ergonomics	Low-level, bytes everywhere	High-level, Pythonic	High-level, Exchange-native
Gmail support	Manual label handling	Good	N/A
OAuth2	Manual	Manual	Built-in for Microsoft

For simple scripts, imaplib is sufficient. For production pipelines, imapclient provides a much cleaner API. For Microsoft Exchange environments, exchangelib speaks the native EWS protocol.

The one thing to remember: Reliable email reading requires UID-based tracking, proper MIME parsing with the modern email policy, and reconnection logic — the IMAP protocol gives you the tools, but you must handle the statefulness yourself.