Python Conversation Memory — Core Concepts

What Is Conversation Memory?

Conversation memory is the mechanism that allows a chatbot to maintain context across multiple exchanges. Without it, every user message is processed in isolation — the bot cannot resolve pronouns (“it”, “that”), recall previously mentioned entities, or track the progress of a multi-step task.

Types of Memory

Turn-Level Context

The most basic form of memory passes the previous turn’s data to the current turn’s processing. This lets the bot resolve simple references like “Yes, that one” when the previous message offered a choice.

Session Memory (Short-Term)

Session memory persists for the duration of a conversation. It stores:

• Slot values collected so far (destination, date, name)
• Conversation history — the sequence of user messages and bot responses
• Dialog state — which step of the flow the bot is on

When the user closes the chat or the session times out, this memory is typically cleared.

Persistent Memory (Long-Term)

Long-term memory survives across sessions. Common use cases:

• User preferences (“They always want window seats”)
• Past interactions (“Last time they booked a flight to Tokyo”)
• Account information linked to a user ID

This data lives in a database and is loaded when the user starts a new conversation.

How Context Flows Through Turns

Each turn in a conversation follows a cycle:

1. Load — Retrieve the current memory state for this conversation.
2. Update — Add the new user message and any extracted entities to memory.
3. Process — The dialog manager uses the full memory to decide the next action.
4. Store — Save the updated memory state back to storage.

In Python, this is usually implemented as middleware that wraps the core bot logic:

User message → Load state → NLU → Dialog Manager (reads state) → NLG → Save state → Reply

The Sliding Window

LLM-based chatbots face a practical limit: the model can only process a finite amount of text (its context window). A sliding window keeps the most recent N turns and discards older ones. This trades completeness for efficiency — the bot always has recent context but may “forget” things mentioned twenty messages ago.

The window size is a tradeoff. Too small (3-5 turns) and the bot forgets mid-conversation. Too large (50+ turns) and you burn tokens or exceed model limits.

Entity Tracking

Beyond raw message history, bots track entities — structured data extracted from the conversation. If the user says “I’m flying to Berlin” in turn 3, the entity destination=Berlin should be available in turn 10 without re-extraction.

Entity tracking also handles updates. If the user says “Actually, change that to Munich,” the tracker overwrites the destination. This is simpler than re-processing the entire history to figure out the current state.

Memory Strategies for LLM Chatbots

When using language models (GPT, Claude) as the conversation engine, memory becomes particularly important because the model has no built-in state:

• Full history: Pass the entire conversation as the prompt. Simple but expensive and eventually hits token limits.
• Summary memory: Periodically summarize older parts of the conversation into a compact paragraph. The model sees the summary plus recent turns.
• Retrieval-augmented memory: Store all turns in a vector database. Before each response, retrieve the most relevant past turns and include them in the prompt.

Common Misconception

Many people assume that chatbots “remember” things the way humans do — forming a continuous, evolving understanding. In reality, most chatbot memory is mechanical: data is stored, retrieved, and discarded according to explicit rules. The bot does not “understand” that two conversations are related unless the code explicitly links them through a user ID or session token.

Python Tools

• Redis: Fast key-value store for session memory. TTL (time-to-live) handles automatic expiry.
• SQLite / PostgreSQL: Persistent storage for long-term memory and conversation logs.
• LangChain Memory modules: Pre-built memory types (buffer, summary, entity) for LLM chatbot applications.
• Rasa Tracker Store: Built-in session memory with pluggable backends (in-memory, Redis, SQL).

The one thing to remember: Conversation memory is not a single feature but a layered system — turn context for immediate references, session memory for multi-step tasks, and persistent memory for cross-session personalization — each with its own storage and lifecycle.