Lark Parsing Library — Core Concepts

What Is Lark?

Lark is a modern parsing library for Python that takes a grammar-first approach. You define your language’s structure in an EBNF (Extended Backus-Naur Form) grammar, and Lark generates a parser that converts text into a parse tree. Unlike older tools where grammar rules live inside Python code, Lark keeps the grammar clean and separate.

Grammar Syntax

Lark grammars use a readable notation. Rules are defined with names, colons, and patterns:

start: item ("," item)*

item: NUMBER "x" NAME

NUMBER: /[0-9]+/
NAME: /[a-zA-Z]+/

%ignore /\s+/

Rules (lowercase) define structure — how elements combine. Terminals (UPPERCASE) define atomic tokens matched by regular expressions. The %ignore directive tells the lexer to skip matching patterns (typically whitespace).

Two Parsing Algorithms

Lark offers two built-in parsing strategies:

LALR(1) — Fast and memory-efficient. Works best with unambiguous grammars. This is the default choice for production use. It processes input in a single left-to-right pass with one token of lookahead.

Earley — Handles any context-free grammar, including ambiguous ones. Slower (O(n³) worst case) but incredibly flexible. Choose Earley when your grammar has inherent ambiguity or when rapid prototyping matters more than speed.

from lark import Lark

# LALR parser (fast, strict)
parser_lalr = Lark(grammar_text, parser='lalr')

# Earley parser (flexible, slower)
parser_earley = Lark(grammar_text, parser='earley')

Parse Trees and Transformers

Parsing produces a Tree object with named nodes matching your grammar rules. Each node contains children that are either sub-trees or Token objects:

tree = parser.parse("3x apples, 2x bananas")
# Tree('start', [Tree('item', [Token('NUMBER', '3'), Token('NAME', 'apples')]), ...])

To process the tree, Lark provides Transformers — classes where methods named after grammar rules receive the children and return transformed values:

from lark import Transformer

class GroceryTransformer(Transformer):
    def item(self, children):
        quantity, name = children
        return {"name": str(name), "quantity": int(quantity)}

    def start(self, items):
        return list(items)

result = GroceryTransformer().transform(tree)
# [{"name": "apples", "quantity": 3}, {"name": "bananas", "quantity": 2}]

Transformers process the tree bottom-up: leaf nodes first, then their parents. This makes them natural for evaluation, compilation, or data extraction.

How It Works

Lark’s processing pipeline has three stages:

1. Lexing — The input string is split into tokens based on terminal definitions. Lark can use its own regex-based lexer or delegate to a contextual lexer that adapts based on parser state.
2. Parsing — Tokens are organized into a parse tree according to grammar rules. The algorithm depends on your choice (LALR or Earley).
3. Transformation — Optional post-processing converts the raw tree into your desired output format using Transformers or Visitors.

Common Misconception

People often think Lark is “just another PLY.” The fundamental difference is the grammar-first design. In PLY, grammar rules are embedded in Python docstrings and function signatures. In Lark, the grammar is a standalone artifact — you can read it, version it, share it, and even visualize it independently of any Python code. This separation makes Lark grammars dramatically easier to maintain for complex languages.

When to Use Lark

Lark is a strong choice when you want clean grammar files, need to handle ambiguous input, or want built-in tree transformation. It ships with a library of pre-built grammars (JSON, Python, and more) that you can import directly. For performance-critical applications processing millions of records, LALR mode is competitive with PLY, while Earley mode trades speed for grammar flexibility.

One thing to remember: Lark separates grammar from code — you define what your language looks like in EBNF, and Lark handles the parsing, tree building, and transformation pipeline.