Skip to content

Guide & Philosophy

Philosophy & Design

Instead of having to rely on fragile direct source-code manipulation or slightly better magic string-expression-based AST manipulation engines, ast_pattern_engine provides an internal DSL for building explicit, structural patterns.

It explicitly avoids string-based expressions because they simply don't scale well. For robust code-analysis you must have a grasp of the underlying AST structure.

The package started as a component for another project and was later spun off into it's own clean package. Early experiences made it clear that large expressions aren't the way to go for AST manipulation. A staged approach is much more robust and easier to reason about.

The package is intentionally kept somewhat limited to encourage using custom logic for more advanced filtering and analysis. Rather than building extremely nested gigantic expressions, it's encouraged to build small, focused patterns and visitors. Like casting a wide net and progressively filtering down in stages.

Primitives

The library provides several primitives to build robust sequences:

  • NodePattern: Match specific AST node types and assert on their fields.
  • Collect / Bind: Extract sub-trees out of a matched pattern to use in your handlers. Bind assigns a name to a matched node so that it can be processed during transformation.
  • OneOf: Match one of several possible patterns (similar to regex |).
  • Repetition: Match a pattern sequentially 1 or more times (similar to regex * and +).
  • Optional: Match a pattern 0 or 1 times (similar to regex ?).
  • Filter: Apply arbitrary Python lambdas to check node states during matching.

Building Patterns

A typical pattern is a sequence of Pattern objects. For instance, finding a sequence of assignments:

from ast_pattern_engine import NodePattern, Bind
import ast

# Matches an assignment of any value to "x"
assign_to_x = NodePattern(
    ast.Assign,
    targets=[NodePattern(ast.Name, id="x")],
    value=Bind("x_value")
)

Transformers and Finders

Once a pattern is matched, you often want to act on it.

  • Transformers (PatternTransformer, BottomUpPatternTransformer): These visitors replace matched subtrees with new nodes generated by your handlers. A handler receives the bound variables (E.G Bind("x"), `Collect()) and returns the replacement nodes.
  • Finders (PatternFinder, SingleOccurrenceFinder): These visitors just locate where patterns occur in the tree without modifying it. Useful for analysis or linting.

Templates

To reduce boilerplate when building patterns, the library includes a templates module with helpers for common operations: - match_call(func_name, **kwargs) - match_assign(target_name, value) - match_in_expr(pattern)

Using templates allows you to write dense, readable matching rules quickly.