grammar-inference-engine/experiments/RESEARCH_POSITIONING.md

Where We Fit in the Science

The Landscape (2024-2026)

Grammar inference for code is active across 4 distinct research areas:

┌─────────────────────────────────────────────────────────────────────┐
│                    Grammar Inference Landscape                      │
├─────────────────┬─────────────────┬─────────────────┬───────────────┤
│ Black-Box CFG   │ White-Box CFG   │ Constrained     │ Behavioral    │
│ Inference       │ Inference       │ Decoding        │ Type Inference│
├─────────────────┼─────────────────┼─────────────────┼───────────────┤
│ Crucio (ICSE'26)│ Panini (OOPSLA'25)│ XGrammar      │ REST (OOPSLA'25)│
│ Kedavra (ASE'24)│ Leon (ICSE'25)  │ DOMINO         │ Typify (ICPC'26)│
│ Arvada          │                 │ ASAp (NeurIPS'24)│ RightTyper   │
│ Treevada        │                 │ CRANE (ICML'25) │ DAInfer+      │
│                 │                 │ TreeCoder       │               │
└─────────────────┴─────────────────┴─────────────────┴───────────────┘

What Others Do

Black-Box CFG Inference (Crucio, Kedavra, Arvada)

• Input: Example strings + oracle (accept/reject)
• Output: Context-free grammar
• Method: Decompose strings, generalize via distributional analysis
• Target: Programming language grammars (JSON, XML, C, Java)
• Limitation: Needs oracle, assumes regular/context-free languages

White-Box CFG Inference (Panini, Leon)

• Input: Parser source code (ad hoc parsers)
• Output: Regular grammar
• Method: Refinement type inference + abstract interpretation
• Target: String parsing functions (split, regex, format)
• Limitation: Only works on parser functions, not behavioral patterns

Constrained Decoding (XGrammar, DOMINO, ASAp, CRANE)

• Input: Grammar + LLM
• Output: LLM constrained to grammar
• Method: Mask invalid tokens during generation
• Target: JSON, SQL, code with strict syntax
• Limitation: Requires pre-existing grammar, only enforces syntax

Behavioral Type Inference (REST, Typify, RightTyper)

• Input: Source code
• Output: Type annotations
• Method: Static/dynamic analysis + type inference
• Target: Function signatures, return types, parameter types
• Limitation: Types only, not behavioral patterns

Where We Are Different

We are the only approach that infers behavioral grammars from execution patterns.

Others:
  Parser source code → Grammar (for input validation)
  Example strings → Grammar (for language definition)
  Grammar → LLM (for output constraint)

Us:
  Source code → Behavioral sequences → Grammar (for usage patterns)
  Grammar → LLM (for context/constraint)

Key Distinctions

Aspect	Others	Us
Input	Parser code or example strings	Any codebase (behavioral sequences)
Output	Grammar for input validation	Grammar for usage patterns
Target	String parsing functions	API call sequences
Granularity	Per-function or per-language	Per-package/module
Language support	Usually 1 language	Any tree-sitter supported language
Use case	Formal verification, testing	LLM code generation guidance

The Gap We Fill

1. Panini infers grammars for individual ad hoc parsers (e.g., json.loads). We infer grammars for how packages are used (e.g., Flask route patterns).

2. Crucio/Kedavra infer grammars from input/output examples. We infer grammars from observed execution patterns — no oracle needed, we have the source.

3. XGrammar/DOMINO enforce pre-existing grammars during LLM generation. We discover the grammars that should be enforced.

4. REST/Typify infer types (what something is). We infer behavioral patterns (how something is used).

Our Contribution

The Behavioral Grammar Concept

Definition: A behavioral grammar captures the valid sequences of API calls within a package or module, expressed as a regular expression.

# Example: Flask route handler
Grammar: GET_RETURN (POST_RETURN)* RETURN
# Means: Routes often start with GET, sometimes POST, always return

The Pipeline

Source Code → tree-sitter AST → Behavioral Sequences → BEX Algorithms → YAML/GBNF
                                         ↑
                                    Our innovation
                              (language-agnostic preprocessing)

What Makes It Work

1. Behavioral prefix extraction: We capture the intent of code, not the implementation
2. Token coarsening: RETURN, IF, EXCEPTION, LOOP — abstract enough to generalize
3. Package slicing: Context matters (Flask vs FastAPI vs Django)
4. Recursive splitting: Separate mixed groups by first symbol
5. Multiple algorithms: CRX (fast), refined CRX (tighter), iDRegEx (rare)

Where We Don't Fit (Yet)

Limitations Compared to Others

1. Not formal verification: Our grammars are approximate, not proven correct
2. Not parser inference: We don't infer grammars for string parsing
3. Not constraint enforcement: We don't yet integrate with XGrammar/DOMINO
4. Not type inference: We complement types, not replace them
5. Not language-specific: We don't leverage language-specific type systems

What We Could Become

1. Grammar + Type hybrid: Combine our behavioral grammars with Typify's type inference
2. Constrained decoding integration: Feed our grammars to XGrammar for LLM guidance
3. API specification inference: Combine with DAInfer+ for full API contracts
4. Testing: Use behavioral grammars for test case generation
5. Documentation: Auto-generate usage patterns from code

Research Positioning

Our Niche

                    ┌─────────────────────────────┐
                    │    Behavioral Grammar       │
                    │    Inference (Us)           │
                    └──────────┬──────────────────┘
                               │
        ┌──────────────────────┼──────────────────────┐
        │                      │                      │
        ▼                      ▼                      ▼
┌───────────────┐    ┌───────────────┐    ┌───────────────┐
│ Tree-sitter   │    │ BEX Algorithm │    │ LLM Context   │
│ Preprocessing │    │ Adaptation    │    │ Generation    │
│ (Language-    │    │ (XML → Code   │    │ (Grammar →    │
│  agnostic)    │    │  patterns)    │    │  Prompt/      │
│               │    │               │    │  Constraint)  │
└───────────────┘    └───────────────┘    └───────────────┘

Related Work (Cite)

1. Bex et al. (2010): "Inference of concise regular expressions and DTDs" — CRX algorithm
2. Schröder & Cito (2025): "Static inference of regular grammars for ad hoc parsers" — Panini
3. Li et al. (2024): "Incremental context-free grammar inference in black box settings" — Kedavra
4. Li et al. (2026): "Context-free grammar inference for complex programming languages" — Crucio
5. Dong et al. (2024): "XGrammar: Flexible and efficient structured generation engine" — Constrained decoding
6. Park et al. (2025): "Flexible and efficient grammar-constrained decoding" — GCD
7. Tam et al. (2025): "Grammar-constrained decoding makes LLMs better logical parsers" — GCD + reasoning
8. Masoudian et al. (2026): "DAInfer+: Neurosymbolic inference of API specifications" — API contracts
9. Typify (2026): "Usage-driven static analyzer for precise Python type inference" — Type inference

Our Novelty

1. First to apply BEX to behavioral sequences (not XML/input data)
2. Language-agnostic preprocessing via tree-sitter (not language-specific)
3. Package-level behavioral patterns (not per-function or per-language)
4. Grammar as LLM context (not formal verification or testing)
5. Hybrid approach combining BEX algorithms with modern preprocessing

Summary

We occupy a unique position: behavioral grammar inference from source code. Others do:

• Grammar inference for parsers (Panini, Crucio)
• Grammar enforcement for LLMs (XGrammar, DOMINO)
• Type inference for code (Typify, REST)

We do: Discover the patterns that should be inferred, enforced, or typed.