ci/woodpecker/push/woodpecker Pipeline failed

Details

docs: add 8 architecture decision records

ADR 0001: nvim-treesitter highlights.scm as capture source
ADR 0002: language-agnostic method extraction via child_by_field_name
ADR 0003: method-level n-gram clustering before inference
ADR 0004: frequency filter with min_coverage threshold
ADR 0005: import extraction per cluster
ADR 0006: argument pattern extraction via AST node classification
ADR 0007: JSON output for LLM prompt injection
ADR 0008: BEX ensemble for grammar inference

2026-07-03 22:01:35 +02:00

3 KiB

Raw Blame History

8. BEX ensemble for grammar inference

Date: 2026-07-03

Status: Accepted

Context

Given a set of symbol sequences (e.g. ["every", "assertEquals", "verify"]), we need to infer a grammar that concisely describes the pattern. Three algorithms are available:

CRX: Fast, produces unordered CHAREs (e.g. (a+b+c)+). Best for vocabulary discovery.
iDRegEx: Slower, produces ordered regex with alternation and optionality (e.g. a.b.(c|d)?). Best for small, clean sequences.
kOREInference: Probabilistic, handles noise well (e.g. a.b.(b?(a|c))). Best for diverse sequences with outliers.

No single algorithm works best for all codebases. We need to pick the right one for each cluster automatically.

Decision

Run all three algorithms (ensemble), compute MDL (Minimum Description Length) for each, and select the one with the lowest MDL score.

MDL = grammar_length + sum of per-example encoding costs. Lower is better — the grammar explains the data most compactly.

Ensemble logic in infer_ensemble():

def infer_ensemble(sequences, kmax=2, N=3, prefer=None):
    best = None
    best_score = float('inf')
    for name, fn in [('CRX', crx), ('iDRegEx', idregex), ('kOREInference', kore)]:
        if prefer and name.lower() != prefer.lower():
            continue
        grammar = fn(sequences, ...)
        mdl = compute_mdl(grammar, sequences)
        if mdl < best_score:
            best_score = mdl
            best = {'algorithm': name, 'grammar': grammar, 'mdl_score': mdl}
    return {'best': best, 'all': all_results, 'why': {...}}

Default kmax=2, N=3 (max k for k-ORE, random trials).

Consequences

Positive:

CRX handles large clusters with diverse vocabulary — produces useful vocabulary bags.
iDRegEx fires on small, focused clusters (3-12 methods) — produces ordered grammars with exact subsequences.
kOREInference handles noisy clusters where methods share a theme but vary in exact call order.
MDL provides a principled, automatic selection criterion.

Negative:

k-ORE algorithms fail on real code when sequences are too diverse (per-file sequences differ more than per-log sequences they were designed for).
- Clustering helps by grouping similar methods before inference.
iDRegEx can produce overfit grammars on very small clusters (3 methods) — e.g. every.every.verify.(assertEquals)? for 3 methods that happen to share an exact sequence.
MDL comparison assumes grammars are comparable — CRX CHAREs and iDRegEx regex use different notation, so length comparison is approximate.

Alternatives Considered

Single algorithm (CRX only): Fast but produces only unordered vocab — misses ordering conventions entirely.
Single algorithm (iDRegEx only): Produces ordered grammars but fails on diverse inputs (returns ε).
Single algorithm (kORE only): Most robust to noise but slowest, and still fails on highly diverse code sequences.
Algorithm per cluster size: Manual heuristic (CRX for >20 methods, iDRegEx for <10). Harder to tune than MDL-driven selection.

3 KiB Raw Blame History