e23922a1b7
- Multi-assignment clustering (no greedy 'used' set) - Adaptive ngram fallback (shrink when (other) > 60%) - Add docs/adr/ with 10 architecture decision records - Fix ADR 1 (query modification description) - Fix ADR 3 (multi-assignment + adaptive shrink) - Fix ADR 5 (import sort order clarification) - Fix ADR 6 (remove Kotlin call_suffix references) - New ADR 9 (adaptive clustering rationale) - New ADR 10 (universal package mapping via relpath)
2.8 KiB
3. Method-level n-gram clustering before inference
Date: 2026-07-03
Status: Accepted
Context
The BEX ensemble (CRX, iDRegEx, kORE) infers grammars from sets of symbol sequences. When we run inference on all methods in a codebase, the sequences are too diverse — each file has different conventions, and the ensemble produces only a flat vocabulary bag like (any+assertEquals+assertTrue+every+listOf+verify)+.
This doesn't capture the ordering of calls or the distinct methodological styles present in the codebase.
Decision
Group methods by shared n-gram (default: 3-gram) call patterns before running inference.
Pipeline: preprocess_by_method → frequency_filter → cluster_methods → per-cluster infer_ensemble
The clustering algorithm:
- Extract call tokens from each method sequence (filter to
function,reference.call,reference.classcaptures). - Build an n-gram index: for each method, for each sliding window of size N, record the n-gram.
- Sort n-grams by frequency (most shared first).
- Multi-assignment: methods can belong to every cluster whose n-gram they match (no greedy
usedsubtraction). This avoids the first-pattern-hoards-all problem. - Capped at 20 clusters (
max_clusters=20) to prevent output bloat from many single-token n-grams. - Methods matching NO n-gram (sequences shorter than N, or no peers sharing their n-grams) go to
(other).
Adaptive ngram fallback (cluster_methods_adaptive): when (other) exceeds 60% of total methods, retry with ngram-1. Repeats down to ngram=1. This prevents a single dominant call token from leaving 95% of methods unclustered.
Consequences
Positive:
- iDRegEx and kOREInference now produce ordered grammars (e.g.
every+.assertEquals.verify+.any?) because small, focused clusters have enough signal. - Each cluster reveals a distinct methodological style in the codebase (mockist TDD vs data-driven testing vs pure assertion).
- Multi-assignment means a method can reveal multiple patterns simultaneously (e.g., both
assertEquals-heavy andmockk-heavy clusters). - Adaptive ngram shrink finds the right granularity automatically.
Negative:
- Multi-assignment inflates total
method_countacross clusters (one method counted in N clusters). - Clustering adds a hyperparameter (
ngram_size, default 3). Adaptive shrink mitigates the tuning burden. min_cluster_size(default 3) filters out tiny but potentially interesting patterns.
Alternatives Considered
- Infer on all methods (no clustering): Produces flat vocabulary only. CRX works at 100% coverage, but iDRegEx and kORE fail on diverse inputs.
- Infer per file: Too fine-grained — most files have 1-5 methods, not enough for inference.
- Infer per directory: Better, but directories mix unrelated conventions (setup/teardown vs actual test logic).