- idregex_refine=True default across all entry points + CLI
- Added --no-idregex-refine flag to CLI
- Rewrote _count_optionals from SORE string parser to AST walker
- Added _is_pure_bag() helper for bag detection
- Updated _should_try_idregex() to trigger on pure Plus(Alt) bags
- Updated tests for AST-based _count_optionals
- Added RAGSAK v4 experiment results
Heuristic: only run iDRegEx when n_methods ≤ 10 AND CRX grammar has
>50% top-level optional parts (flat chain signal). If iDRegEx grammar
is >10x tighter by lang_size, use it. Otherwise keep CRX.
RAGSAK result: agents/capability (5 methods) refined from
slot?.(defaultCapabilityId+summarize)?... (lang_size=1432) to
(defaultCapabilityId|summarize) (lang_size=3) — 477x tighter.
Speed cost: ~0.7s per candidate, negligible on 74s pipeline.
CLI: --idregex-refine flag (default off).
Also adds _count_optionals() and _should_try_idregex() helpers
with 8 pytest tests. 234 tests pass.
- _extract_call_tokens filters to call-like captures only
- cluster_methods groups sequences by shared 3-gram call patterns
- analyze_clusters runs per-cluster CRX/iDRegEx/kORE inference
- Lists 17 convention clusters for RAGSAK test code
- iDRegEx and kOREInference now produce ordered grammars per cluster
- (other) cluster captures diverse conventions as CRX vocabulary
- min-cluster-size (default 3) and ngram-size (default 3) CLI flags
- _find_method_bodies uses tree-sitter's universal body field (9/10 grammars)
- Kotlin fallback: scan children for body-like types
- preprocess_by_method groups highlight captures by enclosing method body
- Returns per-method sequences for k-ORE ordering analysis
- analyze.py infer() now uses preprocess_by_method
- Document findings in ANALYSIS.md
- 97 tests pass
