Refined wins 7/9 when producing useful grammars (model_cost >= 2).
Trivial output (single symbol) in 5/13 cases on large groups.
CRX wins only once (FastAPI tests, 3618 methods).
Key finding: refined CRX is better ~78% of the time but needs
triviality check (model_cost >= 2) to avoid single-symbol grammars.
Heuristic: only run iDRegEx when n_methods ≤ 10 AND CRX grammar has
>50% top-level optional parts (flat chain signal). If iDRegEx grammar
is >10x tighter by lang_size, use it. Otherwise keep CRX.
RAGSAK result: agents/capability (5 methods) refined from
slot?.(defaultCapabilityId+summarize)?... (lang_size=1432) to
(defaultCapabilityId|summarize) (lang_size=3) — 477x tighter.
Speed cost: ~0.7s per candidate, negligible on 74s pipeline.
CLI: --idregex-refine flag (default off).
Also adds _count_optionals() and _should_try_idregex() helpers
with 8 pytest tests. 234 tests pass.
Key finding: iDRegEx/kORE return None on genuinely diverse groups (flat bags).
CRX is the only algorithm that produces anything for these groups.
For structured groups, CRX already captures ordering well.
Conclusion: kORE/iDRegEx are not better fallbacks for flat bags.
The pipeline's existing filtering (min_structure, split_mixed) is the right approach.
Standard CRX over-approximates when Hasse diagram is non-linear (24% of
RAGSAK packages). Cluster-then-infer groups sequences by (first, last,
length), infers per-cluster, picks largest cluster's grammar.
Results on RAGSAK:
Avg max disjunction: 2.8 → 1.7 (39% tighter)
Packages improved: 6/10
Tradeoff: cluster granularity (too coarse = over-approximation,
too fine = no generalization). Current: (first, last, length_bucket).
Exports crx_refined() and crx_with_confidence() from bex package.
20 new tests. All 199 tests pass.
Sweet spot: 0.01-0.05. At 0.01 RAGSAK gets 61 SOREs (26.1% cov) with
real conventions like warn.status.body.ErrorResponse. At 0.05 coverage
jumps to 45.4% but that pattern disappears. Flask dies at 0.15+.
Current default min_coverage=0.2 is too aggressive for most codebases.
