- idregex_refine=True default across all entry points + CLI
- Added --no-idregex-refine flag to CLI
- Rewrote _count_optionals from SORE string parser to AST walker
- Added _is_pure_bag() helper for bag detection
- Updated _should_try_idregex() to trigger on pure Plus(Alt) bags
- Updated tests for AST-based _count_optionals
- Added RAGSAK v4 experiment results
Run RAGSAK (Kotlin), fastapi (Python), zod (TypeScript) with --slice
package --min-structure 0.5, verbose+timeout, results saved for later.
After _count_concat memoization fix: RAGSAK full run 54.9s -> 6.7s,
no tail stalls. RAGSAK: 121 packages (28 with grammar). fastapi/zod
complete in ~25s (dominated by arg-extraction on 2000+ method test
groups, not grammar explosion). Includes SUMMARY.md with before/after.
- Plan: match rate audit, decomposition A/B, distributional A/B, fragment quality
- Golden config: best-known values for all heuristic parameters
- Single source of truth in bex/golden_config.py
- ACHIEVEMENT_SUMMARY.md: what we built and learned
- Round 14: Full pipeline run on 4 codebases (clean symbols)
- Round 15: kORE/iDRegEx vs CRX comparison
Next: Crucio-inspired clustering for better grouping
- RESEARCH_POSITIONING.md: where we fit in the science
- DECISION_MATRIX.md: when to use CRX vs refined CRX
- Updated AGENTS.md with research context and updated CLI flags
Refined wins 7/9 when producing useful grammars (model_cost >= 2).
Trivial output (single symbol) in 5/13 cases on large groups.
CRX wins only once (FastAPI tests, 3618 methods).
Key finding: refined CRX is better ~78% of the time but needs
triviality check (model_cost >= 2) to avoid single-symbol grammars.
Heuristic: only run iDRegEx when n_methods ≤ 10 AND CRX grammar has
>50% top-level optional parts (flat chain signal). If iDRegEx grammar
is >10x tighter by lang_size, use it. Otherwise keep CRX.
RAGSAK result: agents/capability (5 methods) refined from
slot?.(defaultCapabilityId+summarize)?... (lang_size=1432) to
(defaultCapabilityId|summarize) (lang_size=3) — 477x tighter.
Speed cost: ~0.7s per candidate, negligible on 74s pipeline.
CLI: --idregex-refine flag (default off).
Also adds _count_optionals() and _should_try_idregex() helpers
with 8 pytest tests. 234 tests pass.
Key finding: iDRegEx/kORE return None on genuinely diverse groups (flat bags).
CRX is the only algorithm that produces anything for these groups.
For structured groups, CRX already captures ordering well.
Conclusion: kORE/iDRegEx are not better fallbacks for flat bags.
The pipeline's existing filtering (min_structure, split_mixed) is the right approach.
- iDRegEx now opt-in via --idregex flag (was running on every group,
causing 55s+ on Flask alone — src/flask/json took 55s in iDRegEx)
- GBNF tokenizer strips newlines from literals (multi-line symbols)
- Fix OverflowError: lang_size_score produces huge ints for large
disjunctions, format as string not float
- Flask: 2.7s (was 55s+), RAGSAK: 13s (was 74s)
Standard CRX over-approximates when Hasse diagram is non-linear (24% of
RAGSAK packages). Cluster-then-infer groups sequences by (first, last,
length), infers per-cluster, picks largest cluster's grammar.
Results on RAGSAK:
Avg max disjunction: 2.8 → 1.7 (39% tighter)
Packages improved: 6/10
Tradeoff: cluster granularity (too coarse = over-approximation,
too fine = no generalization). Current: (first, last, length_bucket).
Exports crx_refined() and crx_with_confidence() from bex package.
20 new tests. All 199 tests pass.
Sweet spot: 0.01-0.05. At 0.01 RAGSAK gets 61 SOREs (26.1% cov) with
real conventions like warn.status.body.ErrorResponse. At 0.05 coverage
jumps to 45.4% but that pattern disappears. Flask dies at 0.15+.
Current default min_coverage=0.2 is too aggressive for most codebases.
