B (decompose default):
- analyze_directory + CLI now default decompose=True (--no-decompose to
disable); default max_seq_length 5 -> 4 (matches golden_config).
- Decomposing long sequences into <=4-symbol fragments yields tighter
grammars and disables the too_diverse skip, so far more packages get a
grammar. RAGSAK: 29 -> 95 grammars, pure bags 9 -> 5; fastapi 74 -> 26
pure bags; zod 15 -> 5. All runs exit 0, no stalls.
A (repair Language Size scorer, not a new one):
- _COUNT_CAP was applied to count_words' RETURN value, silently clamping
lang_size_score / model_cost / data_cost at 10^12 for every real codebase
grammar. That broke ADR-13 (Language Size picks most-specific grammar):
bags and tight grammars tied at 10^12, so the scorer could not prefer
specific grammars over generic ones.
- _COUNT_CAP raised to 10**30. Memoization (_count_concat, earlier commit)
already prevents the recursion hang the cap was guarding against, so the
cap no longer needs to clamp scores. lang_size_score now discriminates
(verified: tight=20 vs bag=9975).
During the AST migration, _count_concat lost its @lru_cache (it was
replaced by an uncached recursive version). Distributing a length L
across concat parts then revisited the same (remaining_parts, length)
states exponentially -> 16M function calls for a single 4-method group,
and several RAGSAK groups took 18-52s (looked hung at the tail).
Restore memoization: _count_concat is now @lru_cache-keyed on
(tuple(parts), length). Also fix a stray blank line between the
@lru_cache decorator and count_words.
Impact (RAGSAK, --slice package --min-structure 0.5):
straggler groups 18-52s -> <0.5s; full run 54.9s -> 6.7s.
Adds a regression test asserting a length-20 (a|b)* string counts in <2s.
