tobi/grammar-inference-engine
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
grammar-inference-engine/bex
History
tobjend 52f286a073 fix: restore memoization on _count_concat to stop count_words explosion 
During the AST migration, _count_concat lost its @lru_cache (it was
replaced by an uncached recursive version). Distributing a length L
across concat parts then revisited the same (remaining_parts, length)
states exponentially -> 16M function calls for a single 4-method group,
and several RAGSAK groups took 18-52s (looked hung at the tail).

Restore memoization: _count_concat is now @lru_cache-keyed on
(tuple(parts), length). Also fix a stray blank line between the
@lru_cache decorator and count_words.

Impact (RAGSAK, --slice package --min-structure 0.5):
  straggler groups 18-52s -> <0.5s; full run 54.9s -> 6.7s.
Adds a regression test asserting a length-20 (a|b)* string counts in <2s.
2026-07-13 01:13:48 +02:00
..
tag_preprocessor feat: BEX_VERBOSE env var + real-time parallel progress logging 2026-07-13 01:13:48 +02:00
__init__.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
__main__.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
ast_to_gbnf.py Phase 2: decomposition forest for complex sequences 2026-07-12 17:49:35 +02:00
automaton.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
baum_welch.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
cli.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
crx.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
crx_refined.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
decompose.py Phase 2: decomposition forest for complex sequences 2026-07-12 17:49:35 +02:00
distributional.py feat: distributional clustering for better grouping (Phase 1) 2026-07-12 17:37:36 +02:00
ensemble.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
expr.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
gbnf.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
golden_config.py docs: Round 19 evaluation plan + golden config 2026-07-12 20:18:33 +02:00
grammar.py fix: restore memoization on _count_concat to stop count_words explosion 2026-07-13 01:13:48 +02:00
grammar_index.py feat: grammar index for runtime lookup 2026-07-12 14:08:05 +02:00
idregex.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
ikoa.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
ilocal.py fix badge position; purge remaining German user-reference comments 2026-07-01 13:28:55 +02:00
koa.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
kore.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
marking.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
mcp_server.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
mdl.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
pta.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
reduce.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
repair.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
rwr0.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
rwrsq.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
shrink.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
soa.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
template.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
tokenizer.py fix badge position; purge remaining German user-reference comments 2026-07-01 13:28:55 +02:00
twotinf.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00