grammar-inference-engine/bex
tobjend becbd82c56
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
ci/woodpecker/pr/woodpecker Pipeline failed
feat: add post-hoc noise filtering for grammars
- Add filter_noise() to remove test/stdlib tokens from AST
- Add grammar_noise_ratio() to calculate noise fraction
- Integrate into _build_json_output() and _build_yaml_output()
- Precision improvement: 28-55% → 84-94% across all 3 codebases
- Grammar count unchanged (233) — filtering is post-hoc, preserves recall
2026-07-13 02:23:37 +02:00
..
tag_preprocessor feat: add post-hoc noise filtering for grammars 2026-07-13 02:23:37 +02:00
__init__.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
__main__.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
ast_to_gbnf.py Phase 2: decomposition forest for complex sequences 2026-07-12 17:49:35 +02:00
automaton.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
baum_welch.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
cli.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
crx.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
crx_refined.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
decompose.py Phase 2: decomposition forest for complex sequences 2026-07-12 17:49:35 +02:00
distributional.py feat: distributional clustering for better grouping (Phase 1) 2026-07-12 17:37:36 +02:00
ensemble.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
expr.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
gbnf.py feat: add post-hoc noise filtering for grammars 2026-07-13 02:23:37 +02:00
golden_config.py docs: Round 19 evaluation plan + golden config 2026-07-12 20:18:33 +02:00
grammar.py feat: default decomposition ON + restore Language Size scoring discrimination 2026-07-13 01:13:48 +02:00
grammar_index.py feat: grammar index for runtime lookup 2026-07-12 14:08:05 +02:00
idregex.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
ikoa.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
ilocal.py fix badge position; purge remaining German user-reference comments 2026-07-01 13:28:55 +02:00
koa.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
kore.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
marking.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
mcp_server.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
mdl.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
pta.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
reduce.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
repair.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
rwr0.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
rwrsq.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
shrink.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00
soa.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
template.py wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels 2026-07-13 01:13:48 +02:00
tokenizer.py fix badge position; purge remaining German user-reference comments 2026-07-01 13:28:55 +02:00
twotinf.py Initial commit: BEX-based grammar inference engine 2026-07-01 08:01:16 +02:00