Commit graph

2 commits

Author SHA1 Message Date
tobjend
3468813ec8 fix: validate SORE before returning, skip malformed grammars
- Add validate_sore() to gbnf.py — checks parseability without converting
- _infer_group now validates grammar and returns skip_reason='malformed_grammar'
  for SOREs containing raw code (e.g. w_body=, sult=, (+,+:N+...)
- Results: 130 grammars, 130 GBNF OK, 0 GBNF FAIL
  - Flask: 5 OK, 0 FAIL, 3.4s
  - RAGSAK: 19 OK, 0 FAIL, 11 malformed, 12.9s
  - FastAPI: 106 OK, 0 FAIL, 6 malformed, 30.5s
2026-07-12 02:56:13 +02:00
tobjend
bc7d3b6ca1 perf: iDRegEx opt-in, GBNF newline fix, OverflowError fix
- iDRegEx now opt-in via --idregex flag (was running on every group,
  causing 55s+ on Flask alone — src/flask/json took 55s in iDRegEx)
- GBNF tokenizer strips newlines from literals (multi-line symbols)
- Fix OverflowError: lang_size_score produces huge ints for large
  disjunctions, format as string not float
- Flask: 2.7s (was 55s+), RAGSAK: 13s (was 74s)
2026-07-12 02:52:05 +02:00