tobjend
93d53f164a
fix: byte/char offset mismatch in tree-sitter text extraction
...
Root cause: parser.parse(code.encode()) returns byte offsets, but
code[node.start_byte:node.end_byte] indexed into a Python string
(character offsets). Non-ASCII chars caused cumulative drift → truncated symbols.
Fix: store code_bytes = code.encode(), index into that, decode only final text.
Impact:
- Zod: 653 truncated symbols → 0
- RAGSAK: 653 truncated symbols → 0
- RAGSAK grammars: 8 → 27 (3.4×)
- FastAPI grammars: 33 → 111 (3.4×)
- Malformed: 6 → 2 (RAGSAK), 0 (FastAPI/Flask)
Also removed sanitize_symbol() (dead code after fix) and call_only parameter.
2026-07-12 15:57:27 +02:00
tobjend
fe3261b9de
docs: add Round 13 to experiment log (recursive split results)
2026-07-12 13:54:03 +02:00
tobjend
483526cb6a
docs: Round 12 results — grammar structure scoring
2026-07-12 03:03:16 +02:00
tobjend
92af932e9d
docs: Round 11 results — pipeline speed + GBNF conversion
...
- iDRegEx opt-in: Flask 55s→2.8s, RAGSAK 74s→13s, FastAPI 30s→30s
- GBNF validation: 130/130 OK, 0 FAIL (17 malformed skipped)
- Grammar quality: 43% structured, rest flat/trivial
- Logged to experiments/EXPERIMENT_LOG.md
2026-07-12 02:57:40 +02:00
tobjend
739000e8c6
feat: CRX refined — cluster-then-infer for tighter grammars
...
Standard CRX over-approximates when Hasse diagram is non-linear (24% of
RAGSAK packages). Cluster-then-infer groups sequences by (first, last,
length), infers per-cluster, picks largest cluster's grammar.
Results on RAGSAK:
Avg max disjunction: 2.8 → 1.7 (39% tighter)
Packages improved: 6/10
Tradeoff: cluster granularity (too coarse = over-approximation,
too fine = no generalization). Current: (first, last, length_bucket).
Exports crx_refined() and crx_with_confidence() from bex package.
20 new tests. All 199 tests pass.
2026-07-12 01:50:40 +02:00
tobjend
8028570ceb
feat: frequency threshold sweep — 0.01-0.20 across 4 codebases
...
Sweet spot: 0.01-0.05. At 0.01 RAGSAK gets 61 SOREs (26.1% cov) with
real conventions like warn.status.body.ErrorResponse. At 0.05 coverage
jumps to 45.4% but that pattern disappears. Flask dies at 0.15+.
Current default min_coverage=0.2 is too aggressive for most codebases.
2026-07-12 01:26:45 +02:00
tobjend
dd183f6241
feat: 4-codebase evaluation — conventions vs completions tradeoff
...
Added kotlinx.coroutines (1040 .kt files) and FastAPI (1129 .py files).
Key findings across 4 codebases:
- Coarsening trades per-package coverage for cross-package reach
- FastAPI: coverage IMPROVES (12.4% -> 20.7%), cross-pkg +15
- Coroutines: cross-pkg +15, real exception-handling patterns
- RAGSAK: cross-pkg +18, null-check conventions across 8 packages
- Flask: slight cross-pkg loss, rendering conventions still visible
The conventions vs completions tradeoff is real and measurable.
2026-07-12 01:14:09 +02:00
tobjend
8540947eec
docs: Round 6b results — Kotlin capture fix + minimal coarsening
2026-07-12 01:05:19 +02:00
tobjend
592974f039
feat: structural coarsening experiment — keeps function names raw, coarsens keywords
...
- coarsen_token() maps tree-sitter captures to categories (RETURN, IF, LOOP, etc.)
- Function calls kept as raw text (they ARE the behavioral content)
- Only ~5% structural tokens coarsened
- Results: Flask coverage 17.5% -> 28.8% (+11.3%), RAGSAK unchanged (95% calls)
- Cross-package shapes emerge: ('IF', 'KW', 'RETURN') in 4 Flask packages
2026-07-12 00:56:09 +02:00
tobjend
8e1c7f3767
docs: experiment log + cross-package analysis
...
- Full experiment history: context strategies, Reduce, scoring, GBNF, cross-package
- Documented failures: per-package sparsity, Reduce at wrong level, exact match rarity
- Designed next experiment: structural coarsening via tree-sitter categories
- Updated RESULTS.md with Round 5 findings and summary table
2026-07-12 00:45:21 +02:00
