tobjend
|
dd183f6241
|
feat: 4-codebase evaluation — conventions vs completions tradeoff
Added kotlinx.coroutines (1040 .kt files) and FastAPI (1129 .py files).
Key findings across 4 codebases:
- Coarsening trades per-package coverage for cross-package reach
- FastAPI: coverage IMPROVES (12.4% -> 20.7%), cross-pkg +15
- Coroutines: cross-pkg +15, real exception-handling patterns
- RAGSAK: cross-pkg +18, null-check conventions across 8 packages
- Flask: slight cross-pkg loss, rendering conventions still visible
The conventions vs completions tradeoff is real and measurable.
|
2026-07-12 01:14:09 +02:00 |
|
tobjend
|
8540947eec
|
docs: Round 6b results — Kotlin capture fix + minimal coarsening
|
2026-07-12 01:05:19 +02:00 |
|
tobjend
|
592974f039
|
feat: structural coarsening experiment — keeps function names raw, coarsens keywords
- coarsen_token() maps tree-sitter captures to categories (RETURN, IF, LOOP, etc.)
- Function calls kept as raw text (they ARE the behavioral content)
- Only ~5% structural tokens coarsened
- Results: Flask coverage 17.5% -> 28.8% (+11.3%), RAGSAK unchanged (95% calls)
- Cross-package shapes emerge: ('IF', 'KW', 'RETURN') in 4 Flask packages
|
2026-07-12 00:56:09 +02:00 |
|
tobjend
|
8e1c7f3767
|
docs: experiment log + cross-package analysis
- Full experiment history: context strategies, Reduce, scoring, GBNF, cross-package
- Documented failures: per-package sparsity, Reduce at wrong level, exact match rarity
- Designed next experiment: structural coarsening via tree-sitter categories
- Updated RESULTS.md with Round 5 findings and summary table
|
2026-07-12 00:45:21 +02:00 |
|