tobi/grammar-inference-engine
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Commit graph

  • 8fc459e06b docs: add hype document for grammar inference breakthrough feature/treesitter-tag-queries tobjend 2026-07-13 02:33:29 +02:00
  • 9be44c2964 feat: add quality gate for grammar usefulness scoring tobjend 2026-07-13 02:32:45 +02:00
  • becbd82c56 feat: add post-hoc noise filtering for grammars tobjend 2026-07-13 02:23:37 +02:00
  • 9b3dca46eb feat: loosen filtering thresholds to keep more grammars tobjend 2026-07-13 02:15:45 +02:00
  • 36f9e9173d docs: update experiment log, results, and handover for Round 19-20 tobjend 2026-07-13 01:18:32 +02:00
  • 8b3a454a15 feat: enable idregex_refine by default + AST rewrite of _count_optionals tobjend 2026-07-13 01:12:54 +02:00
  • 8460c3c862 experiments: round 20 v3 results (decompose default + cap fix) tobjend 2026-07-13 00:51:22 +02:00
  • 8f412905c5 feat: default decomposition ON + restore Language Size scoring discrimination tobjend 2026-07-13 00:51:11 +02:00
  • 1810a695d0 experiments: round 20 AST-migration verification across 3 languages tobjend 2026-07-13 00:03:48 +02:00
  • 52f286a073 fix: restore memoization on _count_concat to stop count_words explosion tobjend 2026-07-13 00:03:40 +02:00
  • f8dda557d2 chore: update experiment scripts for AST representation tobjend 2026-07-12 23:46:25 +02:00
  • 19a1db48ef test: update tests for AST representation tobjend 2026-07-12 23:46:25 +02:00
  • dac33db616 feat: BEX_VERBOSE env var + real-time parallel progress logging tobjend 2026-07-12 23:46:05 +02:00
  • ea6cac53e3 wip: AST foundation — grammar.py, expr.py→AST, soa.py→AST labels tobjend 2026-07-12 20:59:53 +02:00
  • 9d206f586f docs: Round 19 evaluation plan + golden config tobjend 2026-07-12 20:18:33 +02:00
  • 44dd34a46f docs: handover document for project continuity tobjend 2026-07-12 19:54:10 +02:00
  • ee60b62854 fix: YAML expansion bug and add GBNF to output tobjend 2026-07-12 19:53:24 +02:00
  • 9ca56e2c69 MCP server: add decomposition + quality filtering params tobjend 2026-07-12 18:56:35 +02:00
  • 8b2899d16e Phase 2: decomposition forest for complex sequences tobjend 2026-07-12 17:49:35 +02:00
  • 841f5efcf5 feat: distributional clustering for better grouping (Phase 1) tobjend 2026-07-12 17:37:36 +02:00
  • 2da3a7161f docs: achievement summary and experiment outputs (Round 14-15) tobjend 2026-07-12 17:25:15 +02:00
  • 242f00a0da docs: research positioning and decision matrix tobjend 2026-07-12 17:14:56 +02:00
  • b2c1263838 docs: ASCII diagrams for pipeline, parameters, and decision matrix tobjend 2026-07-12 17:09:40 +02:00
  • 1d94c09344 experiment: CRX vs refined CRX across 3 codebases (Round 17) tobjend 2026-07-12 17:08:15 +02:00
  • b92b7653e2 feat: iDRegEx refinement for CRX flat bags (Round 16) tobjend 2026-07-12 16:54:40 +02:00
  • e9f672cba4 experiment: kORE/iDRegEx vs CRX on flat bags (Round 15) tobjend 2026-07-12 16:20:01 +02:00
  • 93d53f164a fix: byte/char offset mismatch in tree-sitter text extraction tobjend 2026-07-12 15:57:27 +02:00
  • 617b7ad578 wip: call_only filter for JS/TS + checkpoint before GBNF rewrite tobjend 2026-07-12 15:20:55 +02:00
  • 44415c5b42 feat: runtime grammar lookup via MCP + fix YAML output tobjend 2026-07-12 14:08:12 +02:00
  • 7b54a5ef77 feat: grammar index for runtime lookup tobjend 2026-07-12 14:08:05 +02:00
  • fe3261b9de docs: add Round 13 to experiment log (recursive split results) tobjend 2026-07-12 13:54:03 +02:00
  • ca8a13b549 refactor: recursive split-by-first-symbol (max_depth=3) tobjend 2026-07-12 13:53:33 +02:00
  • ca328b5402 feat: split mixed-pattern groups before CRX inference tobjend 2026-07-12 13:42:02 +02:00
  • 2733840358 feat: add reduce slicing, fix rwr0 timeout, fix sequence format tobjend 2026-07-12 13:13:16 +02:00
  • 17561a2f37 fix: sanitize symbol extraction, fix malformed grammars tobjend 2026-07-12 12:53:48 +02:00
  • 52c90bfdd8 tune: min_methods 5→3, fix DEFAULT_COVERAGE help text tobjend 2026-07-12 03:54:35 +02:00
  • 11364151a6 fix: set DEFAULT_COVERAGE to 0.05 (experimentally determined sweet spot) tobjend 2026-07-12 03:26:22 +02:00
  • 483526cb6a docs: Round 12 results — grammar structure scoring tobjend 2026-07-12 03:03:16 +02:00
  • e62fffc6e0 feat: grammar_structure_score + min_structure filter tobjend 2026-07-12 03:02:50 +02:00
  • 92af932e9d docs: Round 11 results — pipeline speed + GBNF conversion tobjend 2026-07-12 02:57:40 +02:00
  • 3468813ec8 fix: validate SORE before returning, skip malformed grammars tobjend 2026-07-12 02:56:13 +02:00
  • bc7d3b6ca1 perf: iDRegEx opt-in, GBNF newline fix, OverflowError fix tobjend 2026-07-12 02:52:05 +02:00
  • f57c302c91 fix(gbnf): handle disjunction inside parens and compound repetition tobjend 2026-07-12 02:20:58 +02:00
  • 6912841b9e fix: wire crx_refined into pipeline, fix hardcoded min_coverage tobjend 2026-07-12 02:09:52 +02:00
  • 739000e8c6 feat: CRX refined — cluster-then-infer for tighter grammars tobjend 2026-07-12 01:50:40 +02:00
  • 8028570ceb feat: frequency threshold sweep — 0.01-0.20 across 4 codebases tobjend 2026-07-12 01:26:45 +02:00
  • 1b98b1d24e chore: add external_refs to .gitignore tobjend 2026-07-12 01:14:49 +02:00
  • dd183f6241 feat: 4-codebase evaluation — conventions vs completions tradeoff tobjend 2026-07-12 01:14:09 +02:00
  • 8540947eec docs: Round 6b results — Kotlin capture fix + minimal coarsening tobjend 2026-07-12 01:05:19 +02:00
  • d65b78bdf1 fix: bare Kotlin captures + minimal coarsening (RETURN/IF/EXCEPTION/LOOP only) tobjend 2026-07-12 01:04:51 +02:00
  • 592974f039 feat: structural coarsening experiment — keeps function names raw, coarsens keywords tobjend 2026-07-12 00:56:09 +02:00
  • 8e1c7f3767 docs: experiment log + cross-package analysis tobjend 2026-07-12 00:45:21 +02:00
  • 011df391c2 feat: implement SORE → GBNF converter tobjend 2026-07-12 00:31:47 +02:00
  • b516b2985d feat: implement Reduce algorithm (Algorithm 4, TODS 2010) tobjend 2026-07-12 00:11:38 +02:00
  • bbdfe93679 experiments: test all context strategies on RAGSAK — behavioral grouping wins tobjend 2026-07-11 23:40:21 +02:00
  • ab20b03256 feat: add diversity threshold — skip groups with unique_ratio > 0.9 or methods < min_methods tobjend 2026-07-11 23:11:54 +02:00
  • dfb56a083a WIP: language size scoring + diversity threshold (step 1 pending) tobjend 2026-07-11 22:56:42 +02:00
  • 830104b399 feat: add analyze_directory MCP tool — scan codebase, infer conventions, persist to .dervish/ tobjend 2026-07-11 21:28:35 +02:00
  • e94c52b71a docs: update stale docs — remove kORE from default ensemble, add tag preprocessor CLI tobjend 2026-07-11 20:51:10 +02:00
  • ce6521ad5e Revert "parallelize kORE outer (k, n) trials via ProcessPoolExecutor" tobjend 2026-07-11 20:36:48 +02:00
  • 0b5b0e623b parallelize kORE outer (k, n) trials via ProcessPoolExecutor tobjend 2026-07-04 03:04:54 +02:00
  • 5906adfc95 parallelize preprocessing across files via ProcessPoolExecutor tobjend 2026-07-04 02:58:47 +02:00
  • 710da56916 drop kORE from default ensemble, add --kore flag to opt in tobjend 2026-07-04 02:58:07 +02:00
  • e3ad256321 feat: add --slice package (per-directory grouping), --include/--exclude, --verbose, drop root group inference, run full ensemble on all groups tobjend 2026-07-04 02:43:03 +02:00
  • 4a506cd39b fix: add bare function fallback to CALL_PREFIXES, fix IMPORT_PATTERNS order, update pipeline diagram tobjend 2026-07-04 01:28:35 +02:00
  • eb2173442b fix: restore frequency_filter at 0.2, add dual 0.8 BEX coverage tobjend 2026-07-04 01:13:48 +02:00
  • 55d21af48e docs: pipeline overview ASCII diagram tobjend 2026-07-03 22:59:51 +02:00
  • e23922a1b7 feat: adaptive multi-assignment clustering; add ADRs 1-10 tobjend 2026-07-03 22:58:09 +02:00
  • fd574da53d feat: language-agnostic import/arg/package extraction tobjend 2026-07-03 22:41:04 +02:00
  • ca7ccb36ff docs: add 8 architecture decision records feature/kotlin-specific-extras tobjend 2026-07-03 22:01:35 +02:00
  • c059d0b7a4 feat: file-level package mapping; add 7 new tests tobjend 2026-07-03 21:59:26 +02:00
  • b6c18c39f2 feat: per-call argument pattern extraction tobjend 2026-07-03 21:57:15 +02:00
  • 2d4fc8eed5 feat: per-cluster import extraction + JSON output tobjend 2026-07-03 21:52:02 +02:00
  • 73b94af959 feat: cluster methods by n-gram patterns before inference tobjend 2026-07-03 21:41:46 +02:00
  • 2620b6e49f feat: method-level sequence extraction via child_by_field_name('body') tobjend 2026-07-03 21:34:25 +02:00
  • 0c7703f63b feat: universal tag-preprocessor orchestrator with frequency filter tobjend 2026-07-03 20:58:18 +02:00
  • 8bda174293 docs: document Kotlin and JS/TS special cases in analysis tobjend 2026-07-03 20:54:03 +02:00
  • 70300ff917 chore: add Gemini session chat logs as reference tobjend 2026-07-03 20:48:50 +02:00
  • ee5ebc9eb4 feat: universal AST preprocessor with nvim-treesitter queries tobjend 2026-07-03 20:48:47 +02:00
  • 986eaa7f83 feat: add community tags findings audit and kotlin tags query tobjend 2026-07-03 18:40:53 +02:00
  • 8d06ac2d52 chore: add downloaded community queries for kotlin + typescript tobjend 2026-07-03 18:05:19 +02:00
  • 069c63f2c8 feat: inspect TreeSitter tag queries for Dervish universal AST extraction tobjend 2026-07-03 17:46:33 +02:00
  • 28f5f897d5 Update README.md main tobi 2026-07-02 18:17:16 +00:00
  • d74b36e563 Update README.md tobi 2026-07-02 18:14:48 +00:00
  • 197a0a3c22 Update README.md tobi 2026-07-02 18:08:42 +00:00
  • 136ae08fe3 Update README.md tobi 2026-07-02 18:06:38 +00:00
  • ea8e2f1db7 Update README.md tobi 2026-07-02 16:33:51 +00:00
  • 16cbff61a8 Update README.md tobi 2026-07-02 16:33:02 +00:00
  • b037098730 fix: regenerate chart with Comic Neue + path.sketch xkcd style tobjend 2026-07-01 16:13:39 +02:00
  • 2a7111b7ff fix: regenerate chart with Comic Neue + path.sketch xkcd style feature/kore-inference tobjend 2026-07-01 16:13:39 +02:00
  • d2d57bc431 Merge pull request 'feat: kOREInference — Algorithm 4 iDRegEx with MDL scoring + ensemble integration' (#1) from feature/kore-inference into main tobi 2026-07-01 14:08:18 +00:00
  • 90d8c69aa7 fix: broken y-axis bar chart for readability tobjend 2026-07-01 16:05:49 +02:00
  • 2562519718 fix: grouped bar chart with Without Dervish vs With Dervish tobjend 2026-07-01 16:04:55 +02:00
  • ce4088e705 fix: chart bars use Dervish purple #853E91 tobjend 2026-07-01 16:03:20 +02:00
  • d96e99d84f docs: xkcd-style chart tobjend 2026-07-01 16:02:14 +02:00
  • b34e39d4b9 feat: replace single-chart Helm with cross-project convention (15 charts, 6 publishers) tobjend 2026-07-01 16:00:04 +02:00
  • fbdb39bf7c docs: syntax highlighting tags in SHOWCASE.md tobjend 2026-07-01 15:48:36 +02:00
  • bc81bbdce1 docs: extra blank line above coffee button tobjend 2026-07-01 15:42:14 +02:00
  • bb193d000f docs: bump logo 180→216px tobjend 2026-07-01 15:41:55 +02:00
  • 4969819dbb docs: center logo, move coffee button below nav links tobjend 2026-07-01 15:41:35 +02:00