tobi/grammar-inference-engine
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
grammar-inference-engine/experiments
History
tobjend 841f5efcf5 feat: distributional clustering for better grouping (Phase 1) 
Inspired by Crucio's distributional matrix (ICSE 2026).
Groups symbols by context similarity instead of first-symbol.

New module: bex/distributional.py
- extract_contexts(): get (left, right) context pairs
- build_distribution_matrix(): symbol × context counts
- cluster_symbols(): agglomerative clustering by similarity
- split_by_cluster(): split sequences by cluster membership
- distributional_split(): drop-in replacement for _split_by_first_symbol

CLI: --cluster-method distributional (opt-in, first-symbol is default)

Example: 'if' and 'while' both appear before 'return'
→ first-symbol: 3 groups (if, while, return)
→ distributional: 2 groups (if/while merged, return)

23 tests pass. Full suite: 257 tests pass.
2026-07-12 17:37:36 +02:00
..
results feat: grammar_structure_score + min_structure filter 2026-07-12 03:02:50 +02:00
round14 docs: achievement summary and experiment outputs (Round 14-15) 2026-07-12 17:25:15 +02:00
round15 docs: achievement summary and experiment outputs (Round 14-15) 2026-07-12 17:25:15 +02:00
ACHIEVEMENT_SUMMARY.md docs: achievement summary and experiment outputs (Round 14-15) 2026-07-12 17:25:15 +02:00
coarsen_eval.py feat: 4-codebase evaluation — conventions vs completions tradeoff 2026-07-12 01:14:09 +02:00
context_eval.py feat: implement Reduce algorithm (Algorithm 4, TODS 2010) 2026-07-12 00:11:38 +02:00
DECISION_MATRIX.md docs: research positioning and decision matrix 2026-07-12 17:14:56 +02:00
DIAGRAMS.md docs: ASCII diagrams for pipeline, parameters, and decision matrix 2026-07-12 17:09:40 +02:00
EXPERIMENT_LOG.md experiment: CRX vs refined CRX across 3 codebases (Round 17) 2026-07-12 17:08:15 +02:00
freq_eval.py feat: frequency threshold sweep — 0.01-0.20 across 4 codebases 2026-07-12 01:26:45 +02:00
gbnf_eval.py perf: iDRegEx opt-in, GBNF newline fix, OverflowError fix 2026-07-12 02:52:05 +02:00
PHASE1_PLAN.md feat: distributional clustering for better grouping (Phase 1) 2026-07-12 17:37:36 +02:00
RESEARCH_POSITIONING.md docs: research positioning and decision matrix 2026-07-12 17:14:56 +02:00
RESULTS.md docs: experiment log + cross-package analysis 2026-07-12 00:45:21 +02:00