docs: add 8 architecture decision records

ADR 0001: nvim-treesitter highlights.scm as capture source
ADR 0002: language-agnostic method extraction via child_by_field_name
ADR 0003: method-level n-gram clustering before inference
ADR 0004: frequency filter with min_coverage threshold
ADR 0005: import extraction per cluster
ADR 0006: argument pattern extraction via AST node classification
ADR 0007: JSON output for LLM prompt injection
ADR 0008: BEX ensemble for grammar inference

docs/adr/0001-use-nvim-treesitter-highlights-scm.md

@@ -0,0 +1,41 @@
+# 1. Use nvim-treesitter `highlights.scm` as behavioral capture source
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+We need a universal source of behavioral code tokens (function calls, references, definitions) across multiple programming languages. Options:
+
+- **`tags.scm`** (nvim-treesitter): Purpose-built for symbol tagging. Covers definitions and references.
+- **`highlights.scm`** (nvim-treesitter): Built for syntax highlighting. Covers a wider range of tokens including keywords, operators, and built-ins.
+- **Custom per-language queries**: Write and maintain our own query files for each language.
+
+We need tokens that represent *what the code does at runtime* — not just structure.
+
+## Decision
+
+Use nvim-treesitter `highlights.scm` as the capture source for all 10 languages.
+
+We filter captures to a `BEHAVIORAL_PREFIXES` set: `definition.`, `reference.`, `keyword.`, `function`, `attribute`, `constructor`, `label`, `type.definition`, `module`.
+
+For Kotlin, use the `ts-kotlin` (fwcd fork) bundled `highlights.scm` instead of nvim-treesitter's, because nvim-treesitter's Kotlin query references a duplicate `annotation` node type that doesn't exist in the grammar.
+
+## Consequences
+
+**Positive:**
+- `highlights.scm` covers 4 out of 5 behavioral capture types that `tags.scm` misses, across all 10 languages.
+- No per-language custom code or adapters needed.
+- Community-maintained queries stay fresh with language evolution.
+- Same query files work for both parsing and tokenizing.
+
+**Negative:**
+- `highlights.scm` includes non-behavioral captures (comments, punctuation, operators) — we filter these out.
+- Two `jsx` captures use `#set!` with 3 arguments, which `py-tree-sitter` 0.26 rejects. Strip these 2 patterns.
+- Kotlin requires a separate grammar package (`ts-kotlin`) because the nvim-treesitter Kotlin grammar is incompatible.
+
+## Alternatives Considered
+
+- **`tags.scm`**: Cleaner signal-to-noise ratio, but misses `function`, `attribute`, `constructor`, `module` captures that are essential for behavioral understanding.
+- **Custom queries**: Would give full control but require per-language maintenance — violates our universal-preprocessor constraint.

docs/adr/0002-language-agnostic-method-extraction.md

@@ -0,0 +1,50 @@
+# 2. Language-agnostic method extraction via `child_by_field_name("body")`
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+To analyze method-level behavioral conventions, we must extract the body of each function/method from the AST. The standard tree-sitter approach is `node.child_by_field_name("body")`, but this named field is not universal across all language grammars.
+
+We need one code path that works for all 10 supported languages without per-language branches.
+
+## Decision
+
+Use `node.child_by_field_name("body")` as the primary extraction method. When it returns `None`, fall back to scanning the node's children for any child with a type containing `body`, `block`, or `compound_statement`.
+
+Parent nodes are further filtered to only include nodes whose type contains `function` or `method` — avoiding class bodies, loop bodies, and conditional blocks.
+
+This logic lives in `_find_method_bodies()` in `code.py`:
+
+```python
+def walk(node):
+    body = node.child_by_field_name("body")
+    if not body:
+        for child in node.children:
+            ctype = child.type.lower()
+            if "body" in ctype or "block" in ctype or ctype == "compound_statement":
+                body = child; break
+    if body:
+        ptype = node.type.lower()
+        if "function" in ptype or "method" in ptype:
+            bodies.append(body)
+    for child in node.children: walk(child)
+```
+
+## Consequences
+
+**Positive:**
+- Works for 9/10 grammars via `child_by_field_name("body")` alone (Python, Go, Rust, JS, TS, Ruby, Java, C, C++).
+- Kotlin fallback works because the fwcd Kotlin grammar uses `function_body` as a child node type.
+- Zero per-language case analysis — just pattern matching on type strings.
+
+**Negative:**
+- Fallback relies on string matching (`"body" in ctype`) which could produce false positives if future grammar versions introduce new body-like types.
+- C/C++ `function_definition` uses `declarator` field for the function name, not `name` — affects name extraction but not body extraction.
+
+## Alternatives Considered
+
+- **Grammar-specific field names**: Map each language to its body field name. Rejected because it creates a maintenance burden and violates the zero-adapters constraint.
+- **Top-down sibling traversal**: Walk from node start to next sibling to find the body. Fragile across grammars with different compound statement structures.

docs/adr/0003-method-level-n-gram-clustering.md

@@ -0,0 +1,44 @@
+# 3. Method-level n-gram clustering before inference
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+The BEX ensemble (CRX, iDRegEx, kORE) infers grammars from sets of symbol sequences. When we run inference on *all methods in a codebase*, the sequences are too diverse — each file has different conventions, and the ensemble produces only a flat vocabulary bag like `(any+assertEquals+assertTrue+every+listOf+verify)+`.
+
+This doesn't capture the *ordering* of calls or the distinct methodological styles present in the codebase.
+
+## Decision
+
+Group methods by shared n-gram (default: 3-gram) call patterns *before* running inference.
+
+Pipeline: `preprocess_by_method` → `frequency_filter` → `cluster_methods` → per-cluster `infer_ensemble`
+
+The clustering algorithm:
+1. Extract call tokens from each method sequence (filter to `function`, `reference.call`, `reference.class` captures).
+2. Build an n-gram index: for each method, for each sliding window of size N, record the n-gram.
+3. Sort n-grams by frequency (most shared first).
+4. Assign each method to the largest matching cluster, then remove assigned methods.
+5. Remaining unclustered methods go to `(other)`.
+
+This produces 10-30 clusters for a typical test suite, each with 3-100+ methods sharing a call-order pattern.
+
+## Consequences
+
+**Positive:**
+- iDRegEx and kOREInference now produce ordered grammars (e.g. `every+.assertEquals.verify+.any?`) because small, focused clusters have enough signal.
+- Each cluster reveals a distinct *methodological style* in the codebase (mockist TDD vs data-driven testing vs pure assertion).
+- The `(other)` cluster still captures the full vocabulary bag for diverse methods.
+
+**Negative:**
+- Clustering adds a hyperparameter (`ngram_size`, default 3). Wrong value can produce too many tiny clusters or one giant cluster.
+- `min_cluster_size` (default 3) filters out tiny but potentially interesting patterns.
+- Methods in `(other)` never get ordered grammar inference — just vocabulary.
+
+## Alternatives Considered
+
+- **Infer on all methods (no clustering)**: Produces flat vocabulary only. CRX works at 100% coverage, but iDRegEx and kORE fail on diverse inputs.
+- **Infer per file**: Too fine-grained — most files have 1-5 methods, not enough for inference.
+- **Infer per directory**: Better, but directories mix unrelated conventions (setup/teardown vs actual test logic).

docs/adr/0004-frequency-filter-with-min-coverage.md

@@ -0,0 +1,51 @@
+# 4. Frequency filter with `min_coverage` threshold
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+A raw method sequence can contain hundreds of unique call tokens, many of which appear in only 1-2 methods. These rare symbols are noise — they inflate grammar size, confuse the inference algorithm, and dilute the signal of common conventions.
+
+We need a principled way to discard rare symbols while keeping the behavioral patterns that define the codebase.
+
+## Decision
+
+Apply a frequency filter *before* clustering: remove any symbol that appears in fewer than `min_coverage` fraction of method sequences.
+
+Default threshold: `0.2` (20% of methods must contain the symbol).
+
+Filtering is done by `frequency_filter()` in `analyze.py`:
+```python
+n_files = len(sequences)
+threshold = max(1, int(n_files * min_coverage))
+symbol_file_count = Counter()
+for seq in sequences:
+    seen = set()
+    for _, text, _ in seq:
+        symbol_file_count[text] += 1 if text not in seen else 0
+        seen.add(text)
+keep = {text for text, count in symbol_file_count.items() if count >= threshold}
+```
+
+A symbol is counted once per file (not once per occurrence) to avoid skew from files that repeat the same symbol many times.
+
+## Consequences
+
+**Positive:**
+- Removes noise before clustering, improving cluster quality.
+- Prevents rare one-off function calls from creating spurious n-gram matches.
+- The `common` vs `other` cluster distinction is sharper because the filter removes tokens that would appear in neither.
+
+**Negative:**
+- With large codebases (600+ methods), 20% threshold may be too aggressive — a symbol needs 120+ occurrences to survive.
+  - Mitigation: `--min-coverage` flag lets users tune per codebase.
+  - Production code often needs lower values (`0.05`) because methods are more diverse than tests.
+- The threshold is relative, not absolute. A 3-file project keeps anything in 1+ files (`max(1, 3*0.2)` = 1).
+
+## Alternatives Considered
+
+- **No filter**: CRX produces `(a+b+c+d+e+f+g+h+i+j+...)+` — the vocabulary is too large to be informative.
+- **Absolute threshold**: `min_occurrences=5`. Doesn't scale — works for small projects, wrong for large ones.
+- **TF-IDF style weighting**: More sophisticated but adds complexity. The simple coverage filter works well in practice.

docs/adr/0005-import-extraction-per-cluster.md

@@ -0,0 +1,45 @@
+# 5. Import extraction per cluster
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+An LLM prompted with a behavioral convention like `every → assertEquals → verify` still needs to know *which imports to use*. Without imports, it will guess the wrong library — writing `from unittest.mock import patch` instead of `import io.mockk.every`, or importing from `jest` instead of `vitest`.
+
+Imports are the bridge between abstract conventions and actionable code.
+
+## Decision
+
+For each cluster, scan the source files whose methods belong to that cluster and extract all unique import lines.
+
+Language-agnostic approach: match lines against common import patterns:
+- `import ...` (Java, Kotlin, Python, Go, JS/TS)
+- `from ... import ...` (Python)
+- `require ...` / `require_relative ...` (Ruby, JS)
+- `#include ...` (C/C++)
+- `use ...` (Rust)
+- `include ...` (Ruby)
+
+Scan the first 200 lines of each file (imports are always at the top), deduplicate across files, and sort the result.
+
+File-to-cluster mapping is preserved by tracking `(file_path, sequence)` pairs through the pipeline. After `frequency_filter` (which preserves order and count), we use object identity to map each clustered sequence back to its source file.
+
+## Consequences
+
+**Positive:**
+- Each cluster shows exact import lines used by its methods.
+- An LLM can copy these directly — no guessing.
+- Reveals *library choice conventions*: `kotlin.test.*` vs `org.junit.jupiter.api.*`, `io.mockk.coEvery` vs `io.mockk.every`.
+
+**Negative:**
+- Import scanning re-reads files (second pass). Negligible cost since files are small and OS-cached.
+- 200-line scan limit might miss imports in files with very long license headers.
+- Lines containing `import` in prose (comments, strings) may produce false positives — rare in practice.
+
+## Alternatives Considered
+
+- **Single global import list**: Simpler but useless — conflates imports from unrelated clusters.
+- **No imports**: LLM must guess. Leads to wrong imports and broken code.
+- **Per-file imports (not per-cluster)**: Too granular — mixes test imports with production imports in the same file.

docs/adr/0006-argument-pattern-extraction.md

@@ -0,0 +1,53 @@
+# 6. Argument pattern extraction via AST node classification
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+A behavioral token like `assertEquals` tells the LLM that the function is called, but not *how*. Two codebases both use `assertEquals` — one writes `assertEquals(expected, actual)` and the other writes `assertEquals(actual, expected)` with swapped argument order. An LLM guessing the wrong order writes broken tests.
+
+The highlights.scm captures tell us *that* a function is called. We need the argument *structure* — number of arguments, their types, and the common patterns.
+
+## Decision
+
+For each behavioral capture node, walk up to its parent `call_expression` (or equivalent), find the argument list node, and classify each argument by structural role.
+
+Argument classification is language-agnostic:
+
+| Classification | Matches |
+|---|---|
+| `lit` | string, number, boolean, null |
+| `var` | identifiers, names |
+| `call` | nested call expressions, method invocations |
+| `lambda` | lambda expressions, blocks, do-blocks |
+| `kwarg` | keyword/named arguments |
+| `expr` | binary/unary/ternary/operator expressions |
+| `template` | string interpolation, template literals |
+| `other` | anything else (fallback) |
+
+Argument list node detection uses a tiered approach:
+1. `child_by_field_name("arguments")` — works for Python, JS, TS, Java, Go, Ruby, Rust.
+2. Fallback: scan children for `argument_list`, `arguments`, `call_suffix` (Kotlin), `template_string` (JS tagged templates).
+3. Kotlin special case: `call_suffix` may contain a direct `lambda_expression` child (for `every { ... }` syntax) or a `value_arguments → value_argument` chain (for `func(a, b)` syntax).
+
+Results are aggregated per cluster into a summary showing min/max/common arg counts and the top argument-type patterns.
+
+## Consequences
+
+**Positive:**
+- Reveals argument ordering conventions: `assertEquals: n=2 [lit,var]` means expected-first.
+- Reveals calling convention variance: `verify: n=0 [] | n=1 [lambda] | n=1 [var]` means three styles coexist.
+- No per-language branches — the tiered arglist detection handles all 10 grammars.
+
+**Negative:**
+- `kwarg` detection only covers named arguments, not default values or spread operators.
+- Nested destructuring patterns fall into `other` bucket — no granularity for complex argument shapes.
+- `other` is a catch-all that can hide meaningful distinctions we haven't classified yet.
+
+## Alternatives Considered
+
+- **Extract raw argument text**: Language-agnostic but fragile — variable names change per test, producing high variance and low signal.
+- **No argument extraction**: The LLM sees `assertEquals` but doesn't know argument order. Leads to wrong code.
+- **Per-language argument extractors**: Would be more precise but violate the zero-adapters constraint.

docs/adr/0007-json-output-for-llm-prompt-injection.md

@@ -0,0 +1,63 @@
+# 7. JSON output for LLM prompt injection
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+The text table output is human-readable but not directly usable by an LLM. To use Dervish conventions in another agent or coding session, the output must be parsed, reformatted, and injected into a prompt — an extra friction step.
+
+An LLM consuming conventions needs:
+- Structured data it can read directly (no parsing).
+- All metadata per convention (grammar, imports, args, files, packages).
+- Compact enough to fit in context without overflow.
+
+## Decision
+
+Add a `--json` flag that outputs a structured JSON array instead of the text table.
+
+JSON structure:
+```json
+[{
+  "language": ".kt",
+  "conventions": [{
+    "label": "every → assertEquals → verify",
+    "method_count": 16,
+    "algorithm": "CRX",
+    "grammar": "every+.assertEquals.verify+.any?",
+    "mdl_score": 8.64,
+    "imports": ["import io.mockk.every", "..."],
+    "packages": ["eu/corentic/springrag/agent/capability"],
+    "arg_patterns": {
+      "assertEquals": {
+        "occurrences": 42,
+        "arg_count": {"min": 2, "max": 3, "common": 2},
+        "patterns": [{"count": 30, "args": 2, "types": ["lit", "var"]}]
+      }
+    }
+  }],
+  "total_methods": 665
+}]
+```
+
+Also accepts `--format json` and `--format text` for explicit control.
+
+## Consequences
+
+**Positive:**
+- LLM consumes the JSON directly — no parsing step needed.
+- All metadata in one object per convention — imports, args, files, packages all together.
+- `--json` is a single flag — the default text output remains for human review.
+
+**Negative:**
+- JSON is more verbose than text (full import list instead of truncated preview).
+- No easy way to limit output size — a large codebase produces JSON that may overflow context.
+  - Mitigation: `--include` flag filters files before analysis.
+
+## Alternatives Considered
+
+- **YAML output**: More readable, but less universally parseable by LLMs.
+- **CSV output**: Too flat for nested data (arg_patterns, imports list).
+- **Custom prompt template**: Would need per-framework templates. JSON is framework-agnostic.
+- **No structured output**: User must pipe through `jq` or manual reformatting. Bad UX.

docs/adr/0008-bex-ensemble-for-grammar-inference.md

@@ -0,0 +1,60 @@
+# 8. BEX ensemble for grammar inference
+
+**Date:** 2026-07-03
+
+**Status:** Accepted
+
+## Context
+
+Given a set of symbol sequences (e.g. `["every", "assertEquals", "verify"]`), we need to infer a grammar that concisely describes the pattern. Three algorithms are available:
+
+- **CRX**: Fast, produces unordered CHAREs (e.g. `(a+b+c)+`). Best for vocabulary discovery.
+- **iDRegEx**: Slower, produces ordered regex with alternation and optionality (e.g. `a.b.(c|d)?`). Best for small, clean sequences.
+- **kOREInference**: Probabilistic, handles noise well (e.g. `a.b.(b?(a|c))`). Best for diverse sequences with outliers.
+
+No single algorithm works best for all codebases. We need to pick the right one for each cluster automatically.
+
+## Decision
+
+Run all three algorithms (ensemble), compute MDL (Minimum Description Length) for each, and select the one with the lowest MDL score.
+
+MDL = grammar_length + sum of per-example encoding costs. Lower is better — the grammar explains the data most compactly.
+
+Ensemble logic in `infer_ensemble()`:
+```python
+def infer_ensemble(sequences, kmax=2, N=3, prefer=None):
+    best = None
+    best_score = float('inf')
+    for name, fn in [('CRX', crx), ('iDRegEx', idregex), ('kOREInference', kore)]:
+        if prefer and name.lower() != prefer.lower():
+            continue
+        grammar = fn(sequences, ...)
+        mdl = compute_mdl(grammar, sequences)
+        if mdl < best_score:
+            best_score = mdl
+            best = {'algorithm': name, 'grammar': grammar, 'mdl_score': mdl}
+    return {'best': best, 'all': all_results, 'why': {...}}
+```
+
+Default `kmax=2`, `N=3` (max k for k-ORE, random trials).
+
+## Consequences
+
+**Positive:**
+- CRX handles large clusters with diverse vocabulary — produces useful vocabulary bags.
+- iDRegEx fires on small, focused clusters (3-12 methods) — produces ordered grammars with exact subsequences.
+- kOREInference handles noisy clusters where methods share a theme but vary in exact call order.
+- MDL provides a principled, automatic selection criterion.
+
+**Negative:**
+- k-ORE algorithms fail on real code when sequences are too diverse (per-file sequences differ more than per-log sequences they were designed for).
+  - Clustering helps by grouping similar methods before inference.
+- iDRegEx can produce overfit grammars on very small clusters (3 methods) — e.g. `every.every.verify.(assertEquals)?` for 3 methods that happen to share an exact sequence.
+- MDL comparison assumes grammars are comparable — CRX CHAREs and iDRegEx regex use different notation, so length comparison is approximate.
+
+## Alternatives Considered
+
+- **Single algorithm (CRX only)**: Fast but produces only unordered vocab — misses ordering conventions entirely.
+- **Single algorithm (iDRegEx only)**: Produces ordered grammars but fails on diverse inputs (returns `ε`).
+- **Single algorithm (kORE only)**: Most robust to noise but slowest, and still fails on highly diverse code sequences.
+- **Algorithm per cluster size**: Manual heuristic (CRX for >20 methods, iDRegEx for <10). Harder to tune than MDL-driven selection.