ca7ccb36ff
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
ADR 0001: nvim-treesitter highlights.scm as capture source ADR 0002: language-agnostic method extraction via child_by_field_name ADR 0003: method-level n-gram clustering before inference ADR 0004: frequency filter with min_coverage threshold ADR 0005: import extraction per cluster ADR 0006: argument pattern extraction via AST node classification ADR 0007: JSON output for LLM prompt injection ADR 0008: BEX ensemble for grammar inference
41 lines
2.1 KiB
Markdown
41 lines
2.1 KiB
Markdown
# 1. Use nvim-treesitter `highlights.scm` as behavioral capture source
|
|
|
|
**Date:** 2026-07-03
|
|
|
|
**Status:** Accepted
|
|
|
|
## Context
|
|
|
|
We need a universal source of behavioral code tokens (function calls, references, definitions) across multiple programming languages. Options:
|
|
|
|
- **`tags.scm`** (nvim-treesitter): Purpose-built for symbol tagging. Covers definitions and references.
|
|
- **`highlights.scm`** (nvim-treesitter): Built for syntax highlighting. Covers a wider range of tokens including keywords, operators, and built-ins.
|
|
- **Custom per-language queries**: Write and maintain our own query files for each language.
|
|
|
|
We need tokens that represent *what the code does at runtime* — not just structure.
|
|
|
|
## Decision
|
|
|
|
Use nvim-treesitter `highlights.scm` as the capture source for all 10 languages.
|
|
|
|
We filter captures to a `BEHAVIORAL_PREFIXES` set: `definition.`, `reference.`, `keyword.`, `function`, `attribute`, `constructor`, `label`, `type.definition`, `module`.
|
|
|
|
For Kotlin, use the `ts-kotlin` (fwcd fork) bundled `highlights.scm` instead of nvim-treesitter's, because nvim-treesitter's Kotlin query references a duplicate `annotation` node type that doesn't exist in the grammar.
|
|
|
|
## Consequences
|
|
|
|
**Positive:**
|
|
- `highlights.scm` covers 4 out of 5 behavioral capture types that `tags.scm` misses, across all 10 languages.
|
|
- No per-language custom code or adapters needed.
|
|
- Community-maintained queries stay fresh with language evolution.
|
|
- Same query files work for both parsing and tokenizing.
|
|
|
|
**Negative:**
|
|
- `highlights.scm` includes non-behavioral captures (comments, punctuation, operators) — we filter these out.
|
|
- Two `jsx` captures use `#set!` with 3 arguments, which `py-tree-sitter` 0.26 rejects. Strip these 2 patterns.
|
|
- Kotlin requires a separate grammar package (`ts-kotlin`) because the nvim-treesitter Kotlin grammar is incompatible.
|
|
|
|
## Alternatives Considered
|
|
|
|
- **`tags.scm`**: Cleaner signal-to-noise ratio, but misses `function`, `attribute`, `constructor`, `module` captures that are essential for behavioral understanding.
|
|
- **Custom queries**: Would give full control but require per-language maintenance — violates our universal-preprocessor constraint.
|