e23922a1b7
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
- Multi-assignment clustering (no greedy 'used' set) - Adaptive ngram fallback (shrink when (other) > 60%) - Add docs/adr/ with 10 architecture decision records - Fix ADR 1 (query modification description) - Fix ADR 3 (multi-assignment + adaptive shrink) - Fix ADR 5 (import sort order clarification) - Fix ADR 6 (remove Kotlin call_suffix references) - New ADR 9 (adaptive clustering rationale) - New ADR 10 (universal package mapping via relpath)
41 lines
2.2 KiB
Markdown
41 lines
2.2 KiB
Markdown
# 1. Use nvim-treesitter `highlights.scm` as behavioral capture source
|
|
|
|
**Date:** 2026-07-03
|
|
|
|
**Status:** Accepted
|
|
|
|
## Context
|
|
|
|
We need a universal source of behavioral code tokens (function calls, references, definitions) across multiple programming languages. Options:
|
|
|
|
- **`tags.scm`** (nvim-treesitter): Purpose-built for symbol tagging. Covers definitions and references.
|
|
- **`highlights.scm`** (nvim-treesitter): Built for syntax highlighting. Covers a wider range of tokens including keywords, operators, and built-ins.
|
|
- **Custom per-language queries**: Write and maintain our own query files for each language.
|
|
|
|
We need tokens that represent *what the code does at runtime* — not just structure.
|
|
|
|
## Decision
|
|
|
|
Use nvim-treesitter `highlights.scm` as the capture source for all 10 languages.
|
|
|
|
We filter captures to a `BEHAVIORAL_PREFIXES` set: `definition.`, `reference.`, `keyword.`, `function`, `attribute`, `constructor`, `label`, `type.definition`, `module`.
|
|
|
|
For Kotlin, use the `ts-kotlin` (fwcd fork) bundled `highlights.scm` instead of nvim-treesitter's, because nvim-treesitter's Kotlin query references a duplicate `annotation` node type that doesn't exist in the grammar.
|
|
|
|
## Consequences
|
|
|
|
**Positive:**
|
|
- `highlights.scm` covers 4 out of 5 behavioral capture types that `tags.scm` misses, across all 10 languages.
|
|
- No per-language custom code or adapters needed.
|
|
- Community-maintained queries stay fresh with language evolution.
|
|
- Same query files work for both parsing and tokenizing.
|
|
|
|
**Negative:**
|
|
- `highlights.scm` includes non-behavioral captures (comments, punctuation, operators) — we filter these out.
|
|
- Some nvim-treesitter queries use `#set!` directives (`#set! priority`, `#eq?`) that `py-tree-sitter` doesn't support. These patterns are removed in the bundled copies under `queries/`.
|
|
- Kotlin requires a separate grammar package (`ts-kotlin`) because the nvim-treesitter Kotlin grammar is incompatible.
|
|
|
|
## Alternatives Considered
|
|
|
|
- **`tags.scm`**: Cleaner signal-to-noise ratio, but misses `function`, `attribute`, `constructor`, `module` captures that are essential for behavioral understanding.
|
|
- **Custom queries**: Would give full control but require per-language maintenance — violates our universal-preprocessor constraint.
|