# Decision Matrix: CRX vs Refined CRX

## The Question

When should we use standard CRX (fast, always works) vs refined CRX (cluster-then-infer, tighter but sometimes trivial)?

## The Data

Tested on 3 codebases, 14 packages total:

| Package | N | CRX struct | Refined struct | Winner |
|---------|---|-----------|---------------|--------|
| RAGSAK agents | 519 | 0.037 | 0.281 | Refined |
| RAGSAK buildSrc | 13 | 0.146 | 0.361 | Refined |
| RAGSAK entrypoints | 417 | 0.027 | 0.214 | Refined |
| RAGSAK platform | 35 | 0.039 | 0.145 | Refined |
| FastAPI fastapi | 375 | 0.004 | 0.007 | Refined |
| FastAPI scripts | 168 | 0.005 | 0.030 | Refined |
| FastAPI tests | 3618 | 0.117 | 0.043 | CRX |
| Flask examples | 61 | 0.011 | 0.043 | Refined |
| Flask src | 368 | 0.014 | 0.016 | Tie |
| RAGSAK app | 383 | 0.028 | 0.600 | Trivial (return+) |
| RAGSAK infrastructure | 724 | 0.015 | 0.600 | Trivial (single sym) |
| RAGSAK modules | 973 | 0.015 | 0.600 | Trivial (single sym) |
| FastAPI docs_src | 650 | 0.027 | 0.500 | Trivial (single sym) |
| Flask tests | 993 | 0.016 | 0.500 | Trivial (single sym) |

## The Pattern

**Refined CRX wins** (7/14) when:
- CRX structure is low (< 0.05) — flat bags where CRX over-approximates
- Group size is small-to-medium (13-519 methods)
- First symbols are diverse enough to create meaningful clusters

**Refined CRX is trivial** (5/14) when:
- Group size is large (383-973 methods)
- Most sequences share the same first symbol (e.g., all start with `return`)
- Refined clusters everything into one group → CRX on that group → single symbol

**CRX wins** (1/14) when:
- CRX already has decent structure (> 0.1)
- Refined splits too aggressively, losing the overall pattern

## The Decision Matrix

```
                        ┌─────────────────────────────────┐
                        │     Group size (N methods)?      │
                        ├────────────┬────────────────────┤
                        │   N <= 50  │     N > 50         │
┌───────────────────────┼────────────┼────────────────────┤
│ CRX structure < 0.05  │  REFINED   │  REFINED           │
│ (flat bag)            │  (always)  │  (check for trivial)│
├───────────────────────┼────────────┼────────────────────┤
│ CRX structure 0.05-0.2│  REFINED   │  CRX               │
│ (semi-structured)     │  (usually) │  (safe default)     │
├───────────────────────┼────────────┼────────────────────┤
│ CRX structure > 0.2   │  CRX       │  CRX               │
│ (already structured)  │  (already  │  (already good)     │
│                       │   good)    │                     │
└───────────────────────┴────────────┴────────────────────┘
```

## The Rule

```python
if crx_structure >= 0.2:
    use CRX  # already good enough
elif n_methods <= 50:
    use refined  # small group, safe to cluster
elif crx_structure < 0.05:
    use refined with triviality check  # flat bag, worth trying
else:
    use CRX  # medium group, semi-structured, CRX is safer
```

## Triviality Check

When using refined CRX, always check:
```python
if model_cost(refined_grammar) < 2:
    use CRX instead  # refined produced a single symbol, useless
```

This catches the 36% of cases where refined clusters everything into one group.

## Chain of Reasoning

1. **Started with CRX only** — fast, always works, but over-approximates on diverse groups
2. **Tried kORE** — slow (400ms), returns None on real data, no advantage over iDRegEx
3. **Tried iDRegEx** — slow (700ms), returns None on most data, occasionally useful (477x improvement on 1 package)
4. **Tried refined CRX** — cluster-then-infer, better structure on flat bags, but sometimes trivial
5. **Tested across 3 codebases** — refined wins 78% when useful, trivial 36% on large groups
6. **Conclusion**: CRX is the default, refined is opt-in for flat bags, iDRegEx is optional for rare cases

## Final Recommendation

| Scenario | Algorithm | Flag |
|----------|-----------|------|
| Default (most cases) | CRX | `--crx-method standard` |
| Flat bags (struct < 0.05) | Refined CRX | `--crx-method refined` |
| Need absolute best grammar | iDRegEx | `--idregex-refine` |
| Large groups (N > 500) | CRX | (avoid refined, likely trivial) |
| Small groups (N < 20) | Refined CRX | (safe to cluster) |