diff --git a/experiments/RESULTS.md b/experiments/RESULTS.md new file mode 100644 index 0000000..a90266d --- /dev/null +++ b/experiments/RESULTS.md @@ -0,0 +1,99 @@ +# Experiment Results — RAGSAK Context Strategies + +**Date**: 2026-07-11 +**Codebase**: RAGSAK (462 .kt files, 1594 methods, 137 packages) + +## Summary Table + +| Strategy | Contexts | SORE OK | Coverage | Notes | +|----------|----------|---------|----------|-------| +| Baseline (package) | 116 | 2 | 0.6% | Current approach — terrible | +| File path k=1 | 43 | 2 | 1.1% | Structural — no better than baseline | +| File path k=2 | 54 | 2 | 1.1% | Same — structural doesn't help | +| File path k=3 | 54 | 2 | 1.1% | Same — structural doesn't help | +| **First 1 symbol** | 134 | **20** | **4.6%** | Behavioral — 10× better | +| **First 2 symbols** | 141 | **39** | **9.8%** | Behavioral — 20× better | +| **First 3 symbols** | 117 | **47** | **12.0%** | Behavioral — **winner** | +| Two-dim (p1,s1) | 151 | 24 | 6.3% | Hybrid — worse than behavioral alone | +| Two-dim (p1,s2) | 119 | 33 | 8.7% | Hybrid — worse than behavioral alone | +| Two-dim (p2,s1) | 146 | 24 | 6.3% | Hybrid — worse than behavioral alone | +| Two-dim (p2,s2) | 117 | 33 | 8.7% | Hybrid — worse than behavioral alone | +| Return type heuristic | 5 | 0 | 0.0% | Too coarse — useless | + +## Key Findings + +### 1. Behavioral grouping > Structural grouping +- File path strategies (A) produce same results as baseline — structural similarity doesn't predict behavioral similarity +- First-k-symbols strategies (B) produce 20× more SOREs — behavioral prefix is the right signal +- **Winner: Option B, k=3** — 47 SOREs, 12% coverage + +### 2. Two-dimensional doesn't help +- Adding file path to behavioral grouping **reduces** performance +- The structural component drags down the behavioral signal +- Conclusion: **drop the structural dimension entirely** + +### 3. The successful SOREs reveal real patterns + +**Test patterns** (most common): +- `post.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?` — HTTP test flow +- `runTest.post.uri.bodyValue.(ChatRequest|SessionChatRequest).exchange.expectStatus` — test setup +- `every.getJobStatus.get.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?` — polling pattern + +**Assertion patterns**: +- `assertEquals.(of.(assertFailsWith)?)+` — assertion chain +- `filesIn.assertTrue.(hasImport)+` — file validation +- `trim.lowercase.(warn)+` — string processing + +**Mock patterns**: +- `mockk.every....` — mock setup +- `clearAllMocks` — cleanup + +**CRUD patterns**: +- `info.deleteByJobId` — delete operation +- `info.deleteByKnowledgeBaseId` — delete operation +- `update.trimIndent.now.insertRow` — update operation + +### 4. Why coverage is still low (12%) +- Most methods have **unique call patterns** — they don't share a 3-symbol prefix with any other method +- The 3-symbol prefix is too specific for diverse codebases +- Need either: + - More data (more codebases to train on) + - Looser grouping (k=1 or k=2, but with merging) + - Symbol coarsening (collapse specific symbols into categories) + +### 5. Smart filters work well +- `max_unique_ratio=0.85`: Skips 60-80% of groups (too diverse) +- `max_alphabet=20`: Skips groups with too many unique symbols +- `max_soa_edges=100`: Skips complex SOAs that would be slow +- **Result**: Experiment runs in <1s instead of minutes + +## Successful SORE Examples (Top 10 by group size) + +| Context | Methods | Unique Ratio | SORE | +|---------|---------|--------------|------| +| `ery {.stKnowledgeBases().gRequest(m` | 12 | 0.333 | `ery {.stKnowledgeBases().gRequest(m.(ckKnowledgeBase(re.ertEquals(Kn."k\|eckKnowledgeBase(r.(sertEquals(K.("\|(sertEquals(` | +| `mockk` | 11 | 0.091 | `mockk` | +| `filesIn.assertTrue.hasImport` | 7 | 0.286 | `filesIn.assertTrue.(hasImport)+` | +| `of` | 7 | 0.143 | `of` | +| `assertEquals.of.assertFailsWith` | 7 | 0.429 | `assertEquals.(of.(assertFailsWith)?)+` | +| `filesIn.filter.contains` | 5 | 0.400 | `filesIn.filter.contains.assertTrue.(hasImport)+` | +| `trim.lowercase.warn` | 5 | 0.400 | `trim.lowercase.(warn)+` | +| `DoclingConfig.assertThatThrownBy.validateCriticalSettings` | 5 | 0.200 | `DoclingConfig.assertThatThrownBy.validateCriticalSettings.isInstanceOf.hasMessageContaining` | +| `defaultCapabilityId` | 4 | 0.250 | `defaultCapabilityId` | +| `clearAllMocks` | 4 | 0.250 | `clearAllMocks` | + +## Next Steps + +1. **Symbol coarsening**: Collapse specific symbols into categories (GETTER, SETTER, BUILDER, etc.) to reduce fragmentation +2. **Hierarchical grouping**: Group by k=1 first, then sub-group by k=2 within each group +3. **Merge similar contexts**: After initial grouping, merge contexts with similar SOREs (Bex's Reduce algorithm) +4. **Test on more codebases**: RAGSAK may not be representative — need more data +5. **Accept the limit**: Some methods are genuinely unique — 12% coverage may be the ceiling for this approach + +## Files Generated + +- `experiments/results/summary.json` — metrics for all strategies +- `experiments/results/*.json` — detailed results per strategy +- `experiments/context_eval.py` — experiment runner script +- `Papers/02-06-2026/context-design-analysis.md` — analysis of all options +- `Papers/02-06-2026/bex-research-summary.md` — summary of Bex's papers diff --git a/experiments/context_eval.py b/experiments/context_eval.py new file mode 100644 index 0000000..e2b9eed --- /dev/null +++ b/experiments/context_eval.py @@ -0,0 +1,300 @@ +"""Context Strategy Experiments — Test all context strategies on RAGSAK. + +Preserves results in experiments/results/ for future reference. +""" + +import json +import os +import time +from collections import defaultdict, Counter +from pathlib import Path + +from bex.tag_preprocessor.analyze import ( + _preprocess_files, _file_to_package, frequency_filter, scan_directory +) +from bex.twotinf import build_soa +from bex.rwr0 import rwr0 +from bex.crx import CRX +from bex.idregex import idregex +from bex.mdl import lang_size_score + + +BASE = "/home/tobi/Desktop/kesai/RAGSAK" +RESULTS_DIR = Path(__file__).parent / "results" + + +def load_data(): + """Load all Kotlin files from RAGSAK.""" + groups = scan_directory(BASE) + kt_files = groups.get(".kt", []) + sequences, seq_files = _preprocess_files(kt_files) + symbol_seqs = [[text for _, text, _ in seq] for seq in sequences] + packages = [_file_to_package(fp, BASE) for fp in seq_files] + return symbol_seqs, packages, seq_files + + +def context_file_path_k(symbol_seqs, packages, k): + """Option A: Group by last k components of file path.""" + contexts = defaultdict(list) + for i, seq in enumerate(symbol_seqs): + pkg = packages[i] + components = pkg.split("/") + ctx = tuple(components[-k:]) if len(components) >= k else tuple(components) + contexts[ctx].append(seq) + return contexts + + +def context_first_k_symbols(symbol_seqs, k): + """Option B: Group by first k symbols of call sequence.""" + contexts = defaultdict(list) + for seq in symbol_seqs: + if not seq: + contexts[("ε",)].append(seq) + else: + ctx = tuple(seq[:k]) + contexts[ctx].append(seq) + return contexts + + +def context_two_d(symbol_seqs, packages, path_k, sym_k): + """Option C: Two-dimensional (path_k, first_k_symbols).""" + contexts = defaultdict(list) + for i, seq in enumerate(symbol_seqs): + pkg = packages[i] + components = pkg.split("/") + path_ctx = tuple(components[-path_k:]) if len(components) >= path_k else tuple(components) + if not seq: + sym_ctx = ("ε",) + else: + sym_ctx = tuple(seq[:sym_k]) + ctx = path_ctx + sym_ctx + contexts[ctx].append(seq) + return contexts + + +def context_return_type_heuristic(symbol_seqs): + """Option H: Group by return type heuristic (based on last symbol).""" + contexts = defaultdict(list) + for seq in symbol_seqs: + if not seq: + contexts[("ε",)].append(seq) + else: + # Heuristic: last symbol often indicates return type + last = seq[-1] + if last.startswith("return"): + ctx = ("RETURN_" + last.split()[1] if len(last.split()) > 1 else "RETURN_OTHER",) + elif last in ("true", "false"): + ctx = ("RETURN_BOOL",) + elif last.startswith("set") or last.startswith("put"): + ctx = ("SIDE_EFFECT",) + else: + ctx = ("RETURN_VALUE",) + contexts[ctx].append(seq) + return contexts + + +def evaluate_context(contexts, label, min_methods=3, max_methods=50, + max_unique_ratio=0.85, max_alphabet=20, max_soa_edges=100): + """Evaluate a context grouping: learn SOREs, collect metrics. + + Smart filters before attempting REWRITE: + - max_methods: skip groups too large (will fail, waste time) + - max_unique_ratio: skip if sequences are too diverse + - max_alphabet: skip if too many unique symbols (SORE can't compress) + - max_soa_edges: skip if SOA is too complex (REWRITE will be slow) + """ + results = { + "strategy": label, + "total_contexts": len(contexts), + "meaningful_contexts": 0, + "total_methods": 0, + "methods_in_good_groups": 0, + "sore_successes": 0, + "sore_failures": 0, + "skip_reasons": {}, + "groups": [], + } + + for ctx, seqs in sorted(contexts.items(), key=lambda x: -len(x[1])): + n = len(seqs) + results["total_methods"] += n + if n < min_methods: + continue + results["meaningful_contexts"] += 1 + + unique = len(set(tuple(s) for s in seqs)) + unique_ratio = unique / n if n else 1.0 + + # Filter: too many methods + if n > max_methods: + results["skip_reasons"]["too_large"] = results["skip_reasons"].get("too_large", 0) + 1 + results["groups"].append({ + "context": str(ctx), "methods": n, "unique": unique, + "unique_ratio": round(unique_ratio, 3), + "sore": "SKIP(too_large)", "sore_success": False, + }) + continue + + # Filter: too diverse + if unique_ratio > max_unique_ratio: + results["skip_reasons"]["too_diverse"] = results["skip_reasons"].get("too_diverse", 0) + 1 + results["groups"].append({ + "context": str(ctx), "methods": n, "unique": unique, + "unique_ratio": round(unique_ratio, 3), + "sore": "SKIP(too_diverse)", "sore_success": False, + }) + continue + + # Filter: too many unique symbols + alphabet = set() + for seq in seqs: + alphabet.update(seq) + if len(alphabet) > max_alphabet: + results["skip_reasons"]["large_alphabet"] = results["skip_reasons"].get("large_alphabet", 0) + 1 + results["groups"].append({ + "context": str(ctx), "methods": n, "unique": unique, + "unique_ratio": round(unique_ratio, 3), + "sore": "SKIP(large_alphabet)", "sore_success": False, + }) + continue + + # Filter: rare symbols + filtered = frequency_filter( + [[(j, s, 0) for j, s in enumerate(seq)] for seq in seqs], + min_coverage=0.0, + ) + clean = [[text for _, text, _ in r] for r in filtered] + clean = [s for s in clean if s] + + if len(clean) < 2: + results["skip_reasons"]["empty_after_filter"] = results["skip_reasons"].get("empty_after_filter", 0) + 1 + continue + + # Build SOA and check complexity before REWRITE + soa = build_soa(clean) + n_edges = sum(len(v) for v in soa._succ.values()) + if n_edges > max_soa_edges: + results["skip_reasons"]["complex_soa"] = results["skip_reasons"].get("complex_soa", 0) + 1 + results["groups"].append({ + "context": str(ctx), "methods": n, "unique": unique, + "unique_ratio": round(unique_ratio, 3), + "sore": "SKIP(complex_soa)", "sore_success": False, + }) + continue + + # Learn SORE + sore = rwr0(soa) + + group_info = { + "context": str(ctx), + "methods": n, + "unique": unique, + "unique_ratio": round(unique_ratio, 3), + "sore": sore[:200] if sore not in ("∅",) else sore, + "sore_success": sore != "∅", + } + + if sore != "∅": + results["sore_successes"] += 1 + results["methods_in_good_groups"] += n + else: + results["sore_failures"] += 1 + + results["groups"].append(group_info) + + results["coverage"] = ( + round(results["methods_in_good_groups"] / results["total_methods"] * 100, 1) + if results["total_methods"] > 0 + else 0 + ) + return results + + +def run_experiment(name, contexts, label): + """Run one experiment, save results.""" + print(f"\n{'='*60}") + print(f" {label}") + print(f"{'='*60}") + + t0 = time.time() + results = evaluate_context(contexts, label) + elapsed = time.time() - t0 + results["elapsed_seconds"] = round(elapsed, 2) + + print(f" Contexts: {results['total_contexts']}") + print(f" Meaningful (>=3 methods): {results['meaningful_contexts']}") + if results.get("skip_reasons"): + for reason, count in results["skip_reasons"].items(): + print(f" Skipped ({reason}): {count}") + print(f" SORE successes: {results['sore_successes']}") + print(f" SORE failures: {results['sore_failures']}") + print(f" Coverage: {results['coverage']}%") + print(f" Time: {elapsed:.1f}s") + + # Save + out_path = RESULTS_DIR / f"{name}.json" + with open(out_path, "w") as f: + json.dump(results, f, indent=2) + print(f" Saved: {out_path}") + + return results + + +def main(): + print("Loading RAGSAK data...") + symbol_seqs, packages, seq_files = load_data() + print(f"Loaded {len(symbol_seqs)} methods from {len(set(packages))} packages") + + all_results = [] + + # Baseline: package grouping (current approach) + contexts = defaultdict(list) + for i, seq in enumerate(symbol_seqs): + contexts[packages[i]].append(seq) + all_results.append(run_experiment("baseline_package", contexts, "Baseline: Package grouping")) + + # Option A: File path k-equivalence + for k in [1, 2, 3]: + contexts = context_file_path_k(symbol_seqs, packages, k) + all_results.append(run_experiment(f"file_path_k{k}", contexts, f"Option A: File path k={k}")) + + # Option B: First k symbols + for k in [1, 2, 3]: + contexts = context_first_k_symbols(symbol_seqs, k) + all_results.append(run_experiment(f"first_k_sym_{k}", contexts, f"Option B: First {k} symbols")) + + # Option C: Two-dimensional + for path_k in [1, 2]: + for sym_k in [1, 2]: + contexts = context_two_d(symbol_seqs, packages, path_k, sym_k) + all_results.append(run_experiment( + f"two_d_p{path_k}_s{sym_k}", contexts, + f"Option C: Path k={path_k} + Symbol k={sym_k}" + )) + + # Option H: Return type heuristic + contexts = context_return_type_heuristic(symbol_seqs) + all_results.append(run_experiment("return_type_heuristic", contexts, "Option H: Return type heuristic")) + + # Summary + print("\n" + "=" * 80) + print(" SUMMARY") + print("=" * 80) + print(f"{'Strategy':<40} {'Contexts':>8} {'SORE OK':>8} {'Coverage':>8}") + print("-" * 80) + for r in all_results: + print(f"{r['strategy']:<40} {r['meaningful_contexts']:>8} {r['sore_successes']:>8} {r['coverage']:>7}%") + + # Save summary + summary = [ + {k: v for k, v in r.items() if k != "groups"} + for r in all_results + ] + summary_path = RESULTS_DIR / "summary.json" + with open(summary_path, "w") as f: + json.dump(summary, f, indent=2) + print(f"\nSummary saved: {summary_path}") + + +if __name__ == "__main__": + main() diff --git a/experiments/results/baseline_package.json b/experiments/results/baseline_package.json new file mode 100644 index 0000000..196c2fd --- /dev/null +++ b/experiments/results/baseline_package.json @@ -0,0 +1,946 @@ +{ + "strategy": "Baseline: Package grouping", + "total_contexts": 137, + "meaningful_contexts": 116, + "total_methods": 1594, + "methods_in_good_groups": 9, + "sore_successes": 2, + "sore_failures": 10, + "skip_reasons": { + "too_large": 5, + "too_diverse": 89, + "large_alphabet": 10 + }, + "groups": [ + { + "context": "app/src/test/kotlin/eu/corentic/springrag/architecture", + "methods": 87, + "unique": 70, + "unique_ratio": 0.805, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/test/kotlin/eu/corentic/springrag/controller", + "methods": 83, + "unique": 74, + "unique_ratio": 0.892, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/service/job", + "methods": 61, + "unique": 55, + "unique_ratio": 0.902, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/main/kotlin/eu/corentic/springrag/controller", + "methods": 56, + "unique": 53, + "unique_ratio": 0.946, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "agents/rag/embabel/src/main/kotlin/eu/corentic/springrag/agent/rag/embabel", + "methods": 52, + "unique": 52, + "unique_ratio": 1.0, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/service/job", + "methods": 49, + "unique": 48, + "unique_ratio": 0.98, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "app/src/systemTest/kotlin/eu/corentic/springrag/system", + "methods": 46, + "unique": 38, + "unique_ratio": 0.826, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/test/kotlin/eu/corentic/springrag/controller/librechat", + "methods": 46, + "unique": 40, + "unique_ratio": 0.87, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/service/port/adapter", + "methods": 45, + "unique": 44, + "unique_ratio": 0.978, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/service/storage", + "methods": 40, + "unique": 27, + "unique_ratio": 0.675, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "agents/rag/embabel/src/test/kotlin/eu/corentic/springrag/agent/rag/embabel", + "methods": 35, + "unique": 25, + "unique_ratio": 0.714, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch/listener", + "methods": 34, + "unique": 31, + "unique_ratio": 0.912, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/integrationTest/kotlin/eu/corentic/springrag/service/job", + "methods": 33, + "unique": 30, + "unique_ratio": 0.909, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/service/graph", + "methods": 30, + "unique": 25, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/main/kotlin/eu/corentic/springrag/controller/librechat", + "methods": 29, + "unique": 29, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/service/port/adapter", + "methods": 28, + "unique": 27, + "unique_ratio": 0.964, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/config", + "methods": 26, + "unique": 23, + "unique_ratio": 0.885, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/repository", + "methods": 24, + "unique": 21, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/systemTest/kotlin/eu/corentic/springrag/repository/graph", + "methods": 23, + "unique": 22, + "unique_ratio": 0.957, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/main/kotlin/eu/corentic/springrag/service/docling", + "methods": 21, + "unique": 18, + "unique_ratio": 0.857, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/repository", + "methods": 19, + "unique": 18, + "unique_ratio": 0.947, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/common/src/main/kotlin/eu/corentic/springrag/common/ids", + "methods": 18, + "unique": 12, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "app/src/e2eTest/kotlin/eu/corentic/springrag/e2e/cucumber/steps", + "methods": 17, + "unique": 13, + "unique_ratio": 0.765, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/service/chunk", + "methods": 17, + "unique": 17, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch", + "methods": 17, + "unique": 17, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/batch/listener", + "methods": 16, + "unique": 13, + "unique_ratio": 0.812, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "modules/knowledgebase/src/test/kotlin/eu/corentic/springrag/service/knowledgebase", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "app/src/integrationTest/kotlin/eu/corentic/springrag/config", + "methods": 15, + "unique": 5, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "modules/chat/src/systemTest/kotlin/eu/corentic/springrag/config", + "methods": 15, + "unique": 14, + "unique_ratio": 0.933, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/batch", + "methods": 15, + "unique": 15, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/security/src/test/kotlin/eu/corentic/springrag/security/service", + "methods": 15, + "unique": 12, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "agents/examples/simple-rag/src/test/kotlin/eu/corentic/springrag/example/rag/simple", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/summarizer/embabel/src/test/kotlin/eu/corentic/springrag/agent/summarizer/embabel", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "app/src/systemTest/kotlin/eu/corentic/springrag/config", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch/writer", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/embabel/src/main/kotlin/eu/corentic/springrag/agent/rag/embabel/librechat", + "methods": 13, + "unique": 13, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "platform/test-support/src/main/kotlin/eu/corentic/springrag/testcontainers", + "methods": 13, + "unique": 13, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/support/src/main/kotlin/eu/corentic/springrag/agent/rag/support", + "methods": 12, + "unique": 11, + "unique_ratio": 0.917, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/summarizer/embabel/src/main/kotlin/eu/corentic/springrag/agent/summarizer/embabel", + "methods": 12, + "unique": 12, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/tooling/embabel/src/main/kotlin/eu/corentic/springrag/agent/tooling/embabel", + "methods": 12, + "unique": 12, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/test/kotlin/eu/corentic/springrag/config", + "methods": 12, + "unique": 6, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/service/graph", + "methods": 12, + "unique": 12, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch/reader", + "methods": 12, + "unique": 11, + "unique_ratio": 0.917, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/support/src/test/kotlin/eu/corentic/springrag/agent/rag/support", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/tooling/embabel/src/main/kotlin/eu/corentic/springrag/agent/tooling/embabel/recording", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/knowledgebase/src/main/kotlin/eu/corentic/springrag/service/knowledgebase", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/test/kotlin/eu/corentic/springrag/controller/web", + "methods": 10, + "unique": 10, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/test/kotlin/eu/corentic/springrag/service/docling", + "methods": 10, + "unique": 10, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/security/src/main/kotlin/eu/corentic/springrag/security/service", + "methods": 10, + "unique": 10, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/examples/simple-rag/src/main/kotlin/eu/corentic/springrag/example/rag/simple", + "methods": 9, + "unique": 9, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/model/graph", + "methods": 9, + "unique": 9, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/wikipedia/adapters/src/test/kotlin/eu/corentic/springrag/agent/wikipedia", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "buildSrc/src/main/kotlin", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/chat/src/test/kotlin/eu/corentic/springrag/service/chat", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/common/src/test/kotlin/eu/corentic/springrag/common/ids", + "methods": 8, + "unique": 4, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch/model", + "methods": 8, + "unique": 6, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/service/chunk", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/model/graph", + "methods": 7, + "unique": 6, + "unique_ratio": 0.857, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch/partition", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/capability/support/src/test/kotlin/eu/corentic/springrag/agent/capability/support", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "agents/tooling/embabel/src/test/kotlin/eu/corentic/springrag/agent/tooling/embabel", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/main/kotlin/eu/corentic/springrag/controller/web", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/ai-provider/src/test/kotlin/eu/corentic/springrag/config", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/systemTest/kotlin/eu/corentic/springrag/config", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/health", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/health", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/systemTest/kotlin/eu/corentic/springrag/system", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/chat/src/main/kotlin/eu/corentic/springrag/service/chat", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "modules/common/src/main/kotlin/eu/corentic/springrag/common", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/security/src/test/kotlin/eu/corentic/springrag/security/config", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/capability/ports/src/main/kotlin/eu/corentic/springrag/agent/capability", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "(return AgentCapabilityDescriptor(\n id = id,\n capabilityType = capabilityType,\n displayName = describedCapability?.displayName ?: id,\n description = describedCapability?.de", + "sore_success": true + }, + { + "context": "agents/capability/support/src/main/kotlin/eu/corentic/springrag/agent/capability/support", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "app/src/integrationTest/kotlin/eu/corentic/springrag/controller", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "buildSrc/src/test/kotlin", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/test/kotlin/eu/corentic/springrag/controller/auth", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/ai-provider/src/test/kotlin/eu/corentic/springrag/service/embedding", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/chat-memory/src/test/kotlin/eu/corentic/springrag/config", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/test/kotlin/eu/corentic/springrag/batch/processor", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/knowledgebase/ports/src/main/kotlin/eu/corentic/springrag/model", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/security/src/main/kotlin/eu/corentic/springrag/security/config", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "platform/test-support/src/test/kotlin/eu/corentic/springrag/testcontainers", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/ports/src/main/kotlin/eu/corentic/springrag/agent/rag", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/tooling/agent/src/main/kotlin/eu/corentic/springrag/agent/tooling/agent", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/test/kotlin/eu/corentic/springrag/config", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/ai-provider/src/main/kotlin/eu/corentic/springrag/config", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/ai-provider/src/test/kotlin/eu/corentic/springrag/health", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/service/cleanup", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "info.(deleteByJobId|deleteByKnowledgeBaseId)", + "sore_success": true + }, + { + "context": "infrastructure/adapters/search/src/systemTest/kotlin/eu/corentic/springrag/config", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/service/cleanup", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/chat/src/integrationTest/kotlin/eu/corentic/springrag/service/chat", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/chat/src/systemTest/kotlin/eu/corentic/springrag/system", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/batch/model", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/batch/partition", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/batch/reader", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/embabel/src/test/kotlin/eu/corentic/springrag/testsupport", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/ports/src/test/kotlin/eu/corentic/springrag/agent/rag", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/rag/ports/src/test/kotlin/eu/corentic/springrag/model", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/wikipedia/adapters/src/main/kotlin/eu/corentic/springrag/agent/wikipedia", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/wikipedia/static/src/test/kotlin/eu/corentic/springrag/agent/wikipedia", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "agents/wikipedia/static/src/main/kotlin/eu/corentic/springrag/agent/wikipedia", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "app/src/integrationTest/kotlin/eu/corentic/springrag/service", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "app/src/test/kotlin/eu/corentic/springrag/config", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/agent-mcp-server/src/main/kotlin/eu/corentic/springrag/mcp", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/agent-mcp-server/src/test/kotlin/eu/corentic/springrag/mcp", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "entrypoints/web/src/main/kotlin/eu/corentic/springrag/config", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/ai-provider/src/main/kotlin/eu/corentic/springrag/health", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/ai-provider/src/test/kotlin/eu/corentic/springrag/testsupport", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/main/kotlin/eu/corentic/springrag/config", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/test/kotlin/eu/corentic/springrag/health", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/doc-parser/src/test/kotlin/eu/corentic/springrag/testsupport", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/main/kotlin/eu/corentic/springrag/service/agent", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/config", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/service/storage", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "infrastructure/adapters/search/src/test/kotlin/eu/corentic/springrag/testsupport", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/batch/writer", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "modules/ingestion/src/main/kotlin/eu/corentic/springrag/service/document", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 0.6, + "elapsed_seconds": 0.07 +} \ No newline at end of file diff --git a/experiments/results/file_path_k1.json b/experiments/results/file_path_k1.json new file mode 100644 index 0000000..e865dfe --- /dev/null +++ b/experiments/results/file_path_k1.json @@ -0,0 +1,362 @@ +{ + "strategy": "Option A: File path k=1", + "total_contexts": 47, + "meaningful_contexts": 43, + "total_methods": 1594, + "methods_in_good_groups": 17, + "sore_successes": 2, + "sore_failures": 1, + "skip_reasons": { + "too_large": 9, + "too_diverse": 28, + "large_alphabet": 3 + }, + "groups": [ + { + "context": "('controller',)", + "methods": 144, + "unique": 132, + "unique_ratio": 0.917, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('job',)", + "methods": 143, + "unique": 133, + "unique_ratio": 0.93, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('config',)", + "methods": 140, + "unique": 97, + "unique_ratio": 0.693, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('embabel',)", + "methods": 131, + "unique": 119, + "unique_ratio": 0.908, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('librechat',)", + "methods": 89, + "unique": 83, + "unique_ratio": 0.933, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('architecture',)", + "methods": 87, + "unique": 70, + "unique_ratio": 0.805, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('graph',)", + "methods": 81, + "unique": 74, + "unique_ratio": 0.914, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('adapter',)", + "methods": 76, + "unique": 74, + "unique_ratio": 0.974, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('system',)", + "methods": 58, + "unique": 50, + "unique_ratio": 0.862, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('listener',)", + "methods": 50, + "unique": 44, + "unique_ratio": 0.88, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository',)", + "methods": 43, + "unique": 39, + "unique_ratio": 0.907, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('storage',)", + "methods": 43, + "unique": 29, + "unique_ratio": 0.674, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('support',)", + "methods": 34, + "unique": 32, + "unique_ratio": 0.941, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch',)", + "methods": 32, + "unique": 32, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('docling',)", + "methods": 31, + "unique": 28, + "unique_ratio": 0.903, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service',)", + "methods": 28, + "unique": 25, + "unique_ratio": 0.893, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase',)", + "methods": 27, + "unique": 27, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ids',)", + "methods": 26, + "unique": 16, + "unique_ratio": 0.615, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('chunk',)", + "methods": 25, + "unique": 25, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('health',)", + "methods": 24, + "unique": 22, + "unique_ratio": 0.917, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('simple',)", + "methods": 23, + "unique": 23, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('model',)", + "methods": 23, + "unique": 21, + "unique_ratio": 0.913, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('wikipedia',)", + "methods": 18, + "unique": 17, + "unique_ratio": 0.944, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('chat',)", + "methods": 18, + "unique": 17, + "unique_ratio": 0.944, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('testcontainers',)", + "methods": 18, + "unique": 18, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('steps',)", + "methods": 17, + "unique": 13, + "unique_ratio": 0.765, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('writer',)", + "methods": 17, + "unique": 17, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('web',)", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('reader',)", + "methods": 16, + "unique": 15, + "unique_ratio": 0.938, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('kotlin',)", + "methods": 13, + "unique": 13, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('testsupport',)", + "methods": 12, + "unique": 3, + "unique_ratio": 0.25, + "sore": "(get|(Builder|Any))", + "sore_success": true + }, + { + "context": "('recording',)", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('partition',)", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent',)", + "methods": 9, + "unique": 9, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cleanup',)", + "methods": 8, + "unique": 6, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('rag',)", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('auth',)", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mcp',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('common',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('processor',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('capability',)", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "(return AgentCapabilityDescriptor(\n id = id,\n capabilityType = capabilityType,\n displayName = describedCapability?.displayName ?: id,\n description = describedCapability?.de", + "sore_success": true + }, + { + "context": "('embedding',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('document',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 1.1, + "elapsed_seconds": 0.01 +} \ No newline at end of file diff --git a/experiments/results/file_path_k2.json b/experiments/results/file_path_k2.json new file mode 100644 index 0000000..d4e7c13 --- /dev/null +++ b/experiments/results/file_path_k2.json @@ -0,0 +1,450 @@ +{ + "strategy": "Option A: File path k=2", + "total_contexts": 59, + "meaningful_contexts": 54, + "total_methods": 1594, + "methods_in_good_groups": 17, + "sore_successes": 2, + "sore_failures": 1, + "skip_reasons": { + "too_large": 8, + "too_diverse": 39, + "large_alphabet": 4 + }, + "groups": [ + { + "context": "('springrag', 'controller')", + "methods": 144, + "unique": 132, + "unique_ratio": 0.917, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('service', 'job')", + "methods": 143, + "unique": 133, + "unique_ratio": 0.93, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('springrag', 'config')", + "methods": 128, + "unique": 85, + "unique_ratio": 0.664, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('rag', 'embabel')", + "methods": 87, + "unique": 77, + "unique_ratio": 0.885, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture')", + "methods": 87, + "unique": 70, + "unique_ratio": 0.805, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('port', 'adapter')", + "methods": 76, + "unique": 74, + "unique_ratio": 0.974, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('controller', 'librechat')", + "methods": 75, + "unique": 69, + "unique_ratio": 0.92, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('springrag', 'system')", + "methods": 58, + "unique": 50, + "unique_ratio": 0.862, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('batch', 'listener')", + "methods": 50, + "unique": 44, + "unique_ratio": 0.88, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'repository')", + "methods": 43, + "unique": 39, + "unique_ratio": 0.907, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'storage')", + "methods": 43, + "unique": 29, + "unique_ratio": 0.674, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('service', 'graph')", + "methods": 42, + "unique": 37, + "unique_ratio": 0.881, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'batch')", + "methods": 32, + "unique": 32, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'docling')", + "methods": 31, + "unique": 28, + "unique_ratio": 0.903, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase')", + "methods": 27, + "unique": 27, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('summarizer', 'embabel')", + "methods": 26, + "unique": 26, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('common', 'ids')", + "methods": 26, + "unique": 16, + "unique_ratio": 0.615, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('service', 'chunk')", + "methods": 25, + "unique": 25, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'service')", + "methods": 25, + "unique": 22, + "unique_ratio": 0.88, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'health')", + "methods": 24, + "unique": 22, + "unique_ratio": 0.917, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'simple')", + "methods": 23, + "unique": 23, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'support')", + "methods": 23, + "unique": 22, + "unique_ratio": 0.957, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'graph')", + "methods": 23, + "unique": 22, + "unique_ratio": 0.957, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'wikipedia')", + "methods": 18, + "unique": 17, + "unique_ratio": 0.944, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('tooling', 'embabel')", + "methods": 18, + "unique": 18, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'chat')", + "methods": 18, + "unique": 17, + "unique_ratio": 0.944, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'testcontainers')", + "methods": 18, + "unique": 18, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cucumber', 'steps')", + "methods": 17, + "unique": 13, + "unique_ratio": 0.765, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('batch', 'writer')", + "methods": 17, + "unique": 17, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'web')", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('model', 'graph')", + "methods": 16, + "unique": 15, + "unique_ratio": 0.938, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'reader')", + "methods": 16, + "unique": 15, + "unique_ratio": 0.938, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'librechat')", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'testsupport')", + "methods": 12, + "unique": 3, + "unique_ratio": 0.25, + "sore": "(get|(Builder|Any))", + "sore_success": true + }, + { + "context": "('batch', 'model')", + "methods": 12, + "unique": 10, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('capability', 'support')", + "methods": 11, + "unique": 10, + "unique_ratio": 0.909, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'model')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'recording')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'partition')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'config')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('main', 'kotlin')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'cleanup')", + "methods": 8, + "unique": 6, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('agent', 'rag')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'auth')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('tooling', 'agent')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'mcp')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'common')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'processor')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'capability')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "(return AgentCapabilityDescriptor(\n id = id,\n capabilityType = capabilityType,\n displayName = describedCapability?.displayName ?: id,\n description = describedCapability?.de", + "sore_success": true + }, + { + "context": "('test', 'kotlin')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'embedding')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'agent')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'document')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 1.1, + "elapsed_seconds": 0.01 +} \ No newline at end of file diff --git a/experiments/results/file_path_k3.json b/experiments/results/file_path_k3.json new file mode 100644 index 0000000..924070d --- /dev/null +++ b/experiments/results/file_path_k3.json @@ -0,0 +1,450 @@ +{ + "strategy": "Option A: File path k=3", + "total_contexts": 59, + "meaningful_contexts": 54, + "total_methods": 1594, + "methods_in_good_groups": 17, + "sore_successes": 2, + "sore_failures": 1, + "skip_reasons": { + "too_large": 8, + "too_diverse": 39, + "large_alphabet": 4 + }, + "groups": [ + { + "context": "('corentic', 'springrag', 'controller')", + "methods": 144, + "unique": 132, + "unique_ratio": 0.917, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'job')", + "methods": 143, + "unique": 133, + "unique_ratio": 0.93, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'config')", + "methods": 128, + "unique": 85, + "unique_ratio": 0.664, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('agent', 'rag', 'embabel')", + "methods": 87, + "unique": 77, + "unique_ratio": 0.885, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'architecture')", + "methods": 87, + "unique": 70, + "unique_ratio": 0.805, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('service', 'port', 'adapter')", + "methods": 76, + "unique": 74, + "unique_ratio": 0.974, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'librechat')", + "methods": 75, + "unique": 69, + "unique_ratio": 0.92, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'system')", + "methods": 58, + "unique": 50, + "unique_ratio": 0.862, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'listener')", + "methods": 50, + "unique": 44, + "unique_ratio": 0.88, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'repository')", + "methods": 43, + "unique": 39, + "unique_ratio": 0.907, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'storage')", + "methods": 43, + "unique": 29, + "unique_ratio": 0.674, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'graph')", + "methods": 42, + "unique": 37, + "unique_ratio": 0.881, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'batch')", + "methods": 32, + "unique": 32, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'docling')", + "methods": 31, + "unique": 28, + "unique_ratio": 0.903, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'knowledgebase')", + "methods": 27, + "unique": 27, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'summarizer', 'embabel')", + "methods": 26, + "unique": 26, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'common', 'ids')", + "methods": 26, + "unique": 16, + "unique_ratio": 0.615, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'chunk')", + "methods": 25, + "unique": 25, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'security', 'service')", + "methods": 25, + "unique": 22, + "unique_ratio": 0.88, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'health')", + "methods": 24, + "unique": 22, + "unique_ratio": 0.917, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('example', 'rag', 'simple')", + "methods": 23, + "unique": 23, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'rag', 'support')", + "methods": 23, + "unique": 22, + "unique_ratio": 0.957, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'graph')", + "methods": 23, + "unique": 22, + "unique_ratio": 0.957, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'agent', 'wikipedia')", + "methods": 18, + "unique": 17, + "unique_ratio": 0.944, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'tooling', 'embabel')", + "methods": 18, + "unique": 18, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'chat')", + "methods": 18, + "unique": 17, + "unique_ratio": 0.944, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'testcontainers')", + "methods": 18, + "unique": 18, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('e2e', 'cucumber', 'steps')", + "methods": 17, + "unique": 13, + "unique_ratio": 0.765, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'writer')", + "methods": 17, + "unique": 17, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'web')", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'model', 'graph')", + "methods": 16, + "unique": 15, + "unique_ratio": 0.938, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'reader')", + "methods": 16, + "unique": 15, + "unique_ratio": 0.938, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'librechat')", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'testsupport')", + "methods": 12, + "unique": 3, + "unique_ratio": 0.25, + "sore": "(get|(Builder|Any))", + "sore_success": true + }, + { + "context": "('springrag', 'batch', 'model')", + "methods": 12, + "unique": 10, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('agent', 'capability', 'support')", + "methods": 11, + "unique": 10, + "unique_ratio": 0.909, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'model')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('tooling', 'embabel', 'recording')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'partition')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'security', 'config')", + "methods": 11, + "unique": 11, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('src', 'main', 'kotlin')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'cleanup')", + "methods": 8, + "unique": 6, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'agent', 'rag')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'auth')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'tooling', 'agent')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'mcp')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'common')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'processor')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'agent', 'capability')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "(return AgentCapabilityDescriptor(\n id = id,\n capabilityType = capabilityType,\n displayName = describedCapability?.displayName ?: id,\n description = describedCapability?.de", + "sore_success": true + }, + { + "context": "('src', 'test', 'kotlin')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'embedding')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('corentic', 'springrag', 'service')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'agent')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'service', 'document')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 1.1, + "elapsed_seconds": 0.01 +} \ No newline at end of file diff --git a/experiments/results/first_k_sym_1.json b/experiments/results/first_k_sym_1.json new file mode 100644 index 0000000..5de736a --- /dev/null +++ b/experiments/results/first_k_sym_1.json @@ -0,0 +1,1090 @@ +{ + "strategy": "Option B: First 1 symbols", + "total_contexts": 550, + "meaningful_contexts": 134, + "total_methods": 1594, + "methods_in_good_groups": 74, + "sore_successes": 20, + "sore_failures": 19, + "skip_reasons": { + "too_large": 2, + "large_alphabet": 13, + "too_diverse": 80 + }, + "groups": [ + { + "context": "('every',)", + "methods": 102, + "unique": 93, + "unique_ratio": 0.912, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('runTest',)", + "methods": 59, + "unique": 54, + "unique_ratio": 0.915, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('listOf',)", + "methods": 36, + "unique": 27, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('mockk',)", + "methods": 31, + "unique": 21, + "unique_ratio": 0.677, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('runBlocking',)", + "methods": 29, + "unique": 28, + "unique_ratio": 0.966, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('trim',)", + "methods": 26, + "unique": 23, + "unique_ratio": 0.885, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('warn',)", + "methods": 26, + "unique": 25, + "unique_ratio": 0.962, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('builder',)", + "methods": 24, + "unique": 23, + "unique_ratio": 0.958, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('info',)", + "methods": 23, + "unique": 19, + "unique_ratio": 0.826, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('parse',)", + "methods": 18, + "unique": 18, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('`when`',)", + "methods": 17, + "unique": 15, + "unique_ratio": 0.882, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('productionClasses',)", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('filesIn',)", + "methods": 16, + "unique": 7, + "unique_ratio": 0.438, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('VectorChunk',)", + "methods": 15, + "unique": 13, + "unique_ratio": 0.867, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('File',)", + "methods": 15, + "unique": 14, + "unique_ratio": 0.933, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('isEmpty',)", + "methods": 14, + "unique": 14, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('debug',)", + "methods": 14, + "unique": 13, + "unique_ratio": 0.929, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('of',)", + "methods": 14, + "unique": 8, + "unique_ratio": 0.571, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('ery {',)", + "methods": 13, + "unique": 5, + "unique_ratio": 0.385, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('forEach',)", + "methods": 13, + "unique": 12, + "unique_ratio": 0.923, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('createKnowledgeBase',)", + "methods": 11, + "unique": 10, + "unique_ratio": 0.909, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('asJobId',)", + "methods": 11, + "unique": 5, + "unique_ratio": 0.455, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('input',)", + "methods": 11, + "unique": 10, + "unique_ratio": 0.909, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('isNullOrBlank',)", + "methods": 9, + "unique": 7, + "unique_ratio": 0.778, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('get',)", + "methods": 9, + "unique": 6, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('from',)", + "methods": 9, + "unique": 6, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('DoclingConfig',)", + "methods": 9, + "unique": 3, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('parseStorageUri',)", + "methods": 9, + "unique": 6, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('classify',)", + "methods": 9, + "unique": 8, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobInstance',)", + "methods": 9, + "unique": 9, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('withContext',)", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('isBlank',)", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('assertEquals',)", + "methods": 8, + "unique": 4, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('createJobExecution',)", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JwtService',)", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('joinToString',)", + "methods": 7, + "unique": 5, + "unique_ratio": 0.714, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('buildObservationContext',)", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('randomUUID',)", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('await',)", + "methods": 7, + "unique": 5, + "unique_ratio": 0.714, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('productionFiles',)", + "methods": 7, + "unique": 6, + "unique_ratio": 0.857, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobStatus',)", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('query',)", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('HybridChunkingConfig',)", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('run',)", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('replace',)", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('map',)", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('very',)", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('fun',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('lowercase',)", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('mutableListOf',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('slot',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('MultipartBodyBuilder',)", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('session',)", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('coEvery',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('getMethod',)", + "methods": 6, + "unique": 2, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('resolveLocation',)", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('skipPolicy',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('state',)", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('trimIndent',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ChatResponse',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('let',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('post',)", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('getenv',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('queryForObject',)", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "queryForObject.(trimIndent)?", + "sore_success": true + }, + { + "context": "('assumeTrue',)", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('scopeFromProject',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('assertThrows',)", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "assertThrows.(asJobId|(EmbabelAiHttpClientProperties|OllamaClientProperties.Timeout).ofSeconds)", + "sore_success": true + }, + { + "context": "('now',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ImageData',)", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('upsertStaging',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('update',)", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "update.trimIndent.(now.insertRow)?", + "sore_success": true + }, + { + "context": "('policy',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DocumentInput',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('processedDocument',)", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mapOf',)", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('defaultCapabilityId',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "defaultCapabilityId", + "sore_success": true + }, + { + "context": "('resolve',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('clearAllMocks',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "clearAllMocks", + "sore_success": true + }, + { + "context": "('return true',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mockkObject',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('Builder',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('Any',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('ToolInvocationPolicyProperties',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('return',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('requireNotNull',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('adminClient',)", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('registerProperties',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('newClient',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('getByType',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('eFromProject()',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('error',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('startsWith',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mutableMapOf',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ilder()',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('leteByFilter(f',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('save',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DocumentGraphJob',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DocumentJobNode',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('execute',)", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "execute.(decorate)?", + "sore_success": true + }, + { + "context": "('resolveUploadStorageUri',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('JobParametersBuilder',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('IngestionDocumentState',)", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('createTempFile',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('AgentExecutionContext',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('AgentCapabilityDescriptor',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('EmbabelRagLoop',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('equals',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('messageWindowMemory',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('answer',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('sortedBy',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "sortedBy.map.toDescriptor", + "sore_success": true + }, + { + "context": "('searchSimilar',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('WikipediaLookupResponse',)", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "WikipediaLookupResponse.(coEvery.invoke.any.wikipediaLookup.block.content.first)?.((assertTrue.text.contains)+)?.(@.(Suppress)+)?.(structuredContent.(assertEquals)+)?", + "sore_success": true + }, + { + "context": "('chat',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('recreateTestCollection',)", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "recreateTestCollection.(truncatePostgresTables.clearNeo4jDatabase)?", + "sore_success": true + }, + { + "context": "('readString',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('assertNoMainProjectDependencies',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mono',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('KnowledgeBaseResponse',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "KnowledgeBaseResponse.(now)+", + "sore_success": true + }, + { + "context": "('repeat',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ChatMemoryConfig',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('Empty()',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('deleteByJobId',)", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "deleteByJobId.(asJobId.verify.delete.match)?.((contains)+)?", + "sore_success": true + }, + { + "context": "('buildImageKey',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('loadObject',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "loadObject", + "sore_success": true + }, + { + "context": "('listObjects',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('TextElement',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('nBlocking {',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('asKnowledgeBaseId',)", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('asDocumentId',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('GraphDocument',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('emptyGraphDocument',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('getString',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('StagedUploadCleanupService',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 4.6, + "elapsed_seconds": 0.08 +} \ No newline at end of file diff --git a/experiments/results/first_k_sym_2.json b/experiments/results/first_k_sym_2.json new file mode 100644 index 0000000..1e67ae6 --- /dev/null +++ b/experiments/results/first_k_sym_2.json @@ -0,0 +1,1145 @@ +{ + "strategy": "Option B: First 2 symbols", + "total_contexts": 961, + "meaningful_contexts": 141, + "total_methods": 1594, + "methods_in_good_groups": 157, + "sore_successes": 39, + "sore_failures": 23, + "skip_reasons": { + "large_alphabet": 7, + "too_diverse": 72 + }, + "groups": [ + { + "context": "('listOf', 'listOf')", + "methods": 36, + "unique": 27, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('VectorChunk', 'mapOf')", + "methods": 13, + "unique": 11, + "unique_ratio": 0.846, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('ery {', 'stKnowledgeBases()')", + "methods": 13, + "unique": 5, + "unique_ratio": 0.385, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('productionClasses', 'filter')", + "methods": 13, + "unique": 13, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mockk',)", + "methods": 11, + "unique": 1, + "unique_ratio": 0.091, + "sore": "mockk", + "sore_success": true + }, + { + "context": "('filesIn', 'assertTrue')", + "methods": 11, + "unique": 5, + "unique_ratio": 0.455, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('runTest', 'ChatResponse')", + "methods": 10, + "unique": 10, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'mockk')", + "methods": 9, + "unique": 7, + "unique_ratio": 0.778, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('trim', 'lowercase')", + "methods": 9, + "unique": 6, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('every', 'every')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('File', 'writeBytes')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JwtService', 'JwtProperties')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'getJobStatus')", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('of',)", + "methods": 7, + "unique": 1, + "unique_ratio": 0.143, + "sore": "of", + "sore_success": true + }, + { + "context": "('assertEquals', 'of')", + "methods": 7, + "unique": 3, + "unique_ratio": 0.429, + "sore": "assertEquals.(of.(assertFailsWith)?)+", + "sore_success": true + }, + { + "context": "('from', 'now')", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "((from.now|(trimIndent|debug)).((update)+)?)+", + "sore_success": true + }, + { + "context": "('mockk', 'mockk')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('very', 'istKnowledgeBases(')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('mutableListOf', 'mutableListOf')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('session', 'use')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('runBlocking', 'controller')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('getMethod', 'invoke')", + "methods": 6, + "unique": 2, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('parseStorageUri', 'return false')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "parseStorageUri.return false.((return try {\n s3Client.headObject(\n HeadObjectRequest.builder()\n .bucket(location.bucket)\n .key(location.", + "sore_success": true + }, + { + "context": "('HybridChunkingConfig', 'service')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('parse', 'KnowledgeBase')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'existsByUsername')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('builder', 'build')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('randomUUID', 'toString')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('post', 'uri')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('createKnowledgeBase', 'copyTestDocument')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('MultipartBodyBuilder', 'part')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('await', 'pollInterval')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('filesIn', 'filter')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "filesIn.filter.contains.assertTrue.(hasImport)+", + "sore_success": true + }, + { + "context": "('slot', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'mockFilePart')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'findPageRendering')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('coEvery', 'ingestMultipart')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DoclingConfig', 'assertThatThrownBy')", + "methods": 5, + "unique": 1, + "unique_ratio": 0.2, + "sore": "DoclingConfig.assertThatThrownBy.validateCriticalSettings.isInstanceOf.hasMessageContaining", + "sore_success": true + }, + { + "context": "('parse', 'KnowledgeBaseNode')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mockk', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'existsById')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('upsertStaging', 'of')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('query', 'trimIndent')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('update', 'trimIndent')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "update.trimIndent.(now.insertRow)?", + "sore_success": true + }, + { + "context": "('state', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DocumentInput', 'asJobId')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('processedDocument', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mapOf', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('defaultCapabilityId',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "defaultCapabilityId", + "sore_success": true + }, + { + "context": "('every', 'defaultCapabilityId')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('clearAllMocks',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "clearAllMocks", + "sore_success": true + }, + { + "context": "('mockkObject', 'slot')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('Builder',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('Any',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('get',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "get", + "sore_success": true + }, + { + "context": "('ToolInvocationPolicyProperties', 'mapOf')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('requireNotNull', 'requireNotNull')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mockk', 'also')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('queryForObject',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "queryForObject", + "sore_success": true + }, + { + "context": "('registerProperties',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('assumeTrue', 'isDockerAvailable')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "assumeTrue.isDockerAvailable.start.pullAndWarmup", + "sore_success": true + }, + { + "context": "('eFromProject()', 'ses()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'coEvery')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('JobStatus', 'every')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "JobStatus.every.getJobStatus.get.uri.exchange.expectStatus.expectBody.(jsonPath.isEqualTo)+", + "sore_success": true + }, + { + "context": "('every', 'getOrCreateAgentKnowledgeBase')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('builder', 'inputType')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mutableMapOf', 'mutableMapOf')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('`when`', 'listModels')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('ilder()', 'ery(\"')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('leteByFilter(f', 'lterEquals(M')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('`when`', 'healthCheckAsync')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('DocumentJobNode', 'now')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('parse', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('asJobId', 'asFilename')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('resolveUploadStorageUri',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('JobParametersBuilder', 'addString')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('IngestionDocumentState', 'asDocumentId')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('createTempFile', 'apply')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'fetchTracking')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobInstance', 'JobParametersBuilder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('AgentExecutionContext',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('runTest', 'TestRequest')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('run', 'Supplier')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('map',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "map", + "sore_success": true + }, + { + "context": "('builder', 'chatMemoryRepository')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('sortedBy', 'map')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "sortedBy.map.toDescriptor", + "sore_success": true + }, + { + "context": "('isNullOrBlank', 'error')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "isNullOrBlank.(error)+", + "sore_success": true + }, + { + "context": "('ChatResponse', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'RagInvocation')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('trim', 'isBlank')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'invoke')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('get', 'uri')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('chat',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('run', 'collectionPointCount')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "run.collectionPointCount", + "sore_success": true + }, + { + "context": "('newClient', 'callTool')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('assertNoMainProjectDependencies', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('scopeFromProject', 'filter')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('productionFiles', 'filter')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "productionFiles.filter.(contains.(assertTrue)?)+", + "sore_success": true + }, + { + "context": "('productionClasses', 'withAnnotationOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('joinToString', 'warn')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "joinToString.warn.return ResponseEntity.status(HttpStatus.BAD_REQUEST)\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors)).status.body.ErrorResponse", + "sore_success": true + }, + { + "context": "('runTest', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'post')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "runTest.post.uri.bodyValue.(ChatRequest|SessionChatRequest).exchange.expectStatus", + "sore_success": true + }, + { + "context": "('every', 'knowledgeBaseExists')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('KnowledgeBaseResponse', 'now')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "KnowledgeBaseResponse.(now)+", + "sore_success": true + }, + { + "context": "('JobStatus', 'now')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('assertThrows', 'EmbabelAiHttpClientProperties')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "assertThrows.EmbabelAiHttpClientProperties.ofSeconds", + "sore_success": true + }, + { + "context": "('ChatMemoryConfig', 'chatMemory')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('fun', 'fun')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('`when`', 'health')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('startsWith', 'return null')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DoclingConfig', 'validateCriticalSettings')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "DoclingConfig.validateCriticalSettings", + "sore_success": true + }, + { + "context": "('info', 'deleteByJobId')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "info.deleteByJobId", + "sore_success": true + }, + { + "context": "('info', 'deleteByKnowledgeBaseId')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "info.deleteByKnowledgeBaseId", + "sore_success": true + }, + { + "context": "('buildImageKey', 'return storeObject(key, bytes, contentTypeFor(format))')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('loadObject',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "loadObject", + "sore_success": true + }, + { + "context": "('resolveLocation', 'builder')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('parseStorageUri', 'return null')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('replace', 'ifBlank')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "replace.ifBlank.lowercase.return StorageUri.of(\"images/${jobId.value}/$sanitizedId.$extension\").of", + "sore_success": true + }, + { + "context": "('lowercase',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "lowercase", + "sore_success": true + }, + { + "context": "('isBlank', 'return null')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('runBlocking', 'createTestImage')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('parse', 'save')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'similaritySearch')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'findByJobId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('asDocumentId', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'findById')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('DocumentGraphJob', 'asDocumentId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('asJobId', 'every')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "asJobId.every.(deleteByJobId.(assertDoesNotThrow)?)+", + "sore_success": true + }, + { + "context": "('emptyGraphDocument', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('trim', 'removeSuffix')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runBlocking', 'adminClient')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('execute',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "execute", + "sore_success": true + }, + { + "context": "('every', 'process')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('policy', 'assertTrue')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('createJobExecution', 'createStepExecution')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('input', 'beforeProcess')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('runTest', 'GraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobInstance', 'JobExecution')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'objectExists')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('createJobExecution', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('StagedUploadCleanupService', 'StorageProperties')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 9.8, + "elapsed_seconds": 0.06 +} \ No newline at end of file diff --git a/experiments/results/first_k_sym_3.json b/experiments/results/first_k_sym_3.json new file mode 100644 index 0000000..8e63655 --- /dev/null +++ b/experiments/results/first_k_sym_3.json @@ -0,0 +1,953 @@ +{ + "strategy": "Option B: First 3 symbols", + "total_contexts": 1112, + "meaningful_contexts": 117, + "total_methods": 1594, + "methods_in_good_groups": 191, + "sore_successes": 47, + "sore_failures": 14, + "skip_reasons": { + "large_alphabet": 6, + "too_diverse": 50 + }, + "groups": [ + { + "context": "('VectorChunk', 'mapOf', 'mapOf')", + "methods": 13, + "unique": 11, + "unique_ratio": 0.846, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('ery {', 'stKnowledgeBases()', 'gRequest(m')", + "methods": 12, + "unique": 4, + "unique_ratio": 0.333, + "sore": "ery {.stKnowledgeBases().gRequest(m.(ckKnowledgeBase(re.ertEquals(Kn.\"k|eckKnowledgeBase(r.(sertEquals(K.(\"|(sertEquals(t|sertNull(o)))", + "sore_success": true + }, + { + "context": "('listOf', 'listOf', 'productionFiles')", + "methods": 12, + "unique": 6, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('mockk',)", + "methods": 11, + "unique": 1, + "unique_ratio": 0.091, + "sore": "mockk", + "sore_success": true + }, + { + "context": "('File', 'writeBytes', 'byteArrayOf')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'ChatResponse', 'emptyList')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('filesIn', 'assertTrue', 'hasImport')", + "methods": 7, + "unique": 2, + "unique_ratio": 0.286, + "sore": "filesIn.assertTrue.(hasImport)+", + "sore_success": true + }, + { + "context": "('of',)", + "methods": 7, + "unique": 1, + "unique_ratio": 0.143, + "sore": "of", + "sore_success": true + }, + { + "context": "('assertEquals', 'of', 'assertFailsWith')", + "methods": 7, + "unique": 3, + "unique_ratio": 0.429, + "sore": "assertEquals.(of.(assertFailsWith)?)+", + "sore_success": true + }, + { + "context": "('runTest', 'mockk', 'mockk')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('session', 'use', 'run')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('listOf', 'listOf', 'forEach')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('from', 'now', 'update')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('HybridChunkingConfig', 'service', 'GraphDocument')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('parse', 'KnowledgeBase', 'asKnowledgeBaseId')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('very', 'istKnowledgeBases(', 'agRequest(')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('mockk', 'mockk', 'mockk')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listOf', 'listOf', 'VectorChunk')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('await', 'pollInterval', 'ofSeconds')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('productionClasses', 'filter', 'contains')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('filesIn', 'filter', 'contains')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "filesIn.filter.contains.assertTrue.(hasImport)+", + "sore_success": true + }, + { + "context": "('runTest', 'mockFilePart', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'findPageRendering', 'of')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('coEvery', 'ingestMultipart', 'any')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('trim', 'lowercase', 'warn')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "trim.lowercase.(warn)+", + "sore_success": true + }, + { + "context": "('DoclingConfig', 'assertThatThrownBy', 'validateCriticalSettings')", + "methods": 5, + "unique": 1, + "unique_ratio": 0.2, + "sore": "DoclingConfig.assertThatThrownBy.validateCriticalSettings.isInstanceOf.hasMessageContaining", + "sore_success": true + }, + { + "context": "('every', 'existsById', 'asKnowledgeBaseId')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('upsertStaging', 'of', 'of')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DocumentInput', 'asJobId', 'asDocumentId')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('defaultCapabilityId',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "defaultCapabilityId", + "sore_success": true + }, + { + "context": "('clearAllMocks',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "clearAllMocks", + "sore_success": true + }, + { + "context": "('mockkObject', 'slot', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('Builder',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('Any',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('get',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "get", + "sore_success": true + }, + { + "context": "('every', 'every', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ToolInvocationPolicyProperties', 'mapOf', 'mapOf')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('queryForObject',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "queryForObject", + "sore_success": true + }, + { + "context": "('registerProperties',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('assumeTrue', 'isDockerAvailable', 'start')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "assumeTrue.isDockerAvailable.start.pullAndWarmup", + "sore_success": true + }, + { + "context": "('filesIn', 'assertTrue', 'contains')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "filesIn.assertTrue.(contains)+", + "sore_success": true + }, + { + "context": "('JobStatus', 'every', 'getJobStatus')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "JobStatus.every.getJobStatus.get.uri.exchange.expectStatus.expectBody.(jsonPath.isEqualTo)+", + "sore_success": true + }, + { + "context": "('every', 'getJobStatus', 'get')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "every.getJobStatus.get.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?", + "sore_success": true + }, + { + "context": "('runBlocking', 'controller', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('builder', 'inputType', 'truncate')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('getMethod', 'invoke', 'return configureAndBuild(builder, config)')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "getMethod.invoke.return configureAndBuild(builder, config).configureAndBuild", + "sore_success": true + }, + { + "context": "('leteByFilter(f', 'lterEquals(M')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('`when`', 'healthCheckAsync', 'thenReturn')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('asJobId', 'asFilename', 'emptyGraphDocument')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('resolveUploadStorageUri',)", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('IngestionDocumentState', 'asDocumentId', 'asJobId')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('mapOf', 'mapOf', 'requireNotNull')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('createTempFile', 'apply', 'writeText')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'fetchTracking', 'asJobId')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobInstance', 'JobParametersBuilder', 'addString')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'existsByUsername', 'any')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "every.existsByUsername.any.assertThrows.registerUser.assertEquals", + "sore_success": true + }, + { + "context": "('AgentExecutionContext',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('runTest', 'TestRequest', 'every')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('run', 'Supplier', 'action')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('map',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "map", + "sore_success": true + }, + { + "context": "('sortedBy', 'map', 'toDescriptor')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "sortedBy.map.toDescriptor", + "sore_success": true + }, + { + "context": "('isNullOrBlank', 'error', 'error')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "isNullOrBlank.(error)+", + "sore_success": true + }, + { + "context": "('ChatResponse', 'listOf', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'RagInvocation', 'RagRequest')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'invoke', 'WikipediaLookupRequest')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('get', 'uri', 'exchange')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('chat',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('mockk', 'also', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('createKnowledgeBase', 'copyTestDocument', 'copyTestDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('MultipartBodyBuilder', 'part', 'readBytes')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('run', 'collectionPointCount')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "run.collectionPointCount", + "sore_success": true + }, + { + "context": "('newClient', 'callTool', 'CallToolRequest')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('assertNoMainProjectDependencies', 'listOf', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('scopeFromProject', 'filter', 'contains')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('productionFiles', 'filter', 'contains')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "productionFiles.filter.(contains.(assertTrue)?)+", + "sore_success": true + }, + { + "context": "('productionClasses', 'filter', 'hasAnnotationWithName')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('eFromProject()', 'ses()', 't { i')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('joinToString', 'warn', 'return ResponseEntity.status(HttpStatus.BAD_REQUEST)\\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors))')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "joinToString.warn.return ResponseEntity.status(HttpStatus.BAD_REQUEST)\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors)).status.body.ErrorResponse", + "sore_success": true + }, + { + "context": "('runTest', 'ChatResponse', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('runTest', 'post', 'uri')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "runTest.post.uri.bodyValue.(ChatRequest|SessionChatRequest).exchange.expectStatus", + "sore_success": true + }, + { + "context": "('runTest', 'coEvery', 'chatWithMemory')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('post', 'uri', 'exchange')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "post.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?", + "sore_success": true + }, + { + "context": "('every', 'knowledgeBaseExists', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('KnowledgeBaseResponse', 'now', 'now')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "KnowledgeBaseResponse.(now)+", + "sore_success": true + }, + { + "context": "('JobStatus', 'now', 'minusMinutes')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('runTest', 'mockk', 'every')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "runTest.mockk.every.filename.assertFailsWith.ingestMultipart", + "sore_success": true + }, + { + "context": "('every', 'getOrCreateAgentKnowledgeBase', 'slot')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'getJobStatus', 'JobStatus')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "every.getJobStatus.JobStatus.get.uri.exchange.expectStatus.expectBody.(jsonPath.isEqualTo)+", + "sore_success": true + }, + { + "context": "('assertThrows', 'EmbabelAiHttpClientProperties', 'ofSeconds')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "assertThrows.EmbabelAiHttpClientProperties.ofSeconds", + "sore_success": true + }, + { + "context": "('`when`', 'listModels', 'thenReturn')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('DoclingConfig', 'validateCriticalSettings')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "DoclingConfig.validateCriticalSettings", + "sore_success": true + }, + { + "context": "('info', 'deleteByJobId')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "info.deleteByJobId", + "sore_success": true + }, + { + "context": "('ilder()', 'ery(\"', 'pK(5')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('info', 'deleteByKnowledgeBaseId')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "info.deleteByKnowledgeBaseId", + "sore_success": true + }, + { + "context": "('buildImageKey', 'return storeObject(key, bytes, contentTypeFor(format))', 'storeObject')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('loadObject',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "loadObject", + "sore_success": true + }, + { + "context": "('resolveLocation', 'builder', 'bucket')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('replace', 'ifBlank', 'lowercase')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "replace.ifBlank.lowercase.return StorageUri.of(\"images/${jobId.value}/$sanitizedId.$extension\").of", + "sore_success": true + }, + { + "context": "('lowercase',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "lowercase", + "sore_success": true + }, + { + "context": "('parse', 'KnowledgeBaseNode', 'save')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('parse', 'save', 'KnowledgeBaseNode')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('every', 'similaritySearch', 'any')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('DocumentGraphJob', 'asDocumentId', 'asJobId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('asJobId', 'every', 'deleteByJobId')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "asJobId.every.(deleteByJobId.(assertDoesNotThrow)?)+", + "sore_success": true + }, + { + "context": "('runBlocking', 'adminClient', 'post')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('execute',)", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "execute", + "sore_success": true + }, + { + "context": "('every', 'process', 'any')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('update', 'trimIndent', 'now')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "update.trimIndent.now.insertRow", + "sore_success": true + }, + { + "context": "('policy', 'assertTrue', 'shouldRetry')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('state', 'every', 'recordFailure')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('input', 'beforeProcess', 'verify')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('processedDocument', 'every', 'enforce')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobParametersBuilder', 'addString', 'toJobParameters')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JobInstance', 'JobExecution', 'JobParametersBuilder')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('createJobExecution', 'every', 'findExecutionById')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('StagedUploadCleanupService', 'StorageProperties', 'toString')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('JwtService', 'JwtProperties', 'generateToken')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 12.0, + "elapsed_seconds": 0.04 +} \ No newline at end of file diff --git a/experiments/results/return_type_heuristic.json b/experiments/results/return_type_heuristic.json new file mode 100644 index 0000000..e91bac0 --- /dev/null +++ b/experiments/results/return_type_heuristic.json @@ -0,0 +1,57 @@ +{ + "strategy": "Option H: Return type heuristic", + "total_contexts": 60, + "meaningful_contexts": 5, + "total_methods": 1594, + "methods_in_good_groups": 0, + "sore_successes": 0, + "sore_failures": 0, + "skip_reasons": { + "too_large": 1, + "too_diverse": 4 + }, + "groups": [ + { + "context": "('RETURN_VALUE',)", + "methods": 1517, + "unique": 1312, + "unique_ratio": 0.865, + "sore": "SKIP(too_large)", + "sore_success": false + }, + { + "context": "('SIDE_EFFECT',)", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('RETURN_false',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('RETURN_result',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('RETURN_normalized',)", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 0.0, + "elapsed_seconds": 0.0 +} \ No newline at end of file diff --git a/experiments/results/summary.json b/experiments/results/summary.json new file mode 100644 index 0000000..46d4ce5 --- /dev/null +++ b/experiments/results/summary.json @@ -0,0 +1,187 @@ +[ + { + "strategy": "Baseline: Package grouping", + "total_contexts": 137, + "meaningful_contexts": 116, + "total_methods": 1594, + "methods_in_good_groups": 9, + "sore_successes": 2, + "sore_failures": 10, + "skip_reasons": { + "too_large": 5, + "too_diverse": 89, + "large_alphabet": 10 + }, + "coverage": 0.6, + "elapsed_seconds": 0.07 + }, + { + "strategy": "Option A: File path k=1", + "total_contexts": 47, + "meaningful_contexts": 43, + "total_methods": 1594, + "methods_in_good_groups": 17, + "sore_successes": 2, + "sore_failures": 1, + "skip_reasons": { + "too_large": 9, + "too_diverse": 28, + "large_alphabet": 3 + }, + "coverage": 1.1, + "elapsed_seconds": 0.01 + }, + { + "strategy": "Option A: File path k=2", + "total_contexts": 59, + "meaningful_contexts": 54, + "total_methods": 1594, + "methods_in_good_groups": 17, + "sore_successes": 2, + "sore_failures": 1, + "skip_reasons": { + "too_large": 8, + "too_diverse": 39, + "large_alphabet": 4 + }, + "coverage": 1.1, + "elapsed_seconds": 0.01 + }, + { + "strategy": "Option A: File path k=3", + "total_contexts": 59, + "meaningful_contexts": 54, + "total_methods": 1594, + "methods_in_good_groups": 17, + "sore_successes": 2, + "sore_failures": 1, + "skip_reasons": { + "too_large": 8, + "too_diverse": 39, + "large_alphabet": 4 + }, + "coverage": 1.1, + "elapsed_seconds": 0.01 + }, + { + "strategy": "Option B: First 1 symbols", + "total_contexts": 550, + "meaningful_contexts": 134, + "total_methods": 1594, + "methods_in_good_groups": 74, + "sore_successes": 20, + "sore_failures": 19, + "skip_reasons": { + "too_large": 2, + "large_alphabet": 13, + "too_diverse": 80 + }, + "coverage": 4.6, + "elapsed_seconds": 0.08 + }, + { + "strategy": "Option B: First 2 symbols", + "total_contexts": 961, + "meaningful_contexts": 141, + "total_methods": 1594, + "methods_in_good_groups": 157, + "sore_successes": 39, + "sore_failures": 23, + "skip_reasons": { + "large_alphabet": 7, + "too_diverse": 72 + }, + "coverage": 9.8, + "elapsed_seconds": 0.06 + }, + { + "strategy": "Option B: First 3 symbols", + "total_contexts": 1112, + "meaningful_contexts": 117, + "total_methods": 1594, + "methods_in_good_groups": 191, + "sore_successes": 47, + "sore_failures": 14, + "skip_reasons": { + "large_alphabet": 6, + "too_diverse": 50 + }, + "coverage": 12.0, + "elapsed_seconds": 0.04 + }, + { + "strategy": "Option C: Path k=1 + Symbol k=1", + "total_contexts": 813, + "meaningful_contexts": 151, + "total_methods": 1594, + "methods_in_good_groups": 101, + "sore_successes": 24, + "sore_failures": 19, + "skip_reasons": { + "too_diverse": 99, + "large_alphabet": 9 + }, + "coverage": 6.3, + "elapsed_seconds": 0.06 + }, + { + "strategy": "Option C: Path k=1 + Symbol k=2", + "total_contexts": 1073, + "meaningful_contexts": 119, + "total_methods": 1594, + "methods_in_good_groups": 138, + "sore_successes": 33, + "sore_failures": 21, + "skip_reasons": { + "too_diverse": 61, + "large_alphabet": 4 + }, + "coverage": 8.7, + "elapsed_seconds": 0.14 + }, + { + "strategy": "Option C: Path k=2 + Symbol k=1", + "total_contexts": 832, + "meaningful_contexts": 146, + "total_methods": 1594, + "methods_in_good_groups": 101, + "sore_successes": 24, + "sore_failures": 19, + "skip_reasons": { + "too_diverse": 94, + "large_alphabet": 9 + }, + "coverage": 6.3, + "elapsed_seconds": 0.08 + }, + { + "strategy": "Option C: Path k=2 + Symbol k=2", + "total_contexts": 1080, + "meaningful_contexts": 117, + "total_methods": 1594, + "methods_in_good_groups": 138, + "sore_successes": 33, + "sore_failures": 21, + "skip_reasons": { + "too_diverse": 59, + "large_alphabet": 4 + }, + "coverage": 8.7, + "elapsed_seconds": 0.06 + }, + { + "strategy": "Option H: Return type heuristic", + "total_contexts": 60, + "meaningful_contexts": 5, + "total_methods": 1594, + "methods_in_good_groups": 0, + "sore_successes": 0, + "sore_failures": 0, + "skip_reasons": { + "too_large": 1, + "too_diverse": 4 + }, + "coverage": 0.0, + "elapsed_seconds": 0.0 + } +] \ No newline at end of file diff --git a/experiments/results/two_d_p1_s1.json b/experiments/results/two_d_p1_s1.json new file mode 100644 index 0000000..490510f --- /dev/null +++ b/experiments/results/two_d_p1_s1.json @@ -0,0 +1,1225 @@ +{ + "strategy": "Option C: Path k=1 + Symbol k=1", + "total_contexts": 813, + "meaningful_contexts": 151, + "total_methods": 1594, + "methods_in_good_groups": 101, + "sore_successes": 24, + "sore_failures": 19, + "skip_reasons": { + "too_diverse": 99, + "large_alphabet": 9 + }, + "groups": [ + { + "context": "('controller', 'warn')", + "methods": 22, + "unique": 22, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'every')", + "methods": 18, + "unique": 16, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'runTest')", + "methods": 17, + "unique": 15, + "unique_ratio": 0.882, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'every')", + "methods": 17, + "unique": 16, + "unique_ratio": 0.941, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'productionClasses')", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'filesIn')", + "methods": 16, + "unique": 7, + "unique_ratio": 0.438, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('config', 'mockk')", + "methods": 15, + "unique": 5, + "unique_ratio": 0.333, + "sore": "(mockk.(((also|Neo4jConfig.transactionManager.assertTrue)|(every.((close|session)|(run.any|builder.build)))+)+)?)+", + "sore_success": true + }, + { + "context": "('architecture', 'listOf')", + "methods": 15, + "unique": 8, + "unique_ratio": 0.533, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embabel', 'ery {')", + "methods": 13, + "unique": 5, + "unique_ratio": 0.385, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'every')", + "methods": 13, + "unique": 11, + "unique_ratio": 0.846, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('health', '`when`')", + "methods": 13, + "unique": 11, + "unique_ratio": 0.846, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('system', 'runBlocking')", + "methods": 10, + "unique": 10, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'runTest')", + "methods": 10, + "unique": 8, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('reader', 'File')", + "methods": 10, + "unique": 9, + "unique_ratio": 0.9, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('storage', 'parseStorageUri')", + "methods": 9, + "unique": 6, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('adapter', 'every')", + "methods": 9, + "unique": 8, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'every')", + "methods": 9, + "unique": 9, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'classify')", + "methods": 9, + "unique": 8, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('simple', 'runTest')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'createKnowledgeBase')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'DoclingConfig')", + "methods": 8, + "unique": 2, + "unique_ratio": 0.25, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('graph', 'runBlocking')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'asJobId')", + "methods": 8, + "unique": 3, + "unique_ratio": 0.375, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('ids', 'assertEquals')", + "methods": 8, + "unique": 4, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('job', 'from')", + "methods": 8, + "unique": 5, + "unique_ratio": 0.625, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('listener', 'input')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'JobInstance')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'JwtService')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'await')", + "methods": 7, + "unique": 5, + "unique_ratio": 0.714, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('architecture', 'productionFiles')", + "methods": 7, + "unique": 6, + "unique_ratio": 0.857, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'JobStatus')", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('web', 'runBlocking')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'parse')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ids', 'of')", + "methods": 7, + "unique": 1, + "unique_ratio": 0.143, + "sore": "of", + "sore_success": true + }, + { + "context": "('job', 'query')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('chunk', 'HybridChunkingConfig')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('support', 'every')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'very')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('librechat', 'coEvery')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'getMethod')", + "methods": 6, + "unique": 2, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('storage', 'resolveLocation')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('repository', 'listOf')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('chat', 'runTest')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'forEach')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('config', 'trim')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'skipPolicy')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase', 'parse')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'every')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embabel', 'builder')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'scopeFromProject')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'withContext')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'mockk')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'VectorChunk')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('docling', 'trim')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "trim.lowercase.(warn)+", + "sore_success": true + }, + { + "context": "('job', 'upsertStaging')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'update')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "update.trimIndent.(now.insertRow)?", + "sore_success": true + }, + { + "context": "('batch', 'policy')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('model', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('writer', 'runTest')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('support', 'runTest')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embabel', 'isNullOrBlank')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'buildObservationContext')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'mockkObject')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('testsupport', 'Builder')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('testsupport', 'Any')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('testsupport', 'get')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "get", + "sore_success": true + }, + { + "context": "('wikipedia', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('wikipedia', 'runTest')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'ToolInvocationPolicyProperties')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('steps', 'requireNotNull')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'adminClient')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('system', 'MultipartBodyBuilder')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('system', 'session')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('config', 'registerProperties')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('config', 'assumeTrue')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "assumeTrue.isDockerAvailable.start.pullAndWarmup", + "sore_success": true + }, + { + "context": "('system', 'newClient')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('kotlin', 'getByType')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'eFromProject()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('kotlin', 'builder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'assertThrows')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "assertThrows.(EmbabelAiHttpClientProperties|OllamaClientProperties.Timeout).ofSeconds", + "sore_success": true + }, + { + "context": "('embedding', 'builder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('docling', 'builder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', '')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cleanup', 'info')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "info.(deleteByJobId|deleteByKnowledgeBaseId)", + "sore_success": true + }, + { + "context": "('repository', 'ilder()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'leteByFilter(f')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('graph', 'ImageData')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('repository', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'resolveUploadStorageUri')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('chunk', 'trim')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'JobParametersBuilder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'state')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'createJobExecution')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('job', 'createTempFile')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'createJobExecution')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('testcontainers', 'getenv')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('capability', 'AgentExecutionContext')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('simple', 'isEmpty')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('simple', 'EmbabelRagLoop')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'equals')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'isEmpty')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'runTest')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'map')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'buildObservationContext')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'messageWindowMemory')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'VectorChunk')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'answer')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('model', 'ChatResponse')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('support', 'VectorChunk')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('steps', 'chat')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('config', 'recreateTestCollection')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "recreateTestCollection.(truncatePostgresTables.clearNeo4jDatabase)?", + "sore_success": true + }, + { + "context": "('config', 'run')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "run.collectionPointCount", + "sore_success": true + }, + { + "context": "('architecture', 'readString')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'assertNoMainProjectDependencies')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('mcp', 'mono')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'joinToString')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "joinToString.warn.return ResponseEntity.status(HttpStatus.BAD_REQUEST)\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors)).status.body.ErrorResponse", + "sore_success": true + }, + { + "context": "('controller', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'post')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "post.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?", + "sore_success": true + }, + { + "context": "('auth', 'runBlocking')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'ChatMemoryConfig')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'Empty()')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('storage', 'buildImageKey')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('storage', 'replace')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "replace.ifBlank.lowercase.return StorageUri.of(\"images/${jobId.value}/$sanitizedId.$extension\").of", + "sore_success": true + }, + { + "context": "('storage', 'lowercase')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "lowercase", + "sore_success": true + }, + { + "context": "('storage', 'isBlank')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('graph', 'TextElement')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'nBlocking {')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('adapter', 'DocumentJobNode')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('adapter', 'parse')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'DocumentGraphJob')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'GraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'emptyGraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ids', 'trim')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'execute')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "execute", + "sore_success": true + }, + { + "context": "('batch', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('writer', 'forEach')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'queryForObject')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "queryForObject.(trimIndent)?", + "sore_success": true + }, + { + "context": "('job', 'debug')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'getString')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('model', 'DocumentInput')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('processor', 'input')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('writer', 'processedDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'StagedUploadCleanupService')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase', 'debug')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 6.3, + "elapsed_seconds": 0.06 +} \ No newline at end of file diff --git a/experiments/results/two_d_p1_s2.json b/experiments/results/two_d_p1_s2.json new file mode 100644 index 0000000..12554df --- /dev/null +++ b/experiments/results/two_d_p1_s2.json @@ -0,0 +1,969 @@ +{ + "strategy": "Option C: Path k=1 + Symbol k=2", + "total_contexts": 1073, + "meaningful_contexts": 119, + "total_methods": 1594, + "methods_in_good_groups": 138, + "sore_successes": 33, + "sore_failures": 21, + "skip_reasons": { + "too_diverse": 61, + "large_alphabet": 4 + }, + "groups": [ + { + "context": "('architecture', 'listOf', 'listOf')", + "methods": 15, + "unique": 8, + "unique_ratio": 0.533, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embabel', 'ery {', 'stKnowledgeBases()')", + "methods": 13, + "unique": 5, + "unique_ratio": 0.385, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('architecture', 'productionClasses', 'filter')", + "methods": 13, + "unique": 13, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'mockk')", + "methods": 11, + "unique": 1, + "unique_ratio": 0.091, + "sore": "mockk", + "sore_success": true + }, + { + "context": "('architecture', 'filesIn', 'assertTrue')", + "methods": 11, + "unique": 5, + "unique_ratio": 0.455, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('reader', 'File', 'writeBytes')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'JwtService', 'JwtProperties')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('ids', 'of')", + "methods": 7, + "unique": 1, + "unique_ratio": 0.143, + "sore": "of", + "sore_success": true + }, + { + "context": "('ids', 'assertEquals', 'of')", + "methods": 7, + "unique": 3, + "unique_ratio": 0.429, + "sore": "assertEquals.(of.(assertFailsWith)?)+", + "sore_success": true + }, + { + "context": "('job', 'from', 'now')", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "((from.now|(trimIndent|debug)).((update)+)?)+", + "sore_success": true + }, + { + "context": "('simple', 'runTest', 'mockk')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'very', 'istKnowledgeBases(')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('web', 'runBlocking', 'controller')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'getMethod', 'invoke')", + "methods": 6, + "unique": 2, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('storage', 'parseStorageUri', 'return false')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "parseStorageUri.return false.((return try {\n s3Client.headObject(\n HeadObjectRequest.builder()\n .bucket(location.bucket)\n .key(location.", + "sore_success": true + }, + { + "context": "('repository', 'listOf', 'listOf')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('chunk', 'HybridChunkingConfig', 'service')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'every', 'existsByUsername')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('system', 'createKnowledgeBase', 'copyTestDocument')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'await', 'pollInterval')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('architecture', 'filesIn', 'filter')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "filesIn.filter.contains.assertTrue.(hasImport)+", + "sore_success": true + }, + { + "context": "('controller', 'runTest', 'ChatResponse')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'runTest', 'mockFilePart')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'VectorChunk', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('librechat', 'every', 'findPageRendering')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('librechat', 'coEvery', 'ingestMultipart')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('docling', 'trim', 'lowercase')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "trim.lowercase.(warn)+", + "sore_success": true + }, + { + "context": "('config', 'DoclingConfig', 'assertThatThrownBy')", + "methods": 5, + "unique": 1, + "unique_ratio": 0.2, + "sore": "DoclingConfig.assertThatThrownBy.validateCriticalSettings.isInstanceOf.hasMessageContaining", + "sore_success": true + }, + { + "context": "('job', 'upsertStaging', 'of')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'query', 'trimIndent')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'update', 'trimIndent')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "update.trimIndent.(now.insertRow)?", + "sore_success": true + }, + { + "context": "('model', 'mapOf', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embabel', 'mockkObject', 'slot')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('testsupport', 'Builder')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('testsupport', 'Any')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('testsupport', 'get')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "get", + "sore_success": true + }, + { + "context": "('embabel', 'ToolInvocationPolicyProperties', 'mapOf')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('steps', 'requireNotNull', 'requireNotNull')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'session', 'use')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('config', 'registerProperties')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('config', 'assumeTrue', 'isDockerAvailable')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "assumeTrue.isDockerAvailable.start.pullAndWarmup", + "sore_success": true + }, + { + "context": "('architecture', 'eFromProject()', 'ses()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'runTest', 'coEvery')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'mockk', 'mockk')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'JobStatus', 'every')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "JobStatus.every.getJobStatus.get.uri.exchange.expectStatus.expectBody.(jsonPath.isEqualTo)+", + "sore_success": true + }, + { + "context": "('librechat', 'every', 'getOrCreateAgentKnowledgeBase')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('librechat', 'every', 'getJobStatus')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embedding', 'builder', 'inputType')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('health', '`when`', 'listModels')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('repository', 'ilder()', 'ery(\"')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'leteByFilter(f', 'lterEquals(M')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('health', '`when`', 'healthCheckAsync')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('graph', 'parse', 'KnowledgeBaseNode')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'asJobId', 'asFilename')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('chat', 'runTest', 'ChatResponse')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'resolveUploadStorageUri')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('job', 'JobParametersBuilder', 'addString')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'state', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'createTempFile', 'apply')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'every', 'fetchTracking')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'JobInstance', 'JobParametersBuilder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase', 'parse', 'KnowledgeBase')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('capability', 'AgentExecutionContext')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('support', 'runTest', 'TestRequest')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('embabel', 'listOf', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'VectorChunk', 'mapOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('support', 'every', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('support', 'VectorChunk', 'mapOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('wikipedia', 'every', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('wikipedia', 'runTest', 'invoke')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('steps', 'chat')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('config', 'mockk', 'also')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'MultipartBodyBuilder', 'part')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('config', 'run', 'collectionPointCount')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "run.collectionPointCount", + "sore_success": true + }, + { + "context": "('system', 'newClient', 'callTool')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'assertNoMainProjectDependencies', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'scopeFromProject', 'filter')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('architecture', 'productionFiles', 'filter')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "productionFiles.filter.(contains.(assertTrue)?)+", + "sore_success": true + }, + { + "context": "('architecture', 'productionClasses', 'withAnnotationOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('kotlin', 'builder', 'build')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'joinToString', 'warn')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "joinToString.warn.return ResponseEntity.status(HttpStatus.BAD_REQUEST)\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors)).status.body.ErrorResponse", + "sore_success": true + }, + { + "context": "('controller', 'listOf', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'runTest', 'post')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "runTest.post.uri.bodyValue.(ChatRequest|SessionChatRequest).exchange.expectStatus", + "sore_success": true + }, + { + "context": "('controller', 'every', 'getJobStatus')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "every.getJobStatus.get.uri.exchange.expectStatus", + "sore_success": true + }, + { + "context": "('controller', 'post', 'uri')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "post.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?", + "sore_success": true + }, + { + "context": "('controller', 'every', 'knowledgeBaseExists')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'JobStatus', 'now')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('librechat', 'runTest', 'mockk')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "runTest.mockk.every.filename.assertFailsWith.ingestMultipart", + "sore_success": true + }, + { + "context": "('config', 'assertThrows', 'EmbabelAiHttpClientProperties')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "assertThrows.EmbabelAiHttpClientProperties.ofSeconds", + "sore_success": true + }, + { + "context": "('config', 'ChatMemoryConfig', 'chatMemory')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('health', '`when`', 'health')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('config', 'DoclingConfig', 'validateCriticalSettings')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "DoclingConfig.validateCriticalSettings", + "sore_success": true + }, + { + "context": "('storage', 'buildImageKey', 'return storeObject(key, bytes, contentTypeFor(format))')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('storage', 'resolveLocation', 'builder')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('storage', 'parseStorageUri', 'return null')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('storage', 'replace', 'ifBlank')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "replace.ifBlank.lowercase.return StorageUri.of(\"images/${jobId.value}/$sanitizedId.$extension\").of", + "sore_success": true + }, + { + "context": "('storage', 'lowercase')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "lowercase", + "sore_success": true + }, + { + "context": "('storage', 'isBlank', 'return null')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('graph', 'runBlocking', 'createTestImage')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'parse', 'save')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'every', 'similaritySearch')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('adapter', 'DocumentJobNode', 'now')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'DocumentGraphJob', 'asDocumentId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('graph', 'asJobId', 'every')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "asJobId.every.(deleteByJobId.(assertDoesNotThrow)?)+", + "sore_success": true + }, + { + "context": "('graph', 'emptyGraphDocument', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('system', 'runBlocking', 'adminClient')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'execute')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "execute", + "sore_success": true + }, + { + "context": "('job', 'every', 'process')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'policy', 'assertTrue')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'createJobExecution', 'createStepExecution')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('model', 'DocumentInput', 'asJobId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('listener', 'input', 'beforeProcess')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('writer', 'runTest', 'GraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('writer', 'processedDocument', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'JobInstance', 'JobExecution')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'every', 'objectExists')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'createJobExecution', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('job', 'StagedUploadCleanupService', 'StorageProperties')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('knowledgebase', 'every', 'existsById')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 8.7, + "elapsed_seconds": 0.14 +} \ No newline at end of file diff --git a/experiments/results/two_d_p2_s1.json b/experiments/results/two_d_p2_s1.json new file mode 100644 index 0000000..33e8526 --- /dev/null +++ b/experiments/results/two_d_p2_s1.json @@ -0,0 +1,1185 @@ +{ + "strategy": "Option C: Path k=2 + Symbol k=1", + "total_contexts": 832, + "meaningful_contexts": 146, + "total_methods": 1594, + "methods_in_good_groups": 101, + "sore_successes": 24, + "sore_failures": 19, + "skip_reasons": { + "too_diverse": 94, + "large_alphabet": 9 + }, + "groups": [ + { + "context": "('springrag', 'controller', 'warn')", + "methods": 22, + "unique": 22, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'every')", + "methods": 18, + "unique": 16, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'runTest')", + "methods": 17, + "unique": 15, + "unique_ratio": 0.882, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'every')", + "methods": 17, + "unique": 16, + "unique_ratio": 0.941, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'productionClasses')", + "methods": 16, + "unique": 16, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'filesIn')", + "methods": 16, + "unique": 7, + "unique_ratio": 0.438, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'mockk')", + "methods": 15, + "unique": 5, + "unique_ratio": 0.333, + "sore": "(mockk.(((also|Neo4jConfig.transactionManager.assertTrue)|(every.((close|session)|(run.any|builder.build)))+)+)?)+", + "sore_success": true + }, + { + "context": "('springrag', 'architecture', 'listOf')", + "methods": 15, + "unique": 8, + "unique_ratio": 0.533, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'ery {')", + "methods": 13, + "unique": 5, + "unique_ratio": 0.385, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'every')", + "methods": 13, + "unique": 11, + "unique_ratio": 0.846, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'health', '`when`')", + "methods": 13, + "unique": 11, + "unique_ratio": 0.846, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'runBlocking')", + "methods": 10, + "unique": 10, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'runTest')", + "methods": 10, + "unique": 8, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('batch', 'reader', 'File')", + "methods": 10, + "unique": 9, + "unique_ratio": 0.9, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'storage', 'parseStorageUri')", + "methods": 9, + "unique": 6, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('port', 'adapter', 'every')", + "methods": 9, + "unique": 8, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'every')", + "methods": 9, + "unique": 9, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'classify')", + "methods": 9, + "unique": 8, + "unique_ratio": 0.889, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'simple', 'runTest')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'createKnowledgeBase')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'DoclingConfig')", + "methods": 8, + "unique": 2, + "unique_ratio": 0.25, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('repository', 'graph', 'runBlocking')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'asJobId')", + "methods": 8, + "unique": 3, + "unique_ratio": 0.375, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('common', 'ids', 'assertEquals')", + "methods": 8, + "unique": 4, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'job', 'from')", + "methods": 8, + "unique": 5, + "unique_ratio": 0.625, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'input')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'JobInstance')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'service', 'JwtService')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'await')", + "methods": 7, + "unique": 5, + "unique_ratio": 0.714, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'productionFiles')", + "methods": 7, + "unique": 6, + "unique_ratio": 0.857, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'JobStatus')", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'web', 'runBlocking')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'graph', 'parse')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('common', 'ids', 'of')", + "methods": 7, + "unique": 1, + "unique_ratio": 0.143, + "sore": "of", + "sore_success": true + }, + { + "context": "('service', 'job', 'query')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'chunk', 'HybridChunkingConfig')", + "methods": 7, + "unique": 7, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'very')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'coEvery')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'getMethod')", + "methods": 6, + "unique": 2, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'storage', 'resolveLocation')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'listOf')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('service', 'chat', 'runTest')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'forEach')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'trim')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'skipPolicy')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase', 'parse')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'service', 'every')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('summarizer', 'embabel', 'every')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'scopeFromProject')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'withContext')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'mockk')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'VectorChunk')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'docling', 'trim')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "trim.lowercase.(warn)+", + "sore_success": true + }, + { + "context": "('service', 'job', 'upsertStaging')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'update')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "update.trimIndent.(now.insertRow)?", + "sore_success": true + }, + { + "context": "('springrag', 'batch', 'policy')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'model', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('batch', 'writer', 'runTest')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'buildObservationContext')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'testsupport', 'Builder')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('springrag', 'testsupport', 'Any')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('springrag', 'testsupport', 'get')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "get", + "sore_success": true + }, + { + "context": "('rag', 'support', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'wikipedia', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'wikipedia', 'runTest')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('tooling', 'embabel', 'ToolInvocationPolicyProperties')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cucumber', 'steps', 'requireNotNull')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'adminClient')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'MultipartBodyBuilder')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'session')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'registerProperties')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'assumeTrue')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "assumeTrue.isDockerAvailable.start.pullAndWarmup", + "sore_success": true + }, + { + "context": "('springrag', 'system', 'newClient')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('main', 'kotlin', 'getByType')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'eFromProject()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('test', 'kotlin', 'builder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'assertThrows')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "assertThrows.(EmbabelAiHttpClientProperties|OllamaClientProperties.Timeout).ofSeconds", + "sore_success": true + }, + { + "context": "('service', 'embedding', 'builder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'docling', 'builder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'cleanup', 'info')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "info.(deleteByJobId|deleteByKnowledgeBaseId)", + "sore_success": true + }, + { + "context": "('springrag', 'repository', 'ilder()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'leteByFilter(f')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('springrag', 'repository', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'resolveUploadStorageUri')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('service', 'chunk', 'trim')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'JobParametersBuilder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'state')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'createJobExecution')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'job', 'createTempFile')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'createJobExecution')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'config', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'testcontainers', 'getenv')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'capability', 'AgentExecutionContext')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('rag', 'simple', 'isEmpty')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('capability', 'support', 'runTest')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('rag', 'simple', 'EmbabelRagLoop')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'equals')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'isEmpty')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'isNullOrBlank')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'builder')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('embabel', 'librechat', 'buildObservationContext')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'messageWindowMemory')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'VectorChunk')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'rag', 'answer')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'model', 'ChatResponse')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'support', 'VectorChunk')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cucumber', 'steps', 'chat')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'recreateTestCollection')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "recreateTestCollection.(truncatePostgresTables.clearNeo4jDatabase)?", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'run')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "run.collectionPointCount", + "sore_success": true + }, + { + "context": "('springrag', 'architecture', 'readString')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'assertNoMainProjectDependencies')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'mcp', 'mono')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'joinToString')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "joinToString.warn.return ResponseEntity.status(HttpStatus.BAD_REQUEST)\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors)).status.body.ErrorResponse", + "sore_success": true + }, + { + "context": "('springrag', 'controller', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'post')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "post.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?", + "sore_success": true + }, + { + "context": "('controller', 'auth', 'runBlocking')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'ChatMemoryConfig')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'Empty()')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'storage', 'buildImageKey')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('service', 'storage', 'replace')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "replace.ifBlank.lowercase.return StorageUri.of(\"images/${jobId.value}/$sanitizedId.$extension\").of", + "sore_success": true + }, + { + "context": "('service', 'storage', 'lowercase')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "lowercase", + "sore_success": true + }, + { + "context": "('service', 'storage', 'isBlank')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('repository', 'graph', 'TextElement')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'nBlocking {')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('model', 'graph', 'ImageData')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('port', 'adapter', 'DocumentJobNode')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('port', 'adapter', 'parse')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'DocumentGraphJob')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'GraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'emptyGraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('common', 'ids', 'trim')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'execute')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "execute", + "sore_success": true + }, + { + "context": "('springrag', 'batch', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'writer', 'forEach')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'queryForObject')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "queryForObject.(trimIndent)?", + "sore_success": true + }, + { + "context": "('service', 'job', 'debug')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'getString')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'model', 'DocumentInput')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'processor', 'input')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'writer', 'processedDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'StagedUploadCleanupService')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase', 'info')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase', 'debug')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 6.3, + "elapsed_seconds": 0.08 +} \ No newline at end of file diff --git a/experiments/results/two_d_p2_s2.json b/experiments/results/two_d_p2_s2.json new file mode 100644 index 0000000..e806442 --- /dev/null +++ b/experiments/results/two_d_p2_s2.json @@ -0,0 +1,953 @@ +{ + "strategy": "Option C: Path k=2 + Symbol k=2", + "total_contexts": 1080, + "meaningful_contexts": 117, + "total_methods": 1594, + "methods_in_good_groups": 138, + "sore_successes": 33, + "sore_failures": 21, + "skip_reasons": { + "too_diverse": 59, + "large_alphabet": 4 + }, + "groups": [ + { + "context": "('springrag', 'architecture', 'listOf', 'listOf')", + "methods": 15, + "unique": 8, + "unique_ratio": 0.533, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'ery {', 'stKnowledgeBases()')", + "methods": 13, + "unique": 5, + "unique_ratio": 0.385, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'productionClasses', 'filter')", + "methods": 13, + "unique": 13, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'mockk')", + "methods": 11, + "unique": 1, + "unique_ratio": 0.091, + "sore": "mockk", + "sore_success": true + }, + { + "context": "('springrag', 'architecture', 'filesIn', 'assertTrue')", + "methods": 11, + "unique": 5, + "unique_ratio": 0.455, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('batch', 'reader', 'File', 'writeBytes')", + "methods": 8, + "unique": 7, + "unique_ratio": 0.875, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'service', 'JwtService', 'JwtProperties')", + "methods": 8, + "unique": 8, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('common', 'ids', 'of')", + "methods": 7, + "unique": 1, + "unique_ratio": 0.143, + "sore": "of", + "sore_success": true + }, + { + "context": "('common', 'ids', 'assertEquals', 'of')", + "methods": 7, + "unique": 3, + "unique_ratio": 0.429, + "sore": "assertEquals.(of.(assertFailsWith)?)+", + "sore_success": true + }, + { + "context": "('service', 'job', 'from', 'now')", + "methods": 7, + "unique": 4, + "unique_ratio": 0.571, + "sore": "((from.now|(trimIndent|debug)).((update)+)?)+", + "sore_success": true + }, + { + "context": "('rag', 'simple', 'runTest', 'mockk')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'very', 'istKnowledgeBases(')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'web', 'runBlocking', 'controller')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'getMethod', 'invoke')", + "methods": 6, + "unique": 2, + "unique_ratio": 0.333, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'storage', 'parseStorageUri', 'return false')", + "methods": 6, + "unique": 4, + "unique_ratio": 0.667, + "sore": "parseStorageUri.return false.((return try {\n s3Client.headObject(\n HeadObjectRequest.builder()\n .bucket(location.bucket)\n .key(location.", + "sore_success": true + }, + { + "context": "('springrag', 'repository', 'listOf', 'listOf')", + "methods": 6, + "unique": 5, + "unique_ratio": 0.833, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('service', 'chunk', 'HybridChunkingConfig', 'service')", + "methods": 6, + "unique": 6, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('security', 'service', 'every', 'existsByUsername')", + "methods": 6, + "unique": 3, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'createKnowledgeBase', 'copyTestDocument')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'await', 'pollInterval')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'filesIn', 'filter')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "filesIn.filter.contains.assertTrue.(hasImport)+", + "sore_success": true + }, + { + "context": "('springrag', 'controller', 'runTest', 'ChatResponse')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'runTest', 'mockFilePart')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'VectorChunk', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'every', 'findPageRendering')", + "methods": 5, + "unique": 4, + "unique_ratio": 0.8, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'coEvery', 'ingestMultipart')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'docling', 'trim', 'lowercase')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "trim.lowercase.(warn)+", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'DoclingConfig', 'assertThatThrownBy')", + "methods": 5, + "unique": 1, + "unique_ratio": 0.2, + "sore": "DoclingConfig.assertThatThrownBy.validateCriticalSettings.isInstanceOf.hasMessageContaining", + "sore_success": true + }, + { + "context": "('service', 'job', 'upsertStaging', 'of')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'query', 'trimIndent')", + "methods": 5, + "unique": 5, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'update', 'trimIndent')", + "methods": 5, + "unique": 2, + "unique_ratio": 0.4, + "sore": "update.trimIndent.(now.insertRow)?", + "sore_success": true + }, + { + "context": "('batch', 'model', 'mapOf', 'mapOf')", + "methods": 5, + "unique": 3, + "unique_ratio": 0.6, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'testsupport', 'Builder')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Builder", + "sore_success": true + }, + { + "context": "('springrag', 'testsupport', 'Any')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "Any", + "sore_success": true + }, + { + "context": "('springrag', 'testsupport', 'get')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "get", + "sore_success": true + }, + { + "context": "('tooling', 'embabel', 'ToolInvocationPolicyProperties', 'mapOf')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cucumber', 'steps', 'requireNotNull', 'requireNotNull')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'session', 'use')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'registerProperties')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "registerProperties", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'assumeTrue', 'isDockerAvailable')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "assumeTrue.isDockerAvailable.start.pullAndWarmup", + "sore_success": true + }, + { + "context": "('springrag', 'architecture', 'eFromProject()', 'ses()')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'runTest', 'coEvery')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'mockk', 'mockk')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'JobStatus', 'every')", + "methods": 4, + "unique": 2, + "unique_ratio": 0.5, + "sore": "JobStatus.every.getJobStatus.get.uri.exchange.expectStatus.expectBody.(jsonPath.isEqualTo)+", + "sore_success": true + }, + { + "context": "('controller', 'librechat', 'every', 'getOrCreateAgentKnowledgeBase')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'every', 'getJobStatus')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'embedding', 'builder', 'inputType')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'health', '`when`', 'listModels')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'ilder()', 'ery(\"')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'leteByFilter(f', 'lterEquals(M')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "leteByFilter(f.lterEquals(M", + "sore_success": true + }, + { + "context": "('springrag', 'health', '`when`', 'healthCheckAsync')", + "methods": 4, + "unique": 3, + "unique_ratio": 0.75, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('repository', 'graph', 'parse', 'KnowledgeBaseNode')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'asJobId', 'asFilename')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'chat', 'runTest', 'ChatResponse')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'resolveUploadStorageUri')", + "methods": 4, + "unique": 1, + "unique_ratio": 0.25, + "sore": "resolveUploadStorageUri", + "sore_success": true + }, + { + "context": "('service', 'job', 'JobParametersBuilder', 'addString')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'state', 'every')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'createTempFile', 'apply')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'every', 'fetchTracking')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'JobInstance', 'JobParametersBuilder')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase', 'parse', 'KnowledgeBase')", + "methods": 4, + "unique": 4, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'capability', 'AgentExecutionContext')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "AgentExecutionContext", + "sore_success": true + }, + { + "context": "('capability', 'support', 'runTest', 'TestRequest')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('rag', 'embabel', 'VectorChunk', 'mapOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'support', 'every', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('rag', 'support', 'VectorChunk', 'mapOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'wikipedia', 'every', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('agent', 'wikipedia', 'runTest', 'invoke')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('cucumber', 'steps', 'chat')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "chat", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'mockk', 'also')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'MultipartBodyBuilder', 'part')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "SKIP(large_alphabet)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'run', 'collectionPointCount')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "run.collectionPointCount", + "sore_success": true + }, + { + "context": "('springrag', 'system', 'newClient', 'callTool')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'assertNoMainProjectDependencies', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'scopeFromProject', 'filter')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'architecture', 'productionFiles', 'filter')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "productionFiles.filter.(contains.(assertTrue)?)+", + "sore_success": true + }, + { + "context": "('springrag', 'architecture', 'productionClasses', 'withAnnotationOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('test', 'kotlin', 'builder', 'build')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'joinToString', 'warn')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "joinToString.warn.return ResponseEntity.status(HttpStatus.BAD_REQUEST)\n .body(ErrorResponse(\"VALIDATION_ERROR\", \"Validation failed\", errors)).status.body.ErrorResponse", + "sore_success": true + }, + { + "context": "('springrag', 'controller', 'listOf', 'listOf')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'runTest', 'post')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "runTest.post.uri.bodyValue.(ChatRequest|SessionChatRequest).exchange.expectStatus", + "sore_success": true + }, + { + "context": "('springrag', 'controller', 'every', 'getJobStatus')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "every.getJobStatus.get.uri.exchange.expectStatus", + "sore_success": true + }, + { + "context": "('springrag', 'controller', 'post', 'uri')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "post.uri.exchange.expectStatus.(expectBody.jsonPath.isEqualTo)?", + "sore_success": true + }, + { + "context": "('springrag', 'controller', 'every', 'knowledgeBaseExists')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'controller', 'JobStatus', 'now')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('controller', 'librechat', 'runTest', 'mockk')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "runTest.mockk.every.filename.assertFailsWith.ingestMultipart", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'assertThrows', 'EmbabelAiHttpClientProperties')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "assertThrows.EmbabelAiHttpClientProperties.ofSeconds", + "sore_success": true + }, + { + "context": "('springrag', 'config', 'ChatMemoryConfig', 'chatMemory')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'health', '`when`', 'health')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'config', 'DoclingConfig', 'validateCriticalSettings')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "DoclingConfig.validateCriticalSettings", + "sore_success": true + }, + { + "context": "('service', 'storage', 'buildImageKey', 'return storeObject(key, bytes, contentTypeFor(format))')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "buildImageKey.return storeObject(key, bytes, contentTypeFor(format)).storeObject.contentTypeFor", + "sore_success": true + }, + { + "context": "('service', 'storage', 'resolveLocation', 'builder')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'storage', 'parseStorageUri', 'return null')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('service', 'storage', 'replace', 'ifBlank')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "replace.ifBlank.lowercase.return StorageUri.of(\"images/${jobId.value}/$sanitizedId.$extension\").of", + "sore_success": true + }, + { + "context": "('service', 'storage', 'lowercase')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "lowercase", + "sore_success": true + }, + { + "context": "('service', 'storage', 'isBlank', 'return null')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('repository', 'graph', 'runBlocking', 'createTestImage')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('repository', 'graph', 'parse', 'save')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'repository', 'every', 'similaritySearch')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('port', 'adapter', 'DocumentJobNode', 'now')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'DocumentGraphJob', 'asDocumentId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'graph', 'asJobId', 'every')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "asJobId.every.(deleteByJobId.(assertDoesNotThrow)?)+", + "sore_success": true + }, + { + "context": "('service', 'graph', 'emptyGraphDocument', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'system', 'runBlocking', 'adminClient')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'execute')", + "methods": 3, + "unique": 1, + "unique_ratio": 0.333, + "sore": "execute", + "sore_success": true + }, + { + "context": "('service', 'job', 'every', 'process')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('springrag', 'batch', 'policy', 'assertTrue')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'createJobExecution', 'createStepExecution')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('batch', 'model', 'DocumentInput', 'asJobId')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'listener', 'input', 'beforeProcess')", + "methods": 3, + "unique": 2, + "unique_ratio": 0.667, + "sore": "\u2205", + "sore_success": false + }, + { + "context": "('batch', 'writer', 'runTest', 'GraphDocument')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('batch', 'writer', 'processedDocument', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'JobInstance', 'JobExecution')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'every', 'objectExists')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'createJobExecution', 'every')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'job', 'StagedUploadCleanupService', 'StorageProperties')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + }, + { + "context": "('service', 'knowledgebase', 'every', 'existsById')", + "methods": 3, + "unique": 3, + "unique_ratio": 1.0, + "sore": "SKIP(too_diverse)", + "sore_success": false + } + ], + "coverage": 8.7, + "elapsed_seconds": 0.06 +} \ No newline at end of file