What CodeGraph says, vs. what it scored.
Local SQLite knowledge graph of symbols, call edges, and dependencies, built via tree-sitter and queried over MCP.
“Surgical context, fewer tool calls, faster answers, 100% local.”
“CodeGraph hands the agent the exact code it needs in one call. It's a pre-built knowledge graph of every symbol, call edge, and dependency in your codebase.”
| Tool | MRR | p95 | p100 |
|---|---|---|---|
| ★ LemonCrow +semantic (BGE) | 0.727 | 390ms | 1057ms |
| ★ LemonCrow lexical (default) | 0.676 | 134ms | 319ms |
| CodeGraph | 0.296 | 17ms | 532ms |
Own numbers real, fair with/without comparison. Fastest raw latency measured -- 17ms p95. 0.296 MRR vs. LemonCrow's 0.727.
| Kind | LemonCrow +semantic | LemonCrow lexical | CodeGraph |
|---|---|---|---|
| definition | 0.873 (n=1570) | 0.871 (n=1570) | 0.766 (n=1570) |
| content | 0.873 (n=1444) | 0.864 (n=1444) | 0.000 (n=1444) |
| semantic | 0.759 (n=1800) | 0.576 (n=1800) | 0.051 (n=1800) |
| swebench | 0.500 (n=1908) | 0.493 (n=1908) | 0.364 (n=1908) |
| sessions | 0.587 (n=491) | 0.571 (n=491) | 0.299 (n=491) |
n = query/gold pairs of that kind, out of 7,213 total -- every provider scored on all 5 kinds.
| Repo | LemonCrow +semantic | LemonCrow lexical | CodeGraph |
|---|---|---|---|
| astropy/astropy | 0.772 | 0.715 | 0.310 |
| django/django | 0.689 | 0.652 | 0.286 |
| lemoncrow-lab/lemoncrow-dev | 0.467 | 0.477 | 0.304 |
| lemoncrow/lemoncrow | 0.594 | 0.557 | 0.294 |
| matplotlib/matplotlib | 0.801 | 0.747 | 0.279 |
| mwaskom/seaborn | 0.814 | 0.768 | 0.331 |
| pallets/flask | 0.735 | 0.671 | 0.304 |
| psf/requests | 0.840 | 0.803 | 0.373 |
| pydata/xarray | 0.815 | 0.764 | 0.354 |
| pylint-dev/pylint | 0.856 | 0.784 | 0.359 |
| pytest-dev/pytest | 0.826 | 0.739 | 0.322 |
| scikit-learn/scikit-learn | 0.740 | 0.669 | 0.264 |
| sphinx-doc/sphinx | 0.637 | 0.580 | 0.225 |
| sympy/sympy | 0.694 | 0.637 | 0.283 |
| torvalds/linux | 0.726 | 0.668 | 0.158 |
MRR per repo: n-weighted blend across all 5 query kinds, same 7,213-query run.
| Repo | CodeGraph | vs baseline |
|---|---|---|
| Tokio | $3.44 | 28.1% pricier |
| Alamofire | $2.48 | 48.6% cheaper |
| Django | $2.32 | even |
| OkHttp | $1.35 | 15.6% cheaper |
| VS Code | $2.56 | 16.6% cheaper |
| Gin | $1.36 | 24.9% pricier |
| Excalidraw | $2.47 | 29.6% cheaper |
| Total | $15.99 | 16.3% cheaper |
Measured on LemonCrow's own 7-repo, 5-rep exploration-task suite (claude-opus-4-8), not CodeGraph's self-published with/without numbers. Cheaper than baseline on 5 of 7 repos (16-49%), pricier on Tokio (+28%) and Gin (+25%), and 22.7% slower overall despite the lower total cost -- a noisier result than CodeGraph's own published claims, which show it uniformly faster and never pricier on this same repo set.
Baseline total: $19.11. Wall-clock: 22.7% slower than baseline overall. Full per-repo detail, transcripts, and methodology → BENCHMARKS.md.
The true story
Same 14 repositories, same 7,213 query/gold pairs as every tool here, CodeGraph included. Full methodology, every raw number, and the other9 tools →