← All comparisons
LemonCrow vs codebase-memory-mcp
What codebase-memory-mcp says, vs. what it scored.
Tree-sitter-based persistent knowledge graph (SQLite-backed) across 158 languages -- the most-starred tool in this comparison.
What codebase-memory-mcp says about itself
“The fastest and most efficient code intelligence engine for AI coding agents.”
“Evaluated across 31 real-world repositories: 83% answer quality, 10x fewer tokens, 2.1x fewer tool calls vs. file-by-file exploration.”
What it actually scored — same 14 repos, same 7,213 queries as every other tool
| Tool | MRR | p95 | p100 |
|---|---|---|---|
| ★ LemonCrow +semantic (BGE) | 0.727 | 390ms | 1057ms |
| ★ LemonCrow lexical (default) | 0.676 | 134ms | 319ms |
| codebase-memory-mcp | 0.502 | 541ms | 1817ms |
83%/10x/2.1x numbers are real, peer-reviewed (arXiv:2603.27277), vs. raw file-by-file exploration. 0.502 MRR here vs. LemonCrow's 0.676-0.727.
By query kind -- same benchmark, broken out (no reps in this eval: one deterministic pass per query)
| Kind | LemonCrow +semantic | LemonCrow lexical | codebase-memory-mcp |
|---|---|---|---|
| definition | 0.873 (n=1570) | 0.871 (n=1570) | 0.718 (n=1570) |
| content | 0.873 (n=1444) | 0.864 (n=1444) | 0.709 (n=1444) |
| semantic | 0.759 (n=1800) | 0.576 (n=1800) | 0.252 (n=1800) |
| swebench | 0.500 (n=1908) | 0.493 (n=1908) | 0.417 (n=1908) |
| sessions | 0.587 (n=491) | 0.571 (n=491) | 0.443 (n=491) |
n = query/gold pairs of that kind, out of 7,213 total -- every provider scored on all 5 kinds.
By repo -- all 15 repos in the corpus, same query set
| Repo | LemonCrow +semantic | LemonCrow lexical | codebase-memory-mcp |
|---|---|---|---|
| astropy/astropy | 0.772 | 0.715 | 0.542 |
| django/django | 0.689 | 0.652 | 0.406 |
| lemoncrow-lab/lemoncrow-dev | 0.467 | 0.477 | 0.344 |
| lemoncrow/lemoncrow | 0.594 | 0.557 | 0.415 |
| matplotlib/matplotlib | 0.801 | 0.747 | 0.578 |
| mwaskom/seaborn | 0.814 | 0.768 | 0.566 |
| pallets/flask | 0.735 | 0.671 | 0.547 |
| psf/requests | 0.840 | 0.803 | 0.637 |
| pydata/xarray | 0.815 | 0.764 | 0.636 |
| pylint-dev/pylint | 0.856 | 0.784 | 0.582 |
| pytest-dev/pytest | 0.826 | 0.739 | 0.583 |
| scikit-learn/scikit-learn | 0.740 | 0.669 | 0.463 |
| sphinx-doc/sphinx | 0.637 | 0.580 | 0.437 |
| sympy/sympy | 0.694 | 0.637 | 0.487 |
| torvalds/linux | 0.726 | 0.668 | 0.456 |
MRR per repo: n-weighted blend across all 5 query kinds, same 7,213-query run.
The true story
Same 14 repositories, same 7,213 query/gold pairs as every tool here, codebase-memory-mcp included. Full methodology, every raw number, and the other9 tools →