← back
How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou
Takeaway
For production code retrieval, replace generic embedding benchmarks with product-derived multi-needle evals like recall@50 — and own your retrieval stack end-to-end.
Summary
- Kevin Hou argues embeddings hit a ceiling for code retrieval: long context is too slow/expensive (Gemini takes 36s on 325k tokens; enterprise repos are >1B tokens), fine-tuning is prohibitively expensive per customer.
- Public benchmarks favor single-needle haystack scenarios; Codeium uses recall@50 across PR-derived ground truth (commit message -> modified files) to mimic real product distribution.
- Codeium's edge is vertical integration: their own retrieval models, custom infra inherited from the Exa Function pivot, allowing them to throw more compute at retrieval without runaway cost.
- Mentions 1.5M+ installs, top-rated extension across marketplaces, free unlimited autocomplete across 70 languages / 40 IDEs.
retrievalembeddingscode-search
Original description
Codeium is trailblazing the next frontier in retrieval and hint: it’s not just embeddings. Learn what the next generation of retrieval looks like and how 1M+ developers are already leveraging this superpower using the Codeium IDE plugin for AI autocomplete, chat, and search. We’ll dive deep into how existing benchmarks are failing us, what it takes to serve our custom models at scale, and what the future of AI-assisted software development looks like. Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at https://www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at https://ai.engineer/2025 About Kevin A full stack engineer by trade and a creator by heart. Enjoys the process of creation whether it be in the physical (woodshop, blacksmithing, circuity) or the digital (software engineering, photography, and film). Currently building AI-powered dev tools at Codeium (Exafunction). Previously a tech lead manager at Nuro self-driving. Received a computer science engineering degree from Princeton University with certificates in entrepreneurship and statistics and machine learning.