← back
RAG Agents in Prod: 10 Lessons We Learned — Douwe Kiela, creator of RAG
Original: RAG Agents in Prod: 10 Lessons We Learned — Douwe Kiela, creator of RAG
Takeaway
Win enterprise RAG by treating the system (not the model) as the product, specializing aggressively on noisy proprietary data, and shipping iteratively from day one.
Summary
- Douwe Kiela (Contextual AI CEO, original RAG paper co-author) argues only 1 in 4 enterprises get ROI from AI due to a 'context paradox' — LLMs handle generalist tasks but miss enterprise-specific context.
- Lesson: the LLM is only ~20% of an enterprise system; a mediocre model with a great RAG pipeline beats the inverse.
- Favor specialization over AGI; treat noisy enterprise data as the moat rather than scrubbing it.
- Pilots are easy, production at tens of thousands of docs and millions of users is the hard cliff — design for production day one.
- Speed over perfection: ship 'barely functional' to real users early and hill-climb via iteration.
ragenterpriseproduction
Original description
The latest generation of LLMs is demonstrating impressive test time reasoning capabilities. However, to be truly valuable in an enterprise setting requires those agentic capabilities to be applied to the right enterprise data. In all the excitement around AI agents, many of us have somehow forgotten the timeless adage “garbage in; garbage out” – language models can only do their job if they are contextualized properly. In this talk, Douwe Kiela will share lessons learned from deploying enterprise RAG systems at scale and how to design a system robust enough for the Fortune 500. Recorded live at the Leadership Track Session Day from the AI Engineer Summit 2025 in New York. Learn more at https://ai.engineer and purchase tickets to our next event, the AI Engineer World's Fair, in SF June 3 - 5 here: https://ti.to/software-3/ai-engineer-worlds-fair-2025 About Douwe Douwe Kiela is the CEO and Co-Founder of Contextual AI. He is also an Adjunct Professor in Symbolic Systems at Stanford University. Previously, he was the Head of Research at Hugging Face and a Research Lead at Meta’s Fundamental AI Research (FAIR) team, where he pioneered Retrieval-Augmented Generation (RAG) among other key AI breakthroughs. His work in multimodality, alignment, and evaluation has set new standards in the field of AI and has made systems safer, more reliable, and more accurate.