← back
On Engineering AI Systems that Endure The Bitter Lesson - Omar Khattab, DSPy & Databricks
Takeaway
Build LLM systems whose intent (signatures, metrics) is separate from prompts, so optimizers can re-compile them as scale-based models keep leapfrogging.
Summary
- Omar Khattab (creator of DSPy, now at Databricks) addresses Rich Sutton's Bitter Lesson: engineered domain knowledge gets beaten by scale and search.
- Frames the AI-engineer's dilemma — a new LLM every week, undocumented model quirks, new RL/prompt-opt algorithms, and APIs that silently swap underlying models.
- Resolution: build systems that are model-agnostic by encoding intent (signatures, modules) separately from prompts, then let optimizers compile them; this is what makes DSPy endure model churn.
- Argues AI engineering is about specifying problem structure and metrics, not crafting prompts — so your software survives the next bitter-lesson wave.
dspybitter-lessonai-engineering
Original description
Will discuss the principles for building AI software that underpin DSPy, highlighting the differences between conventional prompting (or finetuning/RL) versus the design and programming of truly modular AI systems. About Omar Khattab Omar Khattab is a Research Scientist at Databricks and an incoming Assistant Professor at MIT EECS (July 2025). His research creates models, algorithms, and abstractions for building modular, reliable, and scalable AI systems. He is the author of the ColBERT retrieval model, which has helped shape the modern landscape of neural information retrieval, and the creator of the DSPy framework for building and optimizing declarative natural-language programs. Recorded at the AI Engineer World's Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter Timestamps 00:00 AI Engineer World's Fair 00:22 On Engineering AI Systems that Endure the Bitter Lesson 00:32 The Challenges of AI Software Engineering 00:40 The Bitter Lesson 04:50 AI Engineering's Purpose 06:39 Takeaway 1: Engineering for Scalability 07:19 Premature Optimization 12:18 The Problem with Prompts 14:26 Trusty Old Separation of Concerns 17:11 Takeaway 2: Invest in Decoupling 17:21 The Pyramid of LLM Software and DSPy 17:45 The DSPy Concept: Declarative Signatures