← back
Rethinking how we Scaffold AI Agents - Rahul Sengottuvelu, Ramp
Takeaway
Stop scaffolding around weak models; build agent systems where LLMs orchestrate and you scale by throwing more compute (parallel sampling + verifiers) at the problem.
Summary
- Sengottuvelu (Ramp, author of jsonformer) preaches the Bitter Lesson: systems that scale with compute beat hand-engineered rigid scaffolding because exponentials in model intelligence are rare gifts.
- Walks through Ramp's 'switching report' agent that parses arbitrary third-party card CSVs in three versions: hardcoded per-vendor, classical-flow-with-LLM-column-classification, and fully LLM-driven via code interpreter.
- Pure LLM+code-interpreter fails one-shot but running 50 parallel attempts with a verifier reliably generalizes; uses ~10,000x more compute but costs under $1 per CSV — engineer time is scarcer.
- Generalizes the pattern: future backends invert the classical/fuzzy boundary so LLM is the primary controller dropping into deterministic code when needed.
agentsbitter-lessonscaffolding
Original description
Diagrams, algorithms, and handcrafted heuristics have long played a role in constructing AI agents, but are these traditional scaffolding techniques holding us back? Many current approaches lean too heavily on manual design, neglecting the “bitter lesson” that history has taught us: leveraging massive computation and generic methods often trumps tailored engineering. What if we could rethink our scaffolding strategies for AI agents—aligning them with decades of insight from the broader AI research community? In this talk, Rahul Sengottuvelu, Head of Applied AI at Ramp and co-founder of Cohere.io, will explore how embracing the bitter lesson can transform our approach to building AI agents. Attendees will gain actionable strategies to design more robust, scalable systems that minimize over-engineering while maximizing the power of general-purpose learning. Recorded live at the Agent Engineering Session Day from the AI Engineer Summit 2025 in New York. Learn more at https://ai.engineer and purchase tickets to our next event, the AI Engineer World's Fair, in SF June 3 - 5 here: https://ti.to/software-3/ai-engineer-worlds-fair-2025 About Rahul Rahul Sengottuvelu is the Head of Applied AI at Ramp. Before that, he co-founded Cohere.io, a customer support automation platform that used advanced AI to resolve support tickets more accurately. Cohere.io was acquired by Ramp in May 2023. He is also the author of Jsonformer, a constrained decoding library for LLMs.