Patrick Dougherty: How to Build AI Agents that Actually Work

Summary

• Defines an agent by three criteria: takes directions, calls at least one tool, autonomously reasons about which tool to use (excludes prompt chains).
• At Rosco (data-warehouse SQL agent), favored small retrieval tools like search_tables, get_table_detail, profile_column over stuffing tables into the prompt; GPT-4o would hallucinate SQL where o1 would say the schema doesn't support the question.
• Agent Computer Interface (ACI) iteration is the highest-leverage work: tool-result format markdown→JSON fixed column-blindness in GPT-4o, while Claude prefers XML.
• Reasoning matters more than knowledge; fine-tuning hurt because it overfit specific sequences and reduced reasoning flexibility.
• Claude 3.5 Sonnet still the favorite orchestrator model; observed hallucination patterns reveal the model's expected tool-call format — change the schema to match.