← back
Substrate Launch: the API for modular AI
Takeaway
Substrate runs multi-model computation graphs as a coordinated cluster, replacing many slow API calls with microsecond inter-node hops and reliable structured outputs.
Summary
- Substrate is an SDK plus inference engine for describing computation graphs across many model nodes (text, image, speech, JSON, embeddings, code exec).
- Argues modular intelligence beats monolithic: more legible, debuggable, extensible, evalable because decision boundaries are explicit.
- Engine transfers data node-to-node in microseconds (vs hundreds of milliseconds per separate API call from DNS/auth/proxy overhead) — ~10,000× faster, enabling dozens of nodes in online apps.
- Heavy investment in JSON decoding mode since structured outputs are the connective tissue between multi-inference pipelines.
- Public launch: substrate.run, credits via QR/expo floor, contact Rob@substrate.run.
substratemodular-aiinference
Original description
AI models enable some new and powerful capabilities, but we still need to think of them as functions in larger logical systems, rather than a sort of end-state for software. AI-integrated programs should be built just like any other program: by relating small semantic tasks to each other in order to automate a larger task. Right now, the main problem preventing developers from running 10-20 AI modules on a single user request is an infrastructure one, and that’s what we’re solving. Large companies like Google have built internal infrastructure and SDKs to enable this, but we’re offering this capability to any developer. Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at https://www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at https://ai.engineer/2025 About Rob Rob Cheung (Co-Founder, CEO) was the founding engineer at Fin.com, a 2015 AI agent intermediated by a human-in-the-loop workforce. After Fin pivoted away from AI, he became the founding engineer at Substack, and for the last two years has been experimenting with the ideas that have become Substrate.