← back
The AI Evolution: Mario Rodriguez, GitHub
Takeaway
GitHub Copilot's $100M+ ARR rests on UX (ghost text), latency engineering and Codex/GPT-3.5 — not just the model — and post-PMF success requires global infra and rigorous online evals.
Summary
- Mario Rodriguez (VP Product, GitHub Copilot) reveals an internal August 2020 paper 'An automated AI programmer: fact or fiction?' as the catalyst that led to shipping Copilot in 2021 — the first AI programmer at scale.
- Copilot now serves 20,000+ organizations and 1M+ developers, is over $100M ARR, and accounts for ~46% of code accepted in some measurements via tab-completion (multi-line in many languages).
- V1 success came from four ingredients: ghost text UX, sub-100ms latency (now powered by GPT-3.5 Turbo after originally using Codex), the breakthrough Codex model, and serious prompt engineering.
- Lessons after PMF: pace gets faster not slower; syntax is not software (semantic work still required); global deployment (Japan, Europe, North America) is needed to keep latency under ~150ms; offline evals can pass while online evals tank, so build production scorecards.
- Reflections on developer happiness as Copilot's true success metric — flow state from removing boilerplate, not lines of code accepted.
copilotcode-generationgithub
Original description
Since the days of open source, we've experienced fundamental shifts in how software is built. From the pull request to deploying code, join Mario Rodriguez as he shares the history of GitHub Copilot and invites us to envision a new developer experience completely redefined and powered by AI. Recorded live in San Francisco at the AI Engineer Summit 2023. See the full schedule of talks at https://ai.engineer/summit/schedule & join us at the AI Engineer World's Fair in 2024! Get your tickets today at https://ai.engineer/worlds-fair About Mario Mario Rodriguez is a VP of Product Management at GitHub, currently focusing on all things Productivity. He oversees Repos, Pull Requests, Issues, Projects, Mobile and our AI strategy, including GitHub Copilot. Mario’s core identity is one of a learner and time away from product engineering is spent with his wife and two daughters.