← back
Open Questions for AI Engineering: Simon Willison
Takeaway
AI engineering's open questions — interface, safety, evals, and rapidly falling cost curves — define the discipline more than any single tool does.
Summary
- Willison frames AI engineering as a brand-new discipline created in ~1 year; uses two tests for new tech: does it let me build the impossible, and does it make me faster?
- Walks the 2023 timeline: ChatGPT's chat UI broke things open despite being a 'terrible' interface for LLMs; Bing/Sydney threats triggered his first TV appearance via Elon Musk's algorithmic boost.
- Llama's Feb-2023 release plus Alpaca ($500 fine-tune via leaked PR with bittorrent link) was the 'stable diffusion moment' for local LLMs.
- Key open questions: how to ship safe LLM apps without irritating users, how to evolve UI past chat, and how to manage rapid commoditization of intelligence.
- Notes ~99.5% decline in GPT-3-level intelligence cost over 2 years and 90% decline for GPT-4-level via Llama 3.
ai-engineeringhistorycommodification
Original description
Recapping the past year in AI, and what open questions are worth pursuing in the next year! Covering local models, transparency, tool usage, prompt injection. Please will SOMEBODY solve these?? Recorded live in San Francisco at the AI Engineer Summit 2023. See the full schedule of talks at https://ai.engineer/summit/schedule & join us at the AI Engineer World's Fair in 2024! Get your tickets today at https://ai.engineer/worlds-fair About Simon Simon Willison is the creator of Datasette, an open source tool for exploring and publishing data. He currently works full-time building open source tools for data journalism, built around Datasette and SQLite. Prior to becoming an independent open source developer, Simon was an engineering director at Eventbrite. Simon joined Eventbrite through their acquisition of Lanyrd, a Y Combinator funded company he co-founded in 2010. He is a co-creator of the Django Web Framework, and has been blogging about web development and programming since 2002 at simonwillison.net