← back
ChatGPT is poorly designed. So I fixed it
Takeaway
You can patch ChatGPT's split-personality UX today with Realtime API + tool calls that route between voice, text and reasoning models on the fly.
Summary
- Speaker argues ChatGPT 'ships the org chart' (citing Scott Hanselman) — separate voice-to-text and voice-to-voice buttons, model picker chaos (o3, 4, o1, etc.) and no way to collaborate on a written email mid voice call.
- Builds a fix using OpenAI's 4o Realtime API for live audio plus tool calls — a 'send_chat_message' tool lets the assistant interject text (links, drafts) during a voice conversation, with an iMessage-style panel under the floating orb.
- Adds a second tool to hand off to a reasoning model when the user asks for depth or pros/cons, surfacing 'thinking for X seconds' status and returning a detailed response.
- Cites Warp Terminal as inspiration: simple 'undo my last commit' runs a coding agent, while 'refactor to Flutter' triggers a planning step with a reasoning model.
- Source code published on GitHub as 'fix-gpt'; demonstrates that smart model routing and multimodal text+voice can be done with off-the-shelf APIs and minimal prompts.
productvoicetool-use
Original description
Let's fix ChatGPT's greatest design sins. We'll design and build a working app that makes ChatGPT multi-modal and multi-model. And no, you don't need to know what those words mean to use it. Download the source code: https://github.com/bholmesdev/fixgpt References from this video: - Try https://warp.dev to vibe code your own solution - Watch Scott and Mark's podcast episode, "how to not ship the org chart:" https://www.youtube.com/watch?v=Z1yYcUFzH2A - Read "Why is AI marketing so, so bad?" by Evan Armstrong at The Leverage: https://www.gettheleverage.com/p/why-is-ai-marketing-so-so-bad