← back
Buy Now, Maybe Pay Later: Dealing with Prompt-Tax While Staying at the Frontier - Andrew Thomspson
Takeaway
At the frontier you pay 'prompt tax' to migrate hundreds of domain prompts; favor prompting, embed domain experts, and rigorously experiment before each model switch.
Summary
- Andrew Thompson (CTO Orbital, real estate due-diligence agents, NY/London, ~80 people) defines 'prompt tax' as the cost of migrating prompts each time a frontier model ships.
- Orbital Copilot grew from <1B tokens/month 18 months ago to ~20B tokens/month, near-zero to multiple seven-figure ARR, while migrating GPT-3.5 → 4 → 4-32K → 4-Turbo → 4o → 4.1 → o1-preview → o4-mini.
- Three strategic decisions: optimize for prompting over fine-tuning (faster iteration during PMF), heavy reliance on embedded domain experts (private-practice real estate lawyers writing 1000+ domain-specific prompts), and 'vibes over evals' — controversial but worked.
- Domain-specific prompt count grew from near-zero to 1000+, meaning each model migration multiplies the regression risk surface area.
- Approach to new models: experimentation week (AI engineers + domain experts) to unlock envisioned features, gauge migration tax, and confront fear of unknown regressions.
promptingfrontier-modelsreal-estate
Original description
Frontier LLMs now drop at warp speed. Each upgrade hits you with a Prompt‑Tax: busted prompts, cranky domain experts, and evals that show up fashionably late. In this talk I’ll share 18 months of bruises (and wins) from shipping an agentic product for real‑estate lawyers: • The challenge of an evolving prompt library that breaks every time the model jumps • The bare‑bones tactics that actually work for faster migrations • Our “betting on the model” mantra: ship the newest frontier model even when it’s rough around the edges, then race to close the gaps before anyone else does Walk away with a playbook to stay frontier‑fresh without blowing up your roadmap or your team’s sanity.