← back
Your Support Team Should Ship Code – Lisa Orr, Zapier
Takeaway
Zapier's Scout agent turns the support team into shippers of integration fixes by orchestrating diagnosis and codegen MCP tools end-to-end inside their existing Jira/GitLab workflow.
Summary
- Zapier's 8000+ integrations suffer constant 'app erosion' from third-party API changes; Lisa Orr ran two experiments — empowering support to ship code, and Scout codegen.
- Built APIs for context gathering, diagnosis, unit test generation; biggest win was the Diagnosis API that auto-collects context (third-party docs, internal logs) for each ticket.
- MCP and Cursor adoption made standalone web UIs redundant; Scout MCP tools embed directly into engineer workflows in the IDE.
- Scout Agent orchestrates: categorize ticket -> assess fixability -> generate MR via GitLab CI/CD (plan, execute, validate phases) using Scout MCP and Cursor SDK -> support reviews and iterates.
- Lesson: tool adoption requires embedding in existing workflows; orchestration into a single agent beats a buffet of standalone tools.
agentssupportzapier
Original description
Zapier maintains 8000+ integrations that break as APIs change. We had thousands of backlog support tickets with dozens more arriving weekly. To keep up with the traffic, we started building AI tools to help ship integration fixes faster. We began by shadowing engineers fixing tickets and building tools we believed would expedite the fix process. Our first effort, an API playground hosting AI tools like diagnosis and test generation, failed to get engineering traffic because it pulled builders out of their workflows. We pivoted to MCP tools that engineers could use directly in their IDEs. MCP tools gained traction, but our most valuable tool, Diagnosis, took too long to run. Engineers wouldn't wait for it, revealing we needed an asynchronous approach. We built Scout Agent to string our tools together, autonomously reading support tickets, gathering context, generating fixes with tests, and submitting merge requests ready for review. This agent approach has gained traction with our support team handling high ticket volumes. An MR ready for review means they can validate and ship a fix quickly before needing to jump on the next incoming ticket. Throughout this process we've learned that the real challenge is everything surrounding code generation. Before writing code, Scout Agent needs both the right context and to show its work so engineers trust its recommendations. After generation, engineers need to quickly validate and correct the proposed fix, otherwise MRs sit unreviewed and abandoned. Embedding Scout Agent directly in GitLab solved this. Teams can iterate on proposed solutions without context switching. To track improvement, we measure three distinct failure modes: categorization accuracy (should Scout attempt this ticket?), fixability assessment (does this need a code fix?), and solution quality (does the generated code actually work?). Each reveals different improvement opportunities. Today, Scout drives 40% of support's integration fixes, with expansion to engineering teams and downstream automation (testing, shipping, migration) as our next frontiers. Speaker: Lisa Orr | Product Leader, Zapier https://x.com/orreither https://www.linkedin.com/in/lisaorr/