← back
Leadership in AI Assisted Engineering – Justin Reock, DX (acq. Atlassian)
Takeaway
AI productivity gains are real but wildly variable — leaders must measure with telemetry+sampling+surveys and invest in psychological safety, not adoption mandates.
Summary
- DX research shows huge variability across companies in AI's productivity impact — averages show ~3% gains but per-company data ranges ±20% in change confidence and maintainability.
- METR study found engineers FELT 19% more productive with code AI while actually being 19% less productive — induced-flow illusion.
- Top-down 100% adoption mandates fail; what works is clear AI policies, dedicated learning time, and tying AI skill to employee success.
- Use a mix of telemetry (API metrics), experience sampling (PR-form questions), and high-participation surveys; W. Edwards Deming: 90–95% of productivity is the system, not the worker.
- Augmentation, not replacement — psychological safety (Google's Project Aristotle) is the biggest team performance predictor.
productivityleadershipmetrics
Original description
To realize meaningful returns on AI investments, leadership must take accountability and ownership of establishing best practices, enabling engineers, measuring impact, and ensuring proper guardrails are in place. When prompting practice and reflexive AI use is driven from the top down, engineers can align on the highest value use cases and experience peak productivity gains. When coupled with DX's AI Measurement Framework, leaders can gain a clear picture of AI's true impact, identify the real bottlenecks in the SDLC that can be augmented with AI, and drive improvement. In this session, Justin Reock, Deputy CTO at DX, and author of DX's Guide to AI Assisted Engineering, will explain what the most effective leaders of AI enabled engineering organizations are doing to drive satisfactory utilization, augmentation, and psychological safety across their teams. Based on interviews, use cases, and data, leaders will walk away with an understanding of how to best lead their teams through mature AI rollouts. Speaker: Justin Reock | Deputy CTO, DX https://www.linkedin.com/in/justinreock/