← back
Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford
Takeaway
Rigorous private-repo measurement shows AI coding tools yield a real but modest ~15–20% net productivity gain, largely offset by rework on AI-generated code.
Summary
- Stanford study spans 100k+ engineers, 600+ companies, dozens of millions of commits across private repos in time-series and cross-sectional design.
- Critiques existing AI productivity studies for relying on commit/PR counts, green-field A/B tests, and self-reported surveys (which misjudge productivity by ~30 percentile points).
- Methodology uses an expert-panel-calibrated model that scores commits on functionality/quality/maintainability via Git, not lines of code.
- Average AI productivity gain is ~15–20%: gross +30–40% output, offset by rework fixing AI-introduced bugs; ghost-engineer share ~10%.
- Conclusion: AI is not one-size-fits-all; gains depend on context and are partially eaten by rework.
developer-productivityai-codingmeasurement
Original description
Forget vendor hype: Is AI actually boosting developer productivity, or just shifting bottlenecks? Stop guessing. Our study at Stanford cuts through the noise, analyzing real-world productivity data from nearly 100,000 developers across hundreds of companies. We reveal the hard numbers: while the average productivity boost is significant (~20%), the reality is complex – some teams even see productivity decrease with AI adoption. The crucial insights lie in why this variance occurs. Discover which company types, industries, and tech stacks achieve dramatic gains versus minimal impact (or worse). Leave with the objective, data-driven evidence needed to build a winning AI strategy tailored to your context, not just follow the trend. About Yegor Denisov-Blanch Researcher at Stanford University researching all things developer productivity Recorded at the AI Engineer World's Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter ---- The main thesis of the video is that while AI does increase developer productivity, it is not a one-size-fits-all solution. The speaker, Yegor Denisov-Blanch from Stanford, presents findings from a large-scale study on software engineering productivity to support this claim, arguing that the effectiveness of AI in software development is highly dependent on a variety of factors including task complexity, codebase maturity, language popularity, and codebase size. timestamps: - 00:00 Introduction and the context of AI in software development, including Mark Zuckerberg's bold claims. - 04:37 Limitations of existing studies on AI's impact on developer productivity. - 07:19 The methodology used by the Stanford research group to measure productivity. - 09:50 The overall impact of AI on developer productivity, including the concept of "rework." - 11:42 How productivity gains vary by task complexity and project maturity (Greenfield vs. Brownfield). - 14:21 The impact of programming language popularity on AI's effectiveness. - 15:42 How codebase size affects AI-driven productivity gains. - 17:22 The final conclusions of the study.