The speed data everyone skips: what the studies actually say about AI and developer productivity

Everyone quotes the same stat: 75% of all new code at Google is now AI-generated.¹

It’s the wrong stat.

The share of code an AI wrote tells you nothing about whether work gets done faster. A model can generate every line of a pull request and still cost the team a day in review. So I went looking for the data that measures speed, including the parts that don’t flatter AI. This post is what I found, with sources at the end.

What’s been measured

The largest randomized trial to date. Researchers ran field experiments at Microsoft, Accenture and an unnamed Fortune 100 company covering 4,867 developers. Developers given GitHub Copilot completed 26.08% more tasks than the control group. The paper was published in Management Science in 2025.²

Experience level changed the result. Inside that same study, junior developers gained 21–40% and senior developers 7–16%. The less experienced you were, the more you gained.³

Google’s internal trial. Google ran its own randomized experiment with 96 engineers on an enterprise-grade task. The AI-assisted group finished in roughly 21% less time.⁴

Agents on a real migration. At Cloud Next in April 2026, Sundar Pichai described a complex code migration done by agents and engineers together. He said it finished six times faster than had been possible a year earlier with engineers alone.¹

Outside software. A study of 5,000 customer support agents found 14% more issues resolved per hour with an AI assistant. Novices improved 34%. Experts improved close to zero.⁵

Across the whole US workforce. The St. Louis Fed tracked the share of work hours saved by generative AI. It rose from 1.6% to 2.2% between Q3 2024 and Q2 2026. Workers who use AI average about 2.2 hours saved per week, and a third of daily users report saving four or more hours.⁶⁷

The number I left out. The most-quoted developer productivity figure is 55.8% faster, from the original GitHub Copilot experiment.⁸ It comes from a single scoped lab task: build an HTTP server in JavaScript. It is a real result, but it does not generalise to working in a real codebase, and it is usually quoted as though it does.

The uncomfortable data

METR’s randomized trial. METR recruited 16 experienced open-source developers and had them work on their own repositories, averaging more than a million lines of code each. Tasks were randomly assigned to allow or forbid AI tools.

With AI, the developers were 19% slower.⁹

Before the study they predicted they would be 24% faster. Afterwards, having done the work, they still believed they had been about 20% faster. They were wrong about their own productivity by 39 percentage points.

METR’s 2026 follow-up suggests newer tools have improved this, but the confidence interval was wide enough that they redesigned the experiment rather than publish a headline number.¹⁰ That is what honest measurement looks like.

The verification tax. DORA’s 2026 report on the ROI of AI-assisted development names the mechanism. AI speeds up writing code. Review, testing and deployment don’t speed up automatically, so the bottleneck moves instead of disappearing, and many teams see a productivity dip before they see returns.¹¹

I recognise this from my own work. Generating a service class takes seconds. Convincing myself it handles the edge case the ticket didn’t mention takes as long as it ever did. If I skip that step, the time comes back later as a bug.

What all of it says together

Same technology. 4,867 developers got 26% faster. 16 developers got 19% slower.

Productivity change with AI, by experience of the person using it

Junior developers · Copilot trial+21–40%

Novice support agents+34%

All 4,867 developers · Copilot trial+26%

Senior developers · Copilot trial+7–16%

Expert support agents≈ 0%

Expert open-source devs, own repos · METR−19%

−20%0+20%+40%

Those results stop conflicting once you line them up by who was doing the work and what they were working on:

• Gains are biggest for well-scoped work: enterprise tasks with a clear definition of done, where output can be checked quickly.
• Gains are biggest earlier in the skill curve. Juniors, novices and short-tenure workers gained the most in every study that split by experience.
• Gains shrink or reverse on large codebases the model hasn’t seen, where an expert already knows what to type and can’t check the AI’s output any faster than they could have written it.

The tool was the same in every trial. What differed was whether the person using it had the judgment to direct it, and to verify what came back faster than writing it themselves.

The market is pricing that judgment

PwC’s 2026 AI Jobs Barometer analysed more than a billion job ads across 27 countries.¹²

• AI-skilled roles pay a 62% wage premium, up from 57% last year and 25% two years ago.
• Postings for those roles are growing 69% a year, against 9% for the overall market.
• Entry-level roles most exposed to AI grew 35% since 2019. Other entry-level roles fell 10%.

Meanwhile Microsoft, Google, Amazon and Meta are spending roughly $725 billion on capital expenditure this year, up 77%.¹³ The tools are not going away, and the premium for knowing how to use them has grown every year it has been measured.

What I’m taking from this

Which number you end up as depends on the reps you put in.

Pick one workflow you already own. Rebuild it around AI. Then measure whether you actually got faster. Don’t trust the feeling. The METR developers trusted the feeling and were off by 39 points.

Then do it again next month.

Sources

Footnotes

1. Sundar Pichai, Google Cloud Next, April 2026. 75% of new code AI-generated, 6x faster migration. Google blog. ↩ ↩²

2. Cui, Demirer, Jaffe, Musolff, Peng & Salz. Randomized trials at Microsoft, Accenture and a Fortune 100 firm; 4,867 developers, +26.08% completed tasks. Management Science, 2025. Paper, free preprint. ↩

3. Junior vs senior breakdown from the same study. DX newsletter summary. ↩

4. Paradis et al., Google. Enterprise randomized controlled trial, 96 engineers, about 21% faster with AI. arXiv. ↩

5. Brynjolfsson, Li & Raymond. Generative AI at Work: 5,000 support agents, +14% issues resolved per hour, +34% for novices. NBER Working Paper 31161, 2023. NBER. ↩

6. St. Louis Fed / FRED. Work hours saved rose from 1.6% to 2.2%, Q3 2024 to Q2 2026. FRED blog. ↩

7. Bick, Blandin & Deming, St. Louis Fed. 5.4% of work hours saved, about 2.2 hours per week. Open Vault. ↩

8. Peng, Kalliamvakou, Cihon & Demirer. GitHub Copilot RCT, 55.8% faster on a scoped task. arXiv. ↩

9. METR. Experienced open-source developers 19% slower with AI, July 2025. Blog post. ↩

10. METR. 2026 follow-up and why the design changed. Blog post. ↩

11. DORA. ROI of AI-assisted Software Development: the verification tax and the productivity dip. Publications. ↩

12. PwC 2026 Global AI Jobs Barometer. Press release. ↩

13. Big Tech 2026 capex, $725B (+77%), from FT-compiled earnings data. Tom’s Hardware. ↩