There is a version of the AI pitch that every engineering leader has now heard. A controlled study hands developers an AI coding assistant, they finish the task 55.8% faster, a vendor slide rounds it to “twice as fast”, and the budget conversation ends with a quiet expectation: the same roadmap in half the time, starting next quarter. This is a short essay about why that expectation fails, and what the people who run teams should steer by instead.
The 50% that never arrives
Both the promise and the letdown are well measured. The 55.8% figure is real: Peng et al. timed developers building an HTTP server from scratch, a self-contained task where typing code is the work. But when METR ran a randomized trial in 2025 on experienced open-source developers working in their own mature repositories, the developers using AI took 19% longer, while believing they had been 20% faster. The two studies do not contradict each other. They bracket the truth: AI accelerates the writing of code, and in real work the writing of code was never the constraint.
Fred Brooks made the underlying point in 1986, in “No Silver Bullet”: the hard part of software is essential, deciding what to build and how it must behave, and no aid to the accidental labour of writing it down could deliver even a tenfold gain. Speed up the easy part and the queue simply re-forms downstream, at code review, at verification, at deciding whether the generated change can be trusted. The 2024 DORA report caught this at industry scale: a 25% increase in AI adoption was associated with a 1.5% drop in delivery throughput and a 7.2% drop in delivery stability. More code, arriving faster, into a delivery process nobody redesigned.
So the first thing to steer by is a question, not a metric: now that writing code is cheap, where has our bottleneck gone? For most teams the honest answers are review capacity, test coverage, and the quality of decisions about what to build at all.
Spend the freed hours on learning
The hours the tools free are real. They just arrive as slack, not as shipped features, and slack left unallocated evaporates into more work in progress. The better use is the one delivery pressure never used to permit. Most teams build the first design that comes to mind, not because it is best but because exploring a second one was unaffordable. Cheap code changes that arithmetic: prototype three approaches and throw two away, spike the risky integration before committing a quarter to it, let people rebuild something they use daily just to understand its trade-offs.
That reallocation is a leadership decision, and it has to be explicit, because the alternative is not neutral. A team told “you have AI now, ship faster” produces more unreviewed code, the DORA drop, and a quieter liability underneath it: a kind of cognitive debt. Peter Naur argued back in 1985 that a program is not really its code but the theory of it held in its builders’ heads, and that only people who hold the theory can extend the system without wrecking it. A team that spends its AI dividend purely on generating output is assembling an estate nobody has a theory of, where everything works and nobody is quite sure why, and in a few years no one is comfortable touching it, or willing to. That is debt in the honest sense, deferred understanding that comes due the first time the system has to change. A team told “the exploration is the work” avoids it almost for free, because exploring designs and trade-offs is precisely how the theory gets built: engineers who understand the option space before choosing, who can explain what shipped, and who are quietly building the judgement the tools multiply. The habits that make individual sessions productive are their own subject, covered in an earlier post on working with coding agents; at team level the job is simpler and harder, which is to protect the time in which those habits get learned.
The returns are long-dated, and that is normal
Economists have a name for what a technology like this does to the numbers: the productivity J-curve. When a general-purpose technology arrives, firms spend years on complementary investment, in skills, process redesign, and reorganisation, that shows up in no output statistic, so measured productivity dips before it climbs. The canonical case is electricity: Paul David’s famous study of the dynamo found roughly forty years between the first power stations and visible factory productivity gains, because swapping a steam engine for one large electric motor changed almost nothing. The gains came when factories were rebuilt around many small motors.
AI coding tools are the motor, and the factory is your review practice, your test suite, your deployment pipeline, and your ways of working. The further an organisation sits from the frontier, running legacy estates, regulated processes, or a business that is not tech-first, the more rewiring the payoff demands and the longer the J takes to turn. The 2025 DORA report puts the same finding bluntly: AI is an amplifier, magnifying the strengths of healthy organisations and the dysfunctions of struggling ones.
None of this is a case for sitting it out. It is a case for classifying the spend correctly. This is the next general-purpose technology, and treating it as a tool purchase with a quarterly payback is exactly how to be disappointed by it, then wrongly conclude it does not work. Fund it like infrastructure, judge it over years, and expect the dip, because the dip is what the investment looks like from the outside.
Headcount becomes a lens
The long-term implication is worth saying plainly: for a fixed amount of output, you will probably need fewer people, and pretending otherwise helps no one. There are two defensible responses. The first is to hold the team and redirect the saved spend inward, upskilling the engineers you already trust rather than outsourcing or backfilling, so the capability compounds inside the company. The second is more contentious: part ways with engineers who refuse to engage with the tools at all, and pair a smaller number of strong seniors with AI-fluent junior hires, because a full bench of seniors may no longer be the only viable shape. Which response fits depends on how critical your systems are, your hiring market, and your appetite for risk, and reasonable leaders will land in different places.
What is not defensible is the default: buy the licences, change nothing about the structure, and treat the organisation chart as exempt from the technology’s implications. AI does not just change how code gets written. It hands you a new lens on how the team itself is designed, and leaders who look through it deliberately will make better calls than those who wait for the org design to be forced on them.
What to steer by
Expect the bottleneck to move, and go find where it landed. Spend the freed hours on exploration and learning, on purpose and in the plan, not as leftovers. Judge the returns over years, and read the early dip as investment rather than failure. And treat the shape of the team as a variable you choose for your situation, not something you inherit. The tools are not a shortcut through the work of leading a team. They are a reason to do that work deliberately.
Sources & further reading
- Peng et al., “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot” (2023) — the controlled experiment behind the 55.8% speedup, on a self-contained greenfield task.
- METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” — the RCT in which experienced developers on mature repos took 19% longer with AI while believing they were 20% faster.
- DORA, 2024 Accelerate State of DevOps Report — a 25% increase in AI adoption associated with a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability.
- DORA, 2025 State of AI-assisted Software Development — the “AI is an amplifier” finding: gains flow to organisations whose platforms and practices were already healthy.
- Fred Brooks, “No Silver Bullet” (1986) — the essential-versus-accidental complexity argument for why speeding up code-writing alone cannot transform delivery.
- Peter Naur, “Programming as Theory Building” (1985) — the classic argument that a program lives in its builders’ heads, and that code whose theory nobody holds is effectively dead: the case for treating team understanding as the asset.
- Brynjolfsson, Rock & Syverson, “The Productivity J-Curve” (NBER, 2018) — why general-purpose technologies depress measured productivity before raising it.
- Paul David, “The Dynamo and the Computer” (1990) — the forty-year lag between electrification and factory productivity gains, and why the gains required redesigning the factory.