Every software company now says it uses AI. The claim has become so universal that it carries almost no information. If you are evaluating a development partner in 2026, “we use AI” tells you roughly as much as “we use computers.”<\/p>\n
What is worth knowing is narrower and more useful. Where does AI genuinely speed up a build? Where does it quietly create problems that surface three months later? And what separates the teams getting real value from the teams that just got faster at producing code nobody wants to maintain?<\/p>\n
That is what AI-enabled software development<\/a> means in practice for us. Not a claim that the tooling writes your system. A claim that we use it where it genuinely helps, inside a process that catches it when it is wrong.<\/p>\n The most cited study on this comes from METR, a research organisation that ran a controlled trial with experienced open source developers in early 2025. The result surprised almost everyone: developers using AI tools took about 19% longer to complete their tasks.<\/p>\n The more interesting part was what those same developers believed. After finishing slower, they estimated that AI had made them roughly 20% faster. The gap between what happened and what it felt like was close to 40 percentage points.<\/p>\n That study got quoted relentlessly by AI skeptics<\/a>, and it deserves a fuller reading. METR revisited the work in early 2026 and estimated a speedup of around 18% for the same cohort, a swing of 37 points in a year. They also found a design problem worth knowing about: between 30 and 50% of invited developers declined to take part at all if it meant working without AI access. That biased the original sample toward the developers least dependent on these tools. A revised cohort of 57 developers across more than 800 tasks landed at around a 4% slowdown, with a confidence interval wide enough to include a meaningful speedup. METR’s own summary is that AI likely does provide productivity benefits in early 2026, and that their original study design can no longer measure this reliably because nearly everyone uses these tools now.<\/p>\n Other research points the same general direction with similar messiness. McKinsey, working with around 4,500 developers, found 46% time savings on routine tasks and under 10% on complex work. Opsera, looking at a dataset of roughly 250,000 developers, found time to pull request improving by as much as 58%, while review time on AI-assisted pull requests ran 4.6 times longer.<\/p>\n Every one of those numbers is defensible. They disagree because they measure different things. Task completion time, pull request cycle time, developer sentiment, and downstream code quality are four separate questions, and a team tracking only the first will look like it is winning regardless of what is happening to the other three.<\/p>\n GitClear analysed 211 million changed lines of code from repositories at Google, Microsoft, Meta, and enterprise customers, covering 2020 through 2024. One metric stands out: code churn, defined as code that gets rewritten or deleted within two weeks of being committed, rose from 3.1% to 5.7% over that period.<\/p>\n Churn nearly doubling as AI-assisted coding scaled is not proof of causation. It is a signal worth taking seriously. Alongside it, GitClear found refactoring declining and code duplication rising.<\/p>\n CircleCI’s delivery research points at something similar from a different angle: higher coding velocity showing up alongside less code actually reaching production and more instability once it gets there.<\/p>\n None of this means AI tools make software worse. It means speed at the keyboard is not the same thing as speed to working software, and the two can move in opposite directions.<\/p>\n The pattern across the research is fairly consistent once you stop looking for one headline number. AI assistance produces its largest gains on work that is well defined and repetitive.<\/p>\n The common thread is that these tasks have a verifiable correct answer. When correctness is checkable, AI assistance is genuinely useful and the risk is low.<\/p>\n This also explains why the productivity research disagrees with itself so much. A study measuring developers on well-scoped tasks in a familiar codebase will find large gains. A study measuring experienced engineers solving genuinely hard problems in code they wrote themselves will find something close to zero, or worse. Both are describing real situations. Neither generalises to the other.<\/p>\n The gains shrink or reverse on work that requires judgement about things the model cannot see.<\/p>\n Architecture decisions depend on constraints that live in a business, not a codebase. Budget, team capability, regulatory obligations, what the client plans to build next year. A model producing a technically reasonable architecture with no knowledge of any of that is guessing confidently.<\/p>\n Debugging in unfamiliar systems is another weak spot. AI tools are good at proposing plausible causes. They are not good at knowing when a proposed cause is wrong, which means a developer can spend longer chasing a confident wrong answer than they would have spent reading the code.<\/p>\n Domain-specific compliance work is the one we care about most, because it is where our own projects live. HIPAA safeguards, judicial evidence handling, defence sector data requirements. A model can produce code that looks correct and violates a requirement it was never told about. In a regulated build, nobody finds that out through a failing test. They find out in an audit.<\/p>\n Reading across all of this, the split is not really about which tools a team uses. It is about what happens after the code is generated.<\/p>\n Teams with genuine test coverage, real code review, and working continuous delivery appear to get meaningful benefits from AI assistance. Teams without those things get faster code production and a larger pile of code they cannot verify. The second group is much larger than the first.<\/p>\n The Opsera finding about review time is worth sitting with here. If pull requests arrive 58% faster but each one takes 4.6 times longer to review, the bottleneck has not disappeared. It moved. A team that celebrates the first number without staffing for the second ends up with a review queue that grows quietly until someone starts approving things they have not properly read.<\/p>\n That framing is more useful than any productivity percentage, because it is actionable. If you cannot tell whether a change broke something, generating changes faster is not an improvement.<\/p>\nThe Productivity Numbers Are Messier Than Anyone Advertises<\/strong><\/h2>\n
What Study Says<\/strong><\/h3>\n
The Quality Signal Nobody Wants To Talk About<\/strong><\/h2>\n
Where AI Reliably Helps<\/strong><\/h2>\n
\n
Where It Does Not<\/strong><\/h2>\n
What Actually Separates The Teams That Benefit<\/strong><\/h2>\n