AI coding agents: 30% more code, no more shipped work
A Harvard study of 718 firms found AI coding agents raised code output 30% but not shipped work. Review time rose 49%. Budget for review before more seats.
AI coding agents made engineers write about 30% more code. The firms did not ship measurably more. That is the core finding of a Harvard working paper by Fiona Chen and James Stratton, built on Jellyfish engineering data from 718 companies. The extra code piled up at code review. If you pay for agent seats and expect features to arrive faster, this is the study to read before you buy more.
What actually happened
The paper, Artificial Intelligence in the Firm: Bottlenecks in Software Production, covers about 300 million work events from 700,000 workers, January 2021 to March 2026. It uses GitHub activity, Jira issues, calendar data and usage logs for GitHub Copilot, Cursor and Claude Code. The authors compare firms that adopted AI tools earlier and later (a staggered difference-in-differences design). It is observational, not a randomized trial.
What the authors found after firms adopted coding agents:
- Code output rose. Lines of code went up 30%, commits 20% and pull requests 23%.
- Shipped work did not rise measurably. Resolved Jira issues and epics showed small positive effects that were not statistically significant. For agents, the estimates rule out an output gain larger than 12% of the baseline.
- Review became the bottleneck. Average time to review a pull request rose 49%. Comments per pull request rose 35%. The share of pull requests with changes requested nearly doubled. The share of workers doing reviews rose 14%.
- AI review did not fix it. The bottleneck stayed after firms adopted AI code review tools. Humans still did most of the reviewing.
- No measurable job cuts. The authors found no significant employment effect from agents.
A note on the headlines: "zero extra issues resolved" overstates it. The estimate was positive but too noisy to separate from zero.
Why it matters for your business
Writing code is roughly 30–40% of an engineer's time, per the paper. Speed up only that part and the work waits at the next step. For a small team, the next step is usually one senior person who reviews everything. Give three developers agents and that person becomes the release schedule.
What we do about it:
- Measure merged and deployed work, not lines or pull requests. Agent dashboards that count lines measure the wrong thing.
- Make agent pull requests smaller. One change per pull request, with tests attached, so a review takes minutes.
- Budget review time when you add agent seats. Put a person's hours on it, not a hope.
- Set who pays for AI review. GitHub added organization billing controls for Copilot code review on Oct. 8: bill reviews to the org with a budget, and limit who can trigger them.
Key takeaways
- Harvard researchers studied 718 firms with Jellyfish data from 2021 to March 2026
- Coding agents raised lines of code 30%, commits 20% and pull requests 23%
- Resolved issues and epics did not rise by a statistically significant amount
- Review time rose 49% and changes-requested nearly doubled; AI review tools did not remove the bottleneck
- Measure shipped work, keep agent pull requests small, and budget human review time
Paying for agent seats but not shipping faster? We set up AI-assisted development with small reviewable changes, test gates and metrics that track deployed work, not line counts. See how we build or run the numbers on your team.
Sources: Chen & Stratton, Harvard (working paper), GitHub Changelog.
- #ai-coding-agents
- #code-review
- #developer-productivity
- #engineering-metrics
- #research
Tommy Rush — Founder, Rush Commerce
Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More
Get The Rush Report weekly — one email, zero fluff.
Keep reading
Yandex data center shut down: your cloud says run your DR plan
A drone strike fully shut Yandex's Vladimir data center, its third hit in a week. Yandex told cloud customers to activate disaster recovery. Is yours ready?
Read itJapan cyber emergency: your mobile app API keys are public
Japan declared a cybersecurity emergency after 20+ firms were breached. JPCERT says attackers pulled API keys out of mobile apps. Lock down your backend.
Read it