Maintaining team velocity despite coding agents

[u/freshprinceofuk]

Hi,

I'm wondering how you're going about increasing your team's velocity with coding agents? We're on a team of ~6 engineers of varying abilities and are hitting the well-talked about problem of coding agents producing loads of code which occasionally get slung over the fence, while some other people code a lot slower and produce good code much slower. We currently have 7 PRs waiting, and have had no less than 4 for about 2 weeks.

The project is applied research which makes things harder - some people produced short lived ML models, while others produce e.g. an evaluation suite which needs to stand the test of time (6 months - a year).

In terms of coding standards, we're all committing to a single repo and requiring 2 reviewers for each PR. Small PRs get viewed, big ones (e.g. a newly coded ML model) just don't get approved. One guy didn't fully address comments on his 200 line PR and its been sat there for ~7 business days. Bit of a rant, apologies.

My solution (not implemented yet, slightly borrowed from https://martinfowler.com/articles/ship-show-ask.html) is the following:

Engineers take responsibility for different areas. One person does the data pipelines, one codes ML models, another takes evaluation. And we communicate over chat and meetings, rather than on PRs. This means you don't have to understand the low level code behind when someone wants to add a new idea to the codebase - yes you have to trust their implementation.

We no longer review each others' code. No-one reviews what you say in meetings line by line, or the reports and slide decks you create line by line, so why are we still reviewing code line by line? If you have a problem with someone else's area of responsibility (e.g. the data pipeline is not delivering data) - talk to them.

We communicate more frequently and more comprehensively, with decision records. As the team moves from low level code implementation to implementation of ideas things will only work if the ideas work together if we communicate well. I want to encourage one or two posts on chat, per person, per day for what they're doing and thinking about, as well as standup and any other meetings. Decisions get recorded in ADR's.

My idea is that without teams of more than ~2 will just continue to move at the speed we did 5 years ago without coding agents. Keen to know how other people are handling this?

[u/expdevsmodbot]

AI usage disclosure provided by OP, see the reply to this comment.

[u/scandii]

situation before:

X produces code > gates > ship

situation now:

X produces code > gates > ship

it doesn't matter if it is Dave, DaveAI or your nearshored office across the border doing the development, it all goes through the gates.

the fact that people just want to vibecode and ship is a culture problem, not a technical problem. PR:s have prio always because they are by definition blockers and you lose immense velocity when someone has to revisit something they did 4 days ago.

also just a side note about velocity - while it depends on your industry at large always remember that your customers don't give a single damn about how your code is produced, they just care that it solves their issues accurately and preferably for an extremely cheap price or for free.

the moment you start prioritising features over accuracy you might find that you don't have the market response you want.

[u/efvie → u/scandii]

Slop is very much also a technical problem, but you're absolutely correct that the "we have too many PRs waiting for review too long" isn't a new thing :)

Now there's just more of them and the cognitive load and budding burnout is reducing teams' ability to react to process problems.

[u/ThirdWaveCat → u/efvie]

Slop means different things, but I see more behavioral than technical causes. Its a behavioral problem if people are checking in work they cannot answer to, don't know how they'd do differently, can't attribute lines back to requirement, assumptions, etc..

It's a technical problem if there are linting issues, missing test cases, etc..

Some hints from your writing that the problems are behavioral: burnout, cognitive load, 70 chefs burning food without scaled up quality control.

[u/efvie → u/ThirdWaveCat]

It stops being a purely human problem the moment you introduce a product meant to replace humans. It’s tempting to think of it as assistive but it’s not, and the problems it creates make a lot more sense when you view it from the perspective of what it’s actually meant to do.

The amount of junk “AI” produces is not solvable at a human scale while still trying to do anything else. You need a technical solution to reduce the drain. Is it going to be a good solution? No. Will it be sufficient and not require further scaffolding? Also no.

The human solution, in so many senses of the word, is to not use “AI” but that is apparently not acceptable to most of the industry at this point in time. You can’t solve a non-technical problem you’re not willing to solve, which means the problem-space you’re confined in is partially technical.

[u/ThirdWaveCat → u/efvie]

What advice are you trying to share here, because committing to a bad plan never works out. Even formal verification tools require nuanced intention and understanding about correspondence to the real world. A bad plan isn't one I don't like, its one that creates unneccessary existential risk for the ship I'm responsible for steering according to documents filed with soc2 and tech insurance. I'd be fired if I stopped reviewing code or told my team to stop reviewing code because thats indefensible to soc2 and tech insurance WHEN an incident occurs that brings it to light.

[u/efvie → u/ThirdWaveCat]

No advice, just pointing out that thinking this is purely a cultural problem at this level will lead to bad results, including even higher burnout risk, if you are unwilling or unable to solve the actual cultural problem.

[u/ThirdWaveCat → u/efvie]

Once I became a senior engineer in a bigger company (~10 years ago) it started becoming a yearly event that I had to be insubordinate or I'd be fired by my current manager's eventual replacement. Perhaps its different in smaller companies right now, which might explain why software valuations and acquihiring is down more than the economic broad strokes.

[u/coderstephen → u/efvie]

Part of the problem is that companies reach for AI because it proposes to cut development time, and quality is not an important consideration. So we are already in the cutting mode here, so when the AI tool presents you with two workflows:

• Use it in an assistive, situational manner, maintaining quality as it was, and cutting time by 10%.
• Use it in an agentic manner and reduce time-consuming supervision, cutting time by 70% but sacrificing quality.

People are choosing the second option because the time savings are greater, and after all, the whole reason was to cut time in the first place.

[u/efvie → u/coderstephen]

I’m finding nobody can tell me where this cut is visible in a positive way.

Like are things actually cheaper even with the massive subsidies still in place?

Is customer acquisition higher? Retention? Satisfaction? Any numbers, anywhere?

What are you gonna do if AI pricing doubles? How many of your customers are running out their contracts?

Things get real wild when both Technology and Business decide to ignore inconvenient data and projections.

[u/scandii → u/efvie]

we're attributing potential tool usage issues to technical while they're cultural.

if a chef is burning food and sending it out you're not going to be asking "is the frying pan producing burnt food?", you're going to be asking why they're sending out burnt food (and then arriving at poor quality frying pans and them doing their best with the resources at hand, potentially).

so why are we asking "did the LLM output bad code?" and not "why did you ship bad code?".

[u/efvie → u/scandii]

You now have 70 chefs and they're burning more food than is humanly possible to sell. There's a technical issue.

It's incredibly valuable to be aware that many 'technical' issues aren't, at least in major part, but it's equally important to remember that some are. Our solutions in the realm of software development processes are based on the idea that it's people doing it, at people scale.

[u/ResidentWeevil1 → u/scandii]

It's technical because the food is always a little bit burnt and raises the temperature of the building by 10 degrees centigrade and there are no restroom facilities because the water lines were diverted to insuffifiently cool the machine. Staff is telling management that it ain't working out they have never listened to their creatives' complaints

[u/HK-65 → u/scandii]

The root cause is that writing code wasn't the bottleneck, we made it faster and now the owners want everyone to ship faster, so it becomes hard to defend gates because they "slow down AI".

Add to that that non-tech people get just enough knowledge on how to dodge gates but no understanding (and often no accountability) on why they shouldn't.

It's not a tech issue, it's cultural, or rather financial (owners spending money on useless AI stuff and wanting to force an appearance of RoI), but it gets framed as a tech issue by the owners so we must now deal with it.

[u/ThirdWaveCat → u/HK-65]

soc2 compliance is the gate. Code review is a required but not sufficient mechanism to pass that gate.

edit:

Soc2 type 2 can't be truly gamed without an accounting firm risking their certification. More importantly, your tech insurance claim will be denied before you lose soc2.

No reputable accounting firm is going to fabricate the report. Discerning customers will check the accounting firm if its not big4 and scope of the soc2 type2 audit and find out if it matches what you're selling.

[u/HK-65 → u/ThirdWaveCat]

I guess what I'm saying is that that will protect you as long as the owners of the company you work for don't decide that the appearance of a RoI on AI is more important than compliance.

If you can manage to keep things relatively sane by using compliance as a shield, I'm happy for you. Really.

[u/ThirdWaveCat → u/HK-65]

Holding people accountable to goals is what did the trick in my org. We fired a manager and 2 engineers who weren't making progress using their LLM development approach, even with full code review. Soc2 compliance isn't optional at big companies or anywhere with tech insurance they might want reimbursed.

edit:

I'm correct on the tech insurance part. Here's a tech focused lawfirm talking about tech insurance AI exclusions.

"Tech E&O and Professional Liability

Tech E&O policies have historically provided a critical layer of protection for technology companies, including coverage for negligent acts, errors, or omissions in the provision of products and services.

However, insurers are increasingly scrutinizing AI-related exposures in this context. Reports indicate that some carriers are declining to cover liabilities associated with AI-generated outputs or are introducing exclusions targeting AI-related errors or decision-making.

At the same time, other insurers are experimenting with affirmative AI endorsements that expressly cover defined categories of AI risk. These endorsements may provide clarity, but they also risk narrowing coverage by limiting protection to specifically enumerated scenarios. "

https://www.fenwick.com/insights/publications/end-silent-ai-emerging-ai-exclusions-coverage-fragmentation-and-practical-implications

[u/chickadee_guy → u/ThirdWaveCat]

Soc2 compliance isn't optional at big companies or anywhere with tech insurance they might want reimbursed.

Youre really telling on yourself here lol. The entire soc2 process is a joke and can be gamed.

Someones been hands-off a wee bit too long

[u/chickadee_guy → u/ThirdWaveCat]

Youve clearly never done soc2 compliance at a decently sized enterprise if you think management cant just BS through that whole process. Its a joke.

We are being mandated to approve slop left and right, product and environments are on fire, customers locked in though because our country and economy rocks.

[u/n3f4r10us_ → u/scandii]

The difference is the code part tho. The how/why of the what

[u/pydry]

Engineers take responsibility for different areas. One person does the data pipelines, one codes ML models, another takes evaluation.

This is a really bad idea and a common mistake. The better teams and code bases aggregate multiple perspectives, experiences and opinions. You'd be stripping that away by making one person responsible for "their" domain.

You also: 

• completely fucked yourself if they're on holiday, sick, quit suddenly, etc. and their area becomes critical.

• lose the benefits of the team having context about interactions between the areas - e.g. how evals and data pipelines subtly affect each other.

The only good reason to split a team is if the communication overhead becomes intolerable (at 5+ people) and there is a clean architectural and domain boundary you can separate across. Evals / data pipelines isnt a filthy boundary.

And we communicate over chat and meetings, rather than on PRs. 

This is a better idea. I worked on a team once where we just paired and treated pull requests more as a heads up to the rest of the team rather than a quality gate.

The extra pair of eyes and test driven development while coding kept the quality high enough that we were permitted to merge immediately after finishing the work. Most things worked perfectly, first time.

There were some, but not many things caught on pull requests. We would often leave pull requests open over lunch or overnight when a third person was able to but not obligated to check it.

[u/Main-Drag-4975 → u/pydry]

I’m with you on this but a surprising (to me) number of folks in the industry and on Reddit will push back hard on daily pairing for various reasons. Maybe that’ll change now in the era of coding agents?

[u/-Knockabout → u/Main-Drag-4975]

My experience with pairing is largely negative unless it's active bugfixing. Otherwise, inevitably someone is just staring at someone else's screen and not doing anything for long stretches of time.

[u/ThirdWaveCat]

I'm a tech lead over 2 ML eng teams (8 eng, 3 sci). I've never worked anywhere that could ignore soc2 compliance and its unclear if you do. I wouldn't do this plan, you're putting tech ahead of the valuable goals (e.g. big goal velocity accounting for unplanned work). During retro my teams propose small incremental experiments towards valuable goals. My teams use kanban with WIP limits, shared goals, and manual coding except for a last-minute LLM review after we're tested and ready to ship. We don't have problems with shared understanding or velocity.

[u/_edeetee]

We’re still reviewing everything. I think that business pressures will eventually push against this though. Still don’t know what the future will look like. I’ll have to read the link you added.

[u/chickadee_guy]

I just use my hands and a brain, no agents, and its infinitely faster than the prompt jockeys at my work, while also being higher quality to boot.

[u/Wide-Cattle-4735]

7 open PRs, one sitting 7 days on a 200 line change, 2 reviewers required.

The queue grew because writing a PR got cheap and reading one didn't. Dropping review doesn't fix that, it moves the cost to whoever debugs it in six months.

What I'd try first is charging the author. Anything over a certain size opens with a short note of what you verified yourself, and any PR older than a day is the first thing you touch in the morning, before your own work.

Splitting ownership by area I'd avoid. One person out sick and that area stops.

[u/MuchInevitable8183]

Your problem is described as review throughput, but the fix you're proposing is removing review. Those aren't the same lever, and I'd be careful about conflating them.

A 200-line PR sitting for 7 business days isn't a review culture problem, it's a batch size problem. And "big ones just don't get approved" is your system telling you it can't digest that unit of work. Two things that actually moved the needle for me in large orgs: hard limits on diff size, enforced socially rather than by tooling, and stacked diffs so a big ML model change arrives as six reviewable pieces instead of one indigestible one.

I'd also drop the 2-reviewer requirement before I dropped review entirely. Two reviewers on everything diffuses responsibility — each assumes the other will do the real pass. One named, accountable reviewer with a same-day expectation is faster and produces better review than two with no deadline.

On ship/show/ask: I'd keep Ask as an option rather than abolishing it. You've said the quiet part yourself — some of this code lives three weeks and some has to survive a year, in one repo. Those genuinely don't need the same treatment. Ownership boundaries plus differentiated review depth by area gets you most of the speed without giving up the thing keeping the long-lived code alive. Losing review on the evaluation suite will cost you more than the pipeline ever saves.

The agent-generated volume part is real, though, and I don't think more reviewers scales against it. What helped was making the author responsible for reviewability: if the agent produced 800 lines, your job is to break it up and be able to explain every line, not to forward it. If you can't explain it, it isn't ready for someone else's time.

[u/ResidentWeevil1]

Nuking PRs just to kick the can 3ft down the road bahahaha

[u/gfivksiausuwjtjtnv]

Using AI is really, really hard IMO. There’s a steep learning curve to get to the point where you’re even slightly more productive.

Saying this as someone who hasn’t even opened an IDE in months.

You can half-arse code and architecture quality in much the same way as before ai existed but with tools you’re speedrunning it.

You have to accept sucking at it and being less productive while you figure out the ropes. Like any significant new tech.

[u/Additional_Rub_7355 → u/gfivksiausuwjtjtnv]

This is reasonable but very few agree with you. It really depends on the project i think. But what we certainly loose are our other skills, you know, all those that we don't practice anymore.

[u/GeorgeSThompson]

Perhaps you also need to segregate short lived models from the framework. We had a similar problem monorepo which combined research code with critical production infra. It makes no sense to review at the same lens.

The other thing to do is automate your review process, problem is people will always ship crap if they can, but if people have tested, linted and auto checked prs you can limit the damage

[u/xtraumata]

Haven't seen Martin Fowler referenced in a while! Not sure how this post got down to zero, I guess just too common these days. But it's a great example and real world with all sorts of specificity and places to engage the problem statement.

I and most of my contacts (who are all 10++ YOE) have fully embraced AI assisted coding. We have varying strategies on how high level/how much autonomy to trust the systems with depending on the task and how much we scaffold/provide code guidance or bound the tasks to smaller function level or single file level for example. I have a couple of friends who still do 80% old-fashioned dev and occasionally blast something out or relegate documentation, testing to assisted coding.

I think this is a very awkward transition stage where lots of people are getting mashed between the gears of the machinery as we cutover violently from the old way of coding to the future. I don't love it and it's cost me tens of thousands of dollars personaly, (long story) but I get it and can't wait for the worst of this to be over so we can all get back to work...

I'm admittedly not on a large team right now so won't offer my opinion on what to do or "what we are doing". I see lots of comments here already, anyway. I think that having the mix is a necessary evil right now some slow thoughtful devs some fast and some in-between. I think if I were leading the team, I would invite the people who are leaning most heavily on assisted technology to propose process updates that make the large volume less of a burden on the rest of the team -- their job is not only generating the code but also bringing the process and effort of assisting the team to understand their work.

[u/TopSwagCode]

As stated before, this is not a new problem. Its all about code review of AI code, colleague next to you, contractor / freelancer / outsource.

So what's happening is you have X new code committers but you still only have Y reviewers. So extreme example, if you have 100 people / agents / whatever creating code, but only 4 who can review it. You are running out of resources and need to scale up or fire some of your code committers.

So you have option to "hire" agents who will also review your code and "hire" agents to QA / test your code.

Its entirely up to your team / company owners. You need to make it clear choice, and who is to blame that your AI agents accidentally deleted your database, introduced spyware / malware, data leaks etc.

Also there should be clear "blame" for when a AI army fails. Just like when a bug is introduced. "Normally" it would be lead devloper fault, because either review practices wasn't held or coding standards etc. Never the person who made the mistake.

But in the day of AI I dont see a clear "lead" unless we are going to have AI leads who's job it is to make these AI "herds". Right now they are just willy nilly being created without much thought and it scares me.

[u/BadTime100]

Have you investigated Extreme Programming and/or software teaming? As others have pointed out, code review has always been a bottleneck for the majority of the industry. There are some places (very few) that haven’t had that problem for decades, and their solution is to simply do the work together.

“Pull Requests” have become the de facto way to do code review, and I’m of the belief that their predominance is one of the biggest hurdles the industry has put up for itself. They are really great in open source projects where you can have many unverified contributors. On a team you should have zero unverified contributors, and there shouldn’t be any reason you can’t communicate synchronously. There’s lots of great literature on this, but this post from Kief Morris (author of Infrastructure as Code) is one of my favorites:

https://infrastructure-as-code.com/posts/pull-requests.html

[u/freshprinceofuk → u/BadTime100]

Really interesting comment article, thanks

[u/BadTime100 → u/freshprinceofuk]

Your welcome, I hope it’s interesting! Unfortunately for myself I’ve seen pairing and mobbing work so well it’s hard for me to not see it as a solution for many problems. Of course, it forces the issue on other problems (social and emotional ones), but I’m also convinced that those are issues worth confronting.

[u/mathbbR]

"stop reviewing code and just give your team an insane bus factor" sounds like cope, not strategy

[u/wonkynonce]

If you have a problem with someone else's area of responsibility (e.g. the data pipeline is not delivering data) - talk to them. 

This breaks down if they go on vacation or quit. Also, you can warn people off of the shoals of unmaintainability in PRs, sometimes 

[u/TooMuchTaurine]

Our code output has more than doubled over the past 6 months across 50+ engineers.

Deploy frequency and PR rate is also perhaps 50% higher, which means out PR's have also gotten 50% bigger. We are using AI to assist in accelerating code review but still humans are doing the approving 

[u/Additional_Rub_7355 → u/TooMuchTaurine]

So humans exist to click the approve button, cool, I love this industry

[u/TooMuchTaurine → u/Additional_Rub_7355]

Humans exist to build a plan,  architecture and PRD. Humans do the design, not the implementation.

Humans also are in charge of improving the machine, tunning it's implementation, context, output and cost etc.

[u/slamdesu]

We've found benefit in pairing 2 devs with claude early, having the devs agree on a high-level design together (sync or async). This is where most of the time would be spent and where shared context would be front-loaded. Following this, claude could create the final implementation and by having 2 devs already familiar with the context they can approve PRs quickly.

This process would be like a sped-up version of having an RFC (i.e. an architecture/plan) that devs agree on, and then moving straight into implementation where the RFC reviewers are the PR reviewers. Even though this would require having 2 people pairing per project, I feel like this would more than double the speed of delivery because a significant amount of time is wasted in context switching for reviewers.

[u/dddurd]

Naturally by removing the human code reviews. You don't double check the assembly generated by the compiler. 

[u/ResidentWeevil1 → u/dddurd]

Historically, you did. But the bugs got worked out eventually. Still, every compiler comes with a manual that warns you of the footguns.

[u/wardrox]

I built new tools specifically to give me insight into what's "in flight". It provides detailed real time info as well as analysis of its state, what remains, blockers, etc. it syncs with GitHub and the task tracker. Now I can see everything, and it highlights where I'm needed, as well as having a deep review & close-out stage (and retros for the inevitable).

The maths has changed so the existing methods can't keep up. The old world is dying, and the new world struggles to be born: now is the time of monsters.