The board wants numbers, the newly hired VP read something about needing AI productivity metrics, and the CFO heard that AI tools should be doubling engineering output. So a well-meaning engineering leader (perhaps you) launches a productivity initiative, starts with measurement, and either sees no improvement or, more often, makes things worse.
In a large organization, that leader is usually under a specific kind of pressure. Software engineering has been a black box while every other part of the business gets measured, and now someone needs a number they can take to the board. “It depends” is not an acceptable answer at that altitude. So metrics feel like the obvious first move: they’re concrete and comparable, and definitely something you can put on a PowerPoint slide.
Across the organizations I’ve led and the hundreds more I watched from my seat at Swarmia, the metrics-first version tends to end the same way. A leader invests in developer productivity, leads with measurement before anything else, and comes out the other side with less trust and less management capacity than they started with. At enterprise scale, the fall is longer, because the rollout is bigger, more people are watching, and a visible failure is expensive to walk back.
Improving developer productivity is a change management problem. Metrics matter, but they earn their place only after you’ve built enough trust and really understand what’s slowing your teams down. Making decisions from dashboards before that work is done is where large organizations tend to go wrong.
Rolling out productivity metrics before building trust creates two kinds of harm: one for engineers, one for managers. In a large organization, both get amplified.
Start with the engineers. They hear about new dashboards tracking their cycle times and pull request throughput, and they feel watched. Most of these metrics reflect things an individual engineer doesn’t control: review queues, unclear requirements, flaky CI, and above all, dependencies on other teams. In a big organization, cross-team handoffs and coordination are often the largest single drag on delivery, and no one engineer owns them. Measure someone on an outcome they can’t move, and the rational responses are to game the number or to go find an employer that treats them like a professional.
That instinct is stronger in a top-down enterprise than almost anywhere else. In a lot of large organizations, teams have limited ownership over business outcomes to begin with; people are heads-down, focused on keeping their jobs and getting promoted. Drop a surveillance-flavored metrics rollout into that environment, and it confirms the most cynical read of the place. (Software development is collaborative, and chasing individual metrics breaks the collaboration that makes teams work, something we’ve written about at length.)
The constraint is tighter now than it was a few years ago. When engineers can triple their code output with AI tools, the temptation is to treat that output as proof of productivity. Tokenmaxxing, as the kids call it. But if the bottlenecks stay the same size (review queues, coordination overhead, the four other teams your change depends on), most of that extra output has nowhere to go. The code is faster, but the system isn’t. And where agents are opening pull requests on their own, PR count has stopped measuring anything meaningful (or did it ever?)
The second harm is that instead of removing obstacles and coaching engineers, managers get pulled into dashboards and into meetings about why the numbers moved this sprint. Metric-tending, basically. In a large organization this compounds, because the numbers travel through layers (team lead to EM to director to VP), and context falls away at every step up.
By the time a figure reaches the executive who set the target, it’s a decimal point detached from the reality that produced it. That executive has almost certainly never met the engineer the number describes, which means the trust the whole exercise depends on can’t be built face to face. It has to be built deliberately, and metrics-first skips the part where you’d build it.
The usual “just start small and listen” advice isn’t enough for running this at scale: you can’t pilot in a small corner, in the dark. An enterprise leader owes the executive team a coherent, comparable picture across dozens of teams, on a schedule. A startup can try something with one team and say nothing until it works; a large organization rarely gives its leaders that room. Do nothing until trust is perfect and you’ve missed your reporting cycle and that one exec is still on your back about getting a number. Roll out fifty inconsistent dashboards and you’ve got noise, no shared story, and fifty teams who feel measured for someone else’s slide.
This is why leaders default to metrics-first, and it isn’t laziness. The job comes with an obligation to report upward, and measurement is the only part of the work that looks like an answer. Better to say that plainly than pretend it away: you need enough standardization to tell one story across the organization, and you need to earn that standardization rather than impose it on folks cold.
Improving developer productivity means asking people to work differently: new review practices, a different way of breaking down work, whatever the specifics. That’s behavioral and cultural change, and the central fact of change management is that people resist what they don’t understand or trust. Before you can change how a team works, you have to understand how they experience the current situation and what would make change feel safe rather than threatening.
Most engineering leaders haven’t built that muscle, and large organizations select against it. Engineering culture rewards technical problem-solving; people get promoted for shipping. The feedback loop for shipping is fast and unambiguous. It shipped or it didn’t.
Organizational change is slow, messy, and has no pull request to merge. Metrics feel appealing because they feel like something concrete to do. But skipping the change work is exactly why productivity efforts fail. You can have the cleanest data in the world, and if people don’t trust the process or don’t feel heard about what’s slowing them down, the metrics will only document the decline.
We built an engineering intelligence platform at Swarmia, and we’ve learned the hard way that the tool only works when the approach behind it is right. What I’ve seen work in large organizations starts small and leads with listening, with one adjustment for the reporting duty above.
Metrics come in once trust exists and teams have the context to read them. At that point the data becomes a shared language that teams use to track their own progress, a different dynamic from dashboards rolled out on day one for leadership to watch.
Be deliberate about which metrics. In teams using AI coding tools, individual output numbers like PR count or lines committed mean even less than they used to: when an agent can open a pull request in the time it takes an engineer to clear their Slack inbox, those numbers have stopped meaning much.
The useful signals are system-level: where effort is going, and whether production stability, quality, and cycle time hold as output climbs. Our guide to engineering metrics for leaders covers what those look like in practice. They’re questions about how the whole system works, and they only help once teams trust that the data is there to support them, not to score them.
Yes, this takes longer. But the improvements tend to last, which is more than most metrics-first rollouts end up with.
If you have 90 days to show results (and many leaders do), the approach still works, but what changes is what you can credibly claim. A clear picture of where teams are stuck, and a pilot visibly working with engineers who are engaged rather than gaming the system: those are results, and they’re ones you can defend in a review.
Chris Pope, an engineering leader at Unity, described a version of this at a session we ran recently. He’d inherited a team with almost no visibility: fragmented infrastructure, engineers spread across seven live services, and a lot of the team’s knowledge already out the door.
So he started with a question: where is our effort actually going?
Swarmia’s investment balance confirmed what he’d suspected: too much sustainability, not enough growth. From there he added metrics one or two at a time and set working agreements with the team rather than for them. Cycle time improved 33%, review times dropped by 70%. His advice: introduce metrics with your team, not to your team.
“Introduce metrics with your team, not to your team.”
One other thing to keep in mind is that engineering teams can, in fact, want metrics. They can even appreciate metrics. When a number puts a problem they already live with in front of leadership — reviews that drag on, a release nobody wants to touch — it becomes the case they’ve been missing for getting the blessing to go fix it.
The leader gets an improvement to take upward and the team gets to fix the system that’s been frustrating them, and it’s a win-win. And it spreads: once one team fixes what’s been slowing them down, the next has a reason to opt in rather than brace for it.
If you want a more structured way to think about the stages of introducing metrics to a larger organization, the BRAINS roadmap lays it out in six steps, starting with understanding your baseline and researching what your teams experience before jumping to solutions.
None of the steps above produce anything if you’re going through the motions, and it’s easy to go through the motions, especially in a large organization. You run a developer experience survey, read the results, and then do what you were going to do anyway.
Listening means sitting with what engineers tell you, even when it’s uncomfortable. Especially when the bottleneck turns out to be organizational: a process you designed, a team structure that made sense on the org chart but creates friction in practice, a dependency between two groups that no one owns. The instinct is to defend the structure, or to reframe what engineers are telling you as a communication problem. The more useful instinct is to assume that people mostly intend to do good work, and that when they can’t, something is getting in their way.
Lead with metrics and surveillance and you get more than a failed productivity push. You erode the conditions that make improvement possible in the first place. Engineers who don’t trust their leaders won’t tell them what’s broken. And in an organization large enough that leaders can’t see the work directly, being told is the only way they’ll ever find out.
Subscribe to our newsletter
Get the latest product updates and #goodreads delivered to your inbox once a month.
