Before joining Swarmia, I worked at Stripe for two years in its developer productivity organization. The single most interesting thing I got to work on was the engineering effectiveness initiative. I advocated for and established a team and a measurement framework for improving the cross-cutting developer experience at Stripe, focusing not just on individual tools but rather the overall experience of shipping something to production.
In its first few months of existence, the team focused on engaging directly with software engineers. Quantifying their struggles reshaped the focus and priorities for engineers across dozens of teams in multiple organizations. The work had visibility at frighteningly high levels of leadership. Even far from the tech org, teams across the business came to us to learn what had worked and what hadn’t.
The bad day button
We started with a vague idea that we would make it really, really easy for engineers to complain about things. We called the complaints submitted by engineers “paper cuts” and added cute “bad day” buttons to the UI of most engineering tools to make paper cuts easy to submit. Engineers could tell us what was going wrong for them at the moment — in their editor, on a wiki page, in Slack, in GitHub.
Those buttons got clicked. A lot. Every paper cut turned into a Jira ticket — and showed up at the top of my inbox. Before long, we hit ENGX-1000. A small group of scrappy engineers on the team got to work on fixing the low-hanging fruit. There was a lot of it, stuff that no team would ever prioritize unless that team's job was to prioritize exactly this.
We quickly realized just how many blind spots we had in the lived experience of software engineers. Internal tooling teams would tell us, “We’ve never gotten feedback like this before!” Paper cut reporters would frequently include messages about their gratitude for a forum — even to vent.
At first, some teams were worried we’d get between them and their users; other teams that already felt underwater bristled involuntarily at the all-too-human reminders of where their software was falling short.
For that first group, we focused on providing them with tools to better understand their users, piloted user shadowing, shared our learnings and helped partner teams do the same. For the second group, the task was to convince teams that the information we gathered would shine a user-centric spotlight on the problems they knew they wanted to fix.
“Go where you’re wanted” is some of the best advice I’ve ever gotten (thanks Donal). We leaned in hard when teams wanted to work with us. We helped them identify and quantify user pain and even lent an engineer when that was helpful.
Paper cut surprises
When we started the team, a lot of us — including me — thought that our focus would be on identifying paper cut themes and then, in response, building “paved paths” over bumpy, sparse trails.
We learned there’s an enormous opportunity just in quantifying software engineers’ day-to-day experience writing software and how it’s impacting the business. Other teams are usually better suited to building the paved paths, but the paper cuts program brought a voice — and data — that had been missing.
A core tenet of mine is that software engineers are grown-ups who mostly want to do the right thing. We were explicitly not giving secret demerits to people for reading Hacker News. We wanted to make sure writing and shipping software is easy when writing and shipping software is what people want to be doing.
Sometimes, those opportunities existed in optimizing a tool’s raw performance or in spotting tooling regressions earlier. But sometimes, they existed in quantifying what it costs to seek that second stamp of approval on certain kinds of changes, in recognizing regional inefficiencies in the distribution of automatic code review requests, in driving down tiny but constant interruptions.
Though we focused intently on paper cuts at first, eventually, we looked at them more as data points for ongoing prioritization across the company vs. a priority for the team itself.
Takeaways
- Show empathy to your (internal) users. We assumed that our developer users were grown-ups who wanted to do the right thing, not people trying to slack off. Sure, there will always be underperforming engineers, but if your productivity efforts are focused on finding them, you will quickly create an us vs. them atmosphere. We emphasized a “we’re here to help” message to the developers, and we were explicit about how data would and wouldn’t be used.
- Cooperate and align with teams. We didn’t try to do it all on our own. Cooperation with other teams, within and outside of the developer productivity organization, helped everyone understand their users better and to start to use user-centric thinking as they developed their future roadmap.
- Pay attention to the small things. We didn’t just focus on big problems; indeed, most of the problems we spent time on were “small,” insofar as they’d never bubble up to the top of anyone’s backlog. At the same time, those small things added to a lot of toil and dissatisfaction across the entire engineering organization.
- Reflect and continue learning and improving. This was 100% a journey of continuous learning, reflection, and adjustment. We adapted our approach and our messaging as we learned new things. While we started out focused on paper cuts as a way to identify isolated opportunities for improvement, in the end, the paper cut pipeline became rich source of data points for broader prioritization.
This post is based on a Twitter thread from 2022.
