Matillion’s engineering organization has grown quickly over the past several years. What started as a small startup engineering team has become an enterprise software company with more than 140 engineers spread across 20+ teams. And like many growing teams, the way they measured and talked about performance didn’t keep pace.
“We were managing by feeling,” says Santosh Poonen, Senior Director of Engineering at Matillion. “We had anecdotal evidence or manual ways of collecting data about our productivity. But we were pivoting from being a mid-level enterprise software company to being an AI-first data-driven leader in our space.”
Before Swarmia, there was no consistent approach to measurement. They’d tried a few different tools in this space, but usage was scattered and ad hoc. Individual teams might have glanced at metrics during a retro, but there was no shared standard for what to track or how to talk about it.
Santosh joined Matillion from Oracle with a mandate to standardize the internal engineering platform. That included figuring out how to measure and discuss engineering performance in a way that worked for the whole organization, not just individual teams.
“We explored different tools,” Santosh says. “We had a smorgasbord of different things we used, but we didn’t have anything that gave us a unified picture.”
Daren Roberts, an Engineering Manager at Matillion with seven years at the company, saw the same problem from the team level.
“A lot of teams started from the place of ‘we think we know where we’re strong, we think we know where we want to improve, we think we know where friction is’ — but it was instinct,” Daren says. “It wasn’t measured.”
Making the commitment
Matillion started with a small proof of concept — Santosh, his team, and a couple of engineering managers using Swarmia. Over the first year, a few more leaders started using it, but they hadn’t yet committed to it as their standard. They decided to change that.
My advice to other engineering leaders is to commit to making it your standard. If it’s not clicking, it’s probably not the tool, but your level of commitment.
Senior Director of Engineering
Once Swarmia became the shared platform for discussing engineering performance, the kinds of conversations leadership and teams were able to have improved significantly.
“We’ve got data-driven transparency now,” Santosh says. “I can go to the engineering leaders and the engineering organization as a whole and say, here’s what the data is telling us. And I have the ability to show them exactly where we stand.”
The buy-in goes beyond leadership, right down to the individual engineer. “If you ask any one of the engineers in our organization, or even the people in product who use it, they’re loving it,” Santosh says. “They’re excited that we’ve got a tool that provides what we need.”
Rethinking how leadership talks about performance
One of the first things Santosh changed was how Matillion ran engineering operations reviews. When he joined, each team ran its own weekly review — well-intentioned, but too granular to surface broader patterns.
“They did a great job, but they tended to be too granular to get much out of,” Santosh says. He proposed moving to quarterly, org-wide reviews with a standardized approach. The problem was that he didn’t have a platform to support it.
“Initially I started curating different individual charts and spreadsheets and slides — a lot of manual work,” Santosh says. “But once we started using Swarmia, it allowed us to have the same platform where from one operational review to the next, I could say, ‘We now know what this looked like three months ago because we’re using the same platform, and we’ve got an apples-to-apples comparison.’”
The quarterly reviews now run for about four hours. And rather than feeling like a slog, they’ve become one of the most productive meetings on the calendar.
You’d think you could never get engineering leaders excited about being in a boardroom for four hours, and have them feeling like we didn’t have enough time. That’s what it’s like now.
Senior Director of Engineering
Developer experience surveys play a big part in those reviews. Matillion runs them through Swarmia on a quarterly cadence, and the team spends a dedicated chunk of each leadership operations review going through every response — scores, themes, and action items — without compromising anonymity.
At each quarterly meeting, leadership shares what they’ve done in response to the previous survey, which measurably changed how engineers felt about participating.
“Survey participation six or nine months ago was around 60%,” Santosh says. “We moved it up to 75%, and then in the last one we did three months ago, we were at 96%.”
Daren sees the same thing from the team level. “The main plus for me when encouraging engineers to respond is that it’s a really easy survey to respond to,” he says. “That sounds simple, but you’d be amazed at how many surveys you receive where you’re two questions in and you can’t be bothered. What I always say is: this is your chance to have a voice, and it’s easy to do. It’s the best format and UI for a survey I’ve seen.”
The surveys also get engineers into Swarmia, where they start exploring on their own. “I’ve had feedback from people going, ‘Wow, I didn’t realize Swarmia had all these data points,’” Daren says.
How engineering managers use the data day to day
While Santosh drives the wider strategy, engineering managers like Daren are the ones putting the data to work in their teams.
For Daren, Swarmia is part of his daily routine. “I’ll log in and just check things,” he says. “I’ll know if the data’s changed, and I’ll go, ‘I wonder why that is,’ and start looking at the detail behind it. It gives EMs a mechanism for going: if something’s changed, why has it changed, and do I need to do anything about it?”
Daren uses retrospectives to bring Swarmia data to the team and open a discussion about what to do next.
It’s completely objective data that doesn’t have an opinion about anything. You can’t dispute truth. But you can drill right down to each data point in Swarmia to see what’s up.
Engineering Manager
The team picks one thing to focus on for the next sprint or time period. It’s a small, repeatable loop: look at the data, ask a question, agree on a focus area, check back next time.
Working agreements in Swarmia give that loop a starting point. “I see them as the starting point,” Santosh says. “They don’t change anything on their own, but they let you know where you’re at over time.”
Engineering managers use them to benchmark where a team stands, and the daily digest emails keep everyone aware of how things are tracking. It fits Matillion’s culture well — one of their values is “no person, product, or process is ever finished.”
Swarmia’s Slack notifications are flexible enough that each squad can configure them to fit how they work. Some teams get notified when pull requests are approved; others focus on stale requests or items awaiting review. “Each squad operates differently, and I think that’s pretty good — they use the notifications in slightly different ways,” Daren says. “We get good information, we take action, and we find it valuable.”
Portable data with Swarmia data cloud
Daren is a data engineer by background, so he was always going to want to get his hands on the raw data. Swarmia’s data cloud export to S3 gave him that flexibility. He built an automated reporting pipeline that runs every two weeks, pulling Swarmia’s raw data into their own internal product, Maia.
For the first time, we’ve got an org-wide view on the same measures, same metrics. Everybody’s reporting from the same single source of truth.
Engineering Manager
For Santosh, having consistent metrics across the organization — whether teams access them through Swarmia or through Daren’s dashboards, or MCP — is what matters. “Every single person has access to the same data,” he says. “That’s the foundation for everything else we’re doing.”
What’s next?
Over the past six months, DORA metrics — especially lead time for change, review time, and merge time — have become central to how Matillion tracks engineering velocity. They’re now looking at expanding deeper into areas like investment balance and AI impact tracking.
For Daren, the value comes down to one thing. “Numbers aren’t answers,” he says. “But they help us ask the right next question.”
And Santosh sees plenty of room to keep going. “What I really like about Swarmia is that it’s constantly improving,” he says. “As we adopt an AI-first mindset, it’s reassuring to see that the tool we’re using has that same mindset.”
