Maryanna Rayko leads TrueLayer’s 100-person product engineering organization, who are in charge of building and maintaining Europe’s leading open banking payments network. As the VP of Engineering, one of Maryanna’s main goals is to introduce more structure to the engineering organization so they can scale more effectively.
As a part of the initiative to improve efficiency across the engineering organization, Maryanna started rolling out three types of metrics:
- Product metrics
- Operational metrics
- Productivity metrics
While product and operational metrics were relatively easy to surface with a combination of internal systems and tools like OpsLevel, the productivity aspect turned out to be the trickiest to measure. TrueLayer had previously tried (the now-discontinued) Athenian but had abandoned it because it offered little to no transparency into how different metrics were calculated, resulting in the engineers and managers becoming reluctant to engage with the tool.
In addition to empowering teams, Maryanna was also interested in understanding the cost of KTLO (keeping the lights on) to make informed decisions about prioritizing and addressing tech debt as well as other KTLO reducing initiatives.
CTO Luca Martinetti, on the other hand, needed to measure engineering investment and understand the return on that investment. This became especially important as the engineering industry at large entered a cost-efficiency mode due to rising interest rates.
Increasing visibility at every level of the engineering organization
Maryanna and the managers in the product engineering organization started looking into tools that would help solve their two key objectives: empowering teams to make better decisions and understanding the return on engineering investment.
From the first meeting with Swarmia, Maryanna understood that there was perfect philosophical alignment between how she thinks about the problem space and how Swarmia approaches solving those problems.
Measuring the return on engineering investment
For Maryanna, Luca, and the rest of the engineering leadership team, the investment balance view in Swarmia answers many of the questions they have about where teams are spending their time and whether there are opportunities to allocate engineering effort better.
The investment balance view gives the engineering leaders much-needed visibility into how much time different teams spend keeping the lights on. TrueLayer’s investment category setup in Swarmia also allows them to see the amount of the previous year’s feature work that has rolled over to the new year, which helps them improve planning and prioritization going forward.
The Initiatives view offers another valuable lens into product development work, allowing engineering leaders to quickly see the status and progress of ongoing cross-team initiatives so they can proactively jump in and support the teams that may need it.
Currently, the TrueLayer team is evaluating the feature and comparing it to Jira Advanced Roadmaps. While a wider rollout has yet to happen, many of the engineering managers already prefer Swarmia’s Initiatives view as they find it easier to use than Jira’s corresponding solution.
Enabling managers to eliminate blockers
Silvia Di Nardo, a Senior Engineering Manager in TrueLayer’s product engineering organization, says Swarmia has quickly opened the doors to many valuable conversations in and between teams.
Engineering Manager Michael Slack has found a lot of value in Swarmia’s Working Agreements, as they provide a solution to something he’s been struggling with as a leader: keeping the team accountable to the targets they set for themselves.
Michael’s team has also started to regularly review their Work Log in Swarmia to see all ongoing work at a glance, identify patterns, and assess their progress.
Empowering engineers with better insights and automated feedback loops
While Maryanna and the engineering managers spend a lot of time in the Swarmia app, the engineers at TrueLayer primarily engage with the platform through its two-way Slack notifications. This way, the teams can automate some of the feedback loops that are crucial for their workflows.
81% adoption in three months and unique value for everyone in the organizations
In the first three months of using Swarmia, TrueLayer has seen impressive adoption numbers, with 81% of the engineering organization fully onboarded. Over half of the engineers have also started using Swarmia’s Slack notifications. As a result, after the first quarter, 69% of the users rated their overall experience with Swarmia as either good or excellent.
Additionally, the teams in the product engineering organization have seen impressive productivity improvements over the first three months of actively using the tool. Namely, they’ve cut 18% of their cycle times, improved their already remarkable code review rate by 2%, and reduced time to first review by 30%.
With the product adoption already at an exceptional level, Silvia expects it to increase even further as they’re moving their Swarmia login to Okta, allowing even more people at TrueLayer to get into Swarmia.
The engineering leaders at TrueLayer are also looking forward to the launch of Swarmia’s survey feature. Ultimately, it’ll allow them to correlate developer experience data and feedback from their engineers with the system metrics that are already available to them in Swarmia.
