If you’re an engineering manager and you haven’t been asked lately about your team’s productivity, consider yourself lucky: optimizing the value created by an engineering organization seems to be on everyone’s minds since layoffs started hitting the tech industry in force in 2022.
If, on the other hand, you’re having to answer questions about improving your team’s productivity, ChatGPT might tell you (I asked) to consider whether you’re just under-resourced or need to do more up-front planning, and I can assure you the answer to both is “no.”
In this blog post, I discuss some of the typical bottlenecks engineering teams face, common fallacies to avoid, and what your team can do to keep productivity improvements on track.
What’s really, really slowing you down?
To uncover the real issues that are causing your team’s productivity to dwindle (or to be perceived as dwindling, also not good!), we’ve seen time and again that when teams focus on improving just a few key areas, the payoff comes quickly:
- Workflow
- Priorities and WIP limits
- Keeping the lights on (KTLO)
- Manual work and toil
- Decisions owned outside the team
Let’s look at each of them in more detail.
Workflow
Workflow bottlenecks can manifest as consistent delays in task completion or certain types of tasks frequently getting stuck.
Assess the entire work process from start (for example, when work starts on a ticket) to finish (for example, when the code is in production). Identify where the hold-ups occur and why: are code changes waiting for review for days? Is one team member working alone on a project that only makes progress when they’re not fighting fires?
Implement ideas to streamline these areas, whether it involves redefining processes, improving communication, or utilizing better project management tools. Encouraging a culture of regular feedback and continuous improvement can also significantly enhance workflow.
Priorities and WIP limits
Without clear priorities and manageable work-in-progress (WIP) limits, teams can become overwhelmed, decreasing focus and productivity.
Clearly define and communicate team priorities. Implement WIP limits to ensure that team members are not overburdened with tasks and that the team isn’t working on too many things at once — always a number that is less than the number of people dedicated to your team. Regularly review and adjust these priorities and limits to maintain alignment with overall project goals and team capacity.
Importantly, never aim for 100% utilization of your engineers: 75-85% is more correct for preserving a team’s productivity.
Keeping the lights on (KTLO)
A high burden of KTLO activities, such as maintenance and dealing with technical debt, can detract from new development and innovation. You can employ a few approaches if a team is swamped with KTLO work.
- Prioritize and delegate. Not all KTLO work is equally important. The team should take time to evaluate their KTLO tasks and prioritize them based on their business impact. The low-priority, non-strategic tasks could be automated, outsourced, or temporarily ignored, allowing the team to focus on higher-impact tasks.
- Invest in automation. If a significant proportion of KTLO tasks are routine and repetitive, the team could invest in automation. This may involve using existing tools or developing custom solutions. Automating repetitive tasks can free up significant time, allowing the team to focus on more strategic work.
- Reduce technical debt. Too much KTLO work could be the result of substantial technical debt. For example, a codebase that’s full of one-off exceptions for individual customers can make any change risky; this variance should be managed via configuration, not code. Regularly allocating time to reduce technical debt — through refactoring, improving test coverage, updating documentation, etc. — can reduce the amount of KTLO work over time.
- Reconsider the product roadmap. If KTLO tasks hinder progress, it might be time to revisit the product roadmap. Balancing new features and improvements against maintenance tasks is crucial to ensure the team can deliver on strategic objectives over the long term.
- Ask for more resources. If KTLO tasks are overwhelming and the approaches above aren’t enough, the team might need more help. This could mean hiring more team members, reallocating resources from other parts of the organization, or using third-party service providers.
Manual work and toil
Related to KTLO, manual, repetitive tasks can consume a significant amount of time and are prone to human error, slowing development.
Identify tasks that can be automated, such as testing, deployments, rollbacks, configuration changes, customer support requests, and the like. Invest in automation tools and practices to streamline these processes. Regularly review your workflow to find new areas where automation can be applied.
Decisions owned outside the team
When centralized outside the immediate team, decision-making can lead to delays and dependencies that slow or straight-up block progress.
Move decision-making as close to the team as you can. Establish clear boundaries for decisions that can be made internally and provide the necessary information and authority for teams to make informed decisions. For decisions that must be made externally, establish faster and more efficient communication channels with the decision-makers.
Addressing these bottlenecks enhances the team’s productivity and contributes to a more empowered and engaged workforce. Regularly revisiting and adjusting your approach to these challenges ensures that your team remains agile and capable of adapting to changing demands and circumstances.
I’m sensing some themes
As I mentioned above, some organizations may find themselves initially allergic to these ideas because they contradict a well-established culture. And yet research, math, and observation all show us that they work.
Empowered teams
Teams need to have control over their work. This includes autonomy in decision-making, particularly for decisions that directly impact their workflow. By empowering teams to make decisions internally, you reduce dependencies on external approvals, speeding up the development process.
Continuous improvement
Whether addressing workflow bottlenecks, managing WIP limits, or automating manual tasks, the focus is consistently on making processes more efficient. This involves identifying inefficiencies in the current workflow and implementing tactics to streamline these areas. Automation is crucial here, especially in reducing manual work and toil.
Outcomes over outputs
Be explicit about the outcomes the team should try to drive, leaving room for the team to decide how best to achieve those outcomes. Never over-index on outputs or activity metrics.
Across each of these themes, there’s an underlying focus on optimizing the value created by the team. This means moving away from just being busy to being actually productive.
Convenient fallacies to avoid
Certain fallacies tend to come up whenever you’re talking about bottlenecks. In the course of your conversations, it will be tempting for you or for someone to say things like:
“We aren’t doing enough up-front requirements gathering”
Detailed up-front requirements are not just unnecessary; they can be detrimental when they restrict a team’s ability to adapt and evolve as projects progress. In reality, the most successful projects are those that embrace evolving requirements, allowing for innovation and responsive changes along the way. Adhering too rigidly to initial specifications typically results in inefficiency and can stifle creative solutions.
“What we really need is more people”
The notion that insufficient staffing is a primary bottleneck overlooks the underlying issue of work-in-progress (WIP) limits. Simply adding more staff to a project does not solve productivity problems; it often exacerbates them due to onboarding costs and increased coordination challenges. Productivity stems from managing and optimizing the workload and capabilities of the existing team rather than indiscriminately increasing team size.
“We just need to plan better”
Extensive planning is often mistakenly idolized as the key to successful project execution, but over-planning can bind a team to a trajectory that may become irrelevant as project dynamics evolve. Effective planning requires balance — it provides direction, but not so much that it impedes flexibility and rapid response to change.
“It doesn’t work that way here”
Perhaps when you hear this, you’ll have stumbled upon the exact problem, but it’s not one you can solve with code or perhaps even within your team. Culture change might be needed to embrace all of these recommendations, and culture change is scary and hard.
Generally, be especially wary of claims that more processes will make things go faster. Be skeptical when someone suggests a headcount fix — unless they’re suggesting forming a platform team. Remember that any proposed fix that changes a team’s size, shape, or remit can hurt productivity for months or more.
Change is hard
There’s no switch you can flip to make things better one day. Improving productivity has to be a sustained practice, not an occasional fire drill. Within your team, there are a few things you can do to keep an improvement effort on track over time.
Embrace experimentation
Encourage your team to experiment with new methods, tools, and processes. WIP limits are especially interesting to experiment with if you haven’t before. Create an environment where experimentation is allowed and part of the team’s culture. Whether it’s adopting new software tools or implementing automation, these experiments can lead to significant enhancements in productivity.
Make productivity a regular discussion point
Continuous improvement should be embedded in your team’s routine. Regularly discuss topics like workflow, priority management, and automation opportunities. This keeps the team focused on productivity and encourages a culture of ongoing improvement. Use these discussions not only to identify areas for improvement but also to plan and commit to specific actions. Keep the focus on the team’s way of working, not on any individual.
Engage your stakeholders for support
Involve business stakeholders in your improvement initiatives. Their understanding and support can be pivotal, especially when changes impact timelines or require resources. Be transparent about the benefits of proposed changes and their alignment with business goals, and lean on the business to support you with time, tooling, and training.
Celebrate your progress
Implement metrics that accurately reflect the team’s improvement over time. This could include tracking DORA, SPACE, or other relevant metrics. Regularly review these metrics to assess whether your changes are working. This data-driven approach not only helps fine-tune your approach but also provides tangible evidence of improvement, which can be motivating for the team and reassuring for your stakeholders.
Further reading
This post was heavily influenced by The Principles of Product Development Flow: Second Generation Lean Product Development by Donald G. Reinertsen. It does an incredible job laying out the math and research that suggests that if you really want to maximize the productivity of your software team, there really is no other way.
It’s no coincidence that Swarmia provides several ways to enable the approach I explained in this post. You can read about them further:
- Establish clear, shared expectations around how your team works by adopting Working Agreements in Swarmia.
- Keep track of your team’s progress and identify bottlenecks with the Work Log.
- Use the Investment Balance view in Swarmia to ensure your team’s time is going to the right things.
- Keep your team’s objectives aligned and on track with the Initiatives feature in Swarmia.
