The conversation dives deep into what actually drives — and what kills — productivity in engineering organizations. Max shares hard-won insights about why most developer experience initiatives fail (hint: it’s about not understanding your users), the critical difference between being productive as individuals versus as teams and organizations, and why treating engineers as “ticket takers” destroys software quality.
Watch the episode on YouTube →
(0:00) Introductions
(0:48) Max’s journey to Capital One
(2:58) Unsuccessful developer productivity
(5:37) Doing the ‘right’ work
(11:01) Ticket-taker culture
(15:25) Drivers of productivity
(20:32) Performing KTLO work
(25:40) Drivers and detractors of effective software delivery
(33:16) Setting goals
(40:42) The future of coding with AI
(46:36) Max’s advice to junior engineers
Rebecca: Today on the show, I have Max Kanat-Alexander. He's an executive distinguished engineer for developer experience at Capital One. I've been following Max's career since he was at LinkedIn, and I know there's quite a bit more before that to talk about. So I'm really excited to talk to Max today. So, Max, welcome.
Max: Thanks, Rebecca.
Rebecca: I'm, like I said, so excited to talk. And first, I kinda wanna understand, you have established yourself pretty thoroughly as a thought leader in this developer productivity space these days. I'm loving what you're posting on LinkedIn lately. So much goodness that you are putting out into the world around this space. How'd you end up here? How did this end up being your thing?
Max: Oh, sure. So I'll tell you the short little story of my career. I'll try to compress it into a short period of time.
So once upon a time, I was the chief architect of the Bugzilla Project, and me and the team helped make that into one of the most popular bug tracking systems in the world. Then I went on to work at Google, where I was the technical lead for Code Health, which involved Google's engineering practices, code review, testing, helping people refactor legacy systems, all sorts of things like that.
Then I went on to LinkedIn, where I was one of the tech leads of the Developer Productivity and Happiness organization, and in particular the tech lead of a team called Knowledge Systems that did things like developer feedback and metrics systems as well as knowledge management and discovery and quality and various other things like that.
And now I'm at Capital One.
Rebecca: So what brought you to Capital One? What is the, to the extent you can talk about, what's the challenge? Or is it just the same challenge everybody else has of “we need to do more with what we have?”
Max: I think the thing that excited me about Capital One was that the company really genuinely cares all the way up the executive strata – and I mean all the way to the top – about making the lives of software engineers better and transforming how we build software as a really key part of our business. So it's not a place where you have to go and you have to convince everybody that it's worthwhile. It's a place where everybody's fully committed to really doing the thing.
Rebecca: And Capital One, there are other banks like it now, but it grew up on the internet, right?
Max: It definitely has had a long history that has been heavily involved with data and technology, and went through a very significant cloud transformation and technology transformation over the last 15 years in particular.
Rebecca: Yeah. And that's definitely the story I know, and I know lots of people who have worked there as part of that transformation. Well, thanks for sharing that. I just wanted to kind of establish that you know what you're talking about here. And, yeah, been impressive to watch your journey.
I really wanna talk about, at LinkedIn, you worked on a developer productivity framework, or framework for thinking about developer productivity and developer experience. And I don't wanna dig into that so much as I would just wanna talk about your experience with developer productivity in general. I'm sure you've seen patterns and things that work and things that don't.
So maybe we can start there. I'm especially interested in any unsuccessful efforts you've been around that tried to increase or improve productivity.
Max: Sure. I think the number one way to not succeed in developer experience is to not understand your customers. And when you work in developer experience, your customers are the developers at your company. All the failed efforts that I've seen have been building a tool that didn't understand its users.
Very often, developers just think, “Hey, I'm a developer, so I know what problems developers have,” and they build a tool for themselves. And for everybody who's like them, it works. But it turns out that at a software company or in any kind of software engineering organization, not everybody is like you. In fact, if you shoulder surf some other software engineers, you sit behind 'em and watch them work for a little while, you will be astounded at how differently they work individually.
And not in a way that's bad, it's just in a way that their mind works differently, their backgrounds are different, sometimes their requirements are different. If you really wanna have a wild experience, if you're a software engineer and you've never seen a machine learning engineer work? Go watch a machine learning engineer work, and it will blow your mind. It is nothing like software engineering normally is for most software engineers.
Rebecca: Well, now I wanna go do that. I've watched engineers work. I have not watched an ML engineer work. So, that’s fascinating. Yeah, I hear you, and I talk about it as kind of like product ownership. There is a developer experience. There is a developer experience in a software organization. Whether you own it or not, it still exists. And I love to talk about managing that as kind of a product, like you said.
And I've also seen cases where teams that were in a developer productivity organization were building tools for developers without ever talking to the developers. They just built it according to the spec, and it turned out that their tool was one of the biggest bottlenecks in the software development process, because getting provisions and such was actually quite difficult with the system that they had built.
I'm curious, one thing. There's lots of frameworks out there in the world. There's SPACE, there's DORA, there's DX Core 4, and probably others I'm forgetting. One thing that I feel like is lost in this productivity conversation sometimes is the “what you're working on” part. Yes, it's great to look at cycle time and how fast you're getting code out the door, and how frequently you have issues in production that you have to deal with, and all that.
But I'm curious how you think about this problem of “it's not enough to do the work. It has to be the right work.” These frameworks exist, but the frameworks don't necessarily address this business outcome piece.
Max: Yeah, so the reason that I wrote the LinkedIn Developer Productivity and Happiness framework, which I know you said we don't wanna get into too much, but I wrote it for a reason, okay? And the very first reason that I wrote it was that my job at LinkedIn for a while was to go around and try to figure out what metrics teams should have and get them to implement those metrics and then sort of centralize and standardize them. And I routinely found that when teams couldn't define metrics, it was because they didn't know what their goals were.
And so the most important part of doing any work is actually knowing what you are trying to do, and not like “I want to make the number go up.” That's not what you're trying to do. You're trying to have some effect on the physical universe. Unless you're writing an adding machine, you're not trying to make the number go up.
You start off and you say, “Hey, I want to delight my users by providing them accounting software that is totally straightforward for non-experts to use.” That immediately scopes your whole project, gives you a focus, and tells you where to go. And once you've done that, then you really wanna understand the problems of your– I mean, hopefully that's come out of an observation.
If you really think about it, great software engineering has a lot of similarities to the scientific method, and one of the parts of the scientific method that people most often forget is observation. That even before you're going to hypothesize, you need to have observed something in the physical universe that caused you to form that hypothesis. There needs to be some reasonable basis for you having a hypothesis. Your goal can be the hypothesis. Your goal can be, “I believe that people need great accounting software. That people who are not experts need great accounting software for their small businesses. Or maybe for their individual lives.” See, even that you'd want to go out and you'd wanna find out. And my experience is that once you have sufficiently understood the problem, the solution is obvious. And then what you need to do, and man, I have written a lot about this whole thing. There's more to talk about than I could talk about even in the whole time we have together here.
But you need to do it incrementally. You need to deliver value as you go. You need to figure out what's the smallest core of your product that you could deliver that would provide any value, and then build on that. And real value, not bad value, don't ship a product to people that's gonna hurt your reputation. And then every step of the way, you need to handle the heck out of every problem that comes up as you go. Do not leave them for later. And people often ask, what's the definition of done? And yes, for every individual different software product, you can have a concrete definition of done, but there is a generic definition of done for software problems. And the generic definition of done is “no human being has to pay attention to this problem.” And that's what done means.
And in an absolute sense, you never get there. But in reality, you do get there because there's things that the engineer is thinking about at night, they're thinking about in the shower, they're thinking about when they're taking a walk, they're like, “oh, I still gotta handle this thing. Oh man, I gotta file a ticket for this.” Handle all of those things. All of them. There's trade-offs, right? Like you running a business, you gotta make money. You have a certain amount you can invest in solving your problems. But you should err on the side of just really handling the heck out of it and then going on to the next problem.
And if you do that, you will find that your ability to deliver increases and increases instead of decreases and decreases over time. And you'll be able to meet your goals, and you should stay afloat as long as you have also a functioning business around you. You can't just sit there and write software and have none of the rest of the business.
Rebecca: I hear you and, yes, obviously, of course, everything you just said. And I've worked in places, and I feel like I even work in a place now where we're pretty good at that. Nobody's perfect.
But I also remember a couple of years ago, I was at a conference and talking to a senior software engineer. I don't remember the company. I wouldn't say it anyway if I did. But they were senior software engineer, and we got to talking, and I asked them, “How do you understand the impact of your work?” And it was very clear by the look on their face that that topic hadn't come up before. They were ticket taker,s and that was what they were doing. And hopefully that was enlightening to them when we had that conversation.
But how do you talk to leaders who think that's okay? Who think it's okay for their developers to just be ticket takers and to have their impact is that they finish tickets. I like to forget that those people exist, but in a lot of ways, I think there's still a lot of the software industry that works that way. You're not working at one of those places. I'm not working at one of those places, but plenty of people are working at one of those places still. The long tail of software engineering definitely has that. So what would you say to the leadership in an organization that's making that choice?
Max: Okay, so yes, software engineers that you employ full time should never be ticket takers, and one of the biggest reasons is it destroys the quality of your product. In fact, you could describe all engineering as the process of making the right decisions in the right sequence and taking the right actions as a result of those decisions, in sequence. Right?
But the thing about software engineering is it is a million tiny decisions constantly. One piece of software of any scale or size is composed of millions of small decisions. And unless the engineer knows what they are going for and owns the impact, owns the result, feels some responsibility in themselves for doing a good job, they can't make those decisions correctly. They, at the very least, have to understand why they're doing what they're doing, and at companies where engineers are ticket takers, their leaders are often very frustrated and are wondering, “Why do I keep getting low-quality products out of my engineers?” And sometimes they double down on making everybody ticket takers. They say, “Don't make decisions, don't be independent. Just do what I say, let me review everything.”
But that leads to an engineering organization that actually produces worse and worse quality software, and the right way to do it is encourage independence and when people make mistakes, have a process that allows for safe learning and growth, and help people become better at their jobs. But don't just give them instructions and give them no leeway. Give them why. Make sure they own the result of the thing and allow them to make the best decisions for that product.
Rebecca: And at least in any of my recent roles, the impact has been a huge part of how I was evaluated, right? It wasn't just that they encouraged me to know what I was doing, but they judged me on whether I knew what I was doing and why, and if I was making good decisions around that. And I think having that be part of the performance culture that you need to know, why did the work I just did matter? And it's always a rude reminder that not everyone is thinking that way.
Max: Yeah, and I've definitely seen organizations where people have trouble managing their engineers because of that culture, because it's very difficult to tell. If those engineers don't own the impact, it can be very difficult to understand how they're performing. You end up having very subjective conversations during performance reviews. And it's okay to have those subjective conversations, but you also wanna be having some objective data.
There might be people who are not great at promoting themselves, who are not the most socially outgoing people, but who are great engineers, and they should have a way of objectively demonstrating that they did great work.
Rebecca: What other drivers of productivity do you swear by at this point?
Max: Well, that's such a broad question. You asked the biggest question, right?
Rebecca: Let's start at the team level because I kind of wanna pick this apart at the team and the organization level because I think there's really different things. To make a team more productive, there are a set of pretty well-established practices that you can follow. Let's talk about what those are.
But I also have this hypothesis that, at a certain size – I don't know exactly what this size is, but at a certain size – teams stop being where your leverage is. And you need to start finding leverage across the organization instead of within individual teams. What are the drivers of productivity and maybe also the detractors from productivity in an engineering team?
Max: So I will answer a little bit of your larger question, which is that on the whole, the thing you wanna optimize for, especially if you work in developer experience, is the speed at which the company moves as a whole. And when that is in conflict with optimizing for an individual engineer's productivity, you need to choose the company's productivity.
The time when this comes up the most often is in conversations about programming languages, where an individual engineer might be more productive in an obscure programming language. But if they are allowed to write that tool in that obscure programming language, when they leave, the company will be stuck with this unmaintainable piece of software. And also, it'll be very hard to onboard other engineers onto that team and that code base.
So you know, there are things where standardization speeds up the company even when it slows down individuals. And I fully understand how frustrating this can be. And the time when it's bad is when you try to standardize things that shouldn't be standardized, which means decisions that developers actually do need to make in their day-to-day lives. But the time when it's good is when you standardize things that developers don't actually need to make a decision about.
If everybody else at the company is writing in Python, and Python is an appropriate tool for the problem that you're trying to solve, you need to write it in Python. Regardless of what your opinion is about the programming language. If everybody's writing in C++, same thing. As long as it's an appropriate tool. If you have a technical argument that's, “no, this is literally not an appropriate tool. Here's the technical argument why,” that's a good discussion to have. But if it's just a preference issue? I understand. I have many preferences too. But that does not optimize for the business.
Now let's get down to the individual team level. There is a similar thing here where you wanna optimize for the speed of the team. This is, for example, one of the values of having code review. As an individual, I go faster by myself, but as a team, we actually go faster by enforcing consistency in standards and code quality through code review.
And then at the team level… once again, we need to go back to goals and the goals break down fractally, so the team needs to know what its overall goal is. The team needs to know what impact it's having and how it is supposed to drive that impact, or what it's going to do. But each individual engineer on the team needs to know what they are supposed to be doing and what their current goal is. If that is confusing, the team will become unproductive as a whole. If the team, first of all, doesn't know what its goal is, if you're in a lot of meetings about what you're supposed to be doing, something is wrong. You should have a charter that says, “Here's our goal.” Okay? And then you should have a plan that says, “here's our plan for the next three months, six months, year,” however long you think you need to plan for.
And it should not allocate a hundred percent of engineers' time. By the way, this is another mistake that people make, right? Because that's not a realistic thing to do. Interrupts come up, right? You can actually destroy the productivity of a team by trying to allocate a hundred percent of its time. You know, 60 to 80%, maybe, you can allocate, and you need to leave open that 20 to 40%, depending on how much interruption, stuff like that. People need to be able to have some freedom to be like, “Oh, I had a good idea. It's gonna take me a day. I'm gonna do that thing for a day.”
But then the individual engineers need to know, “I'm responsible for blah. Here's a ticket. The ticket is very clear. It describes a problem that has been given to me by product or by somebody, and it's in a form that I can actually implement it.” So there's that.
Who am I accountable to? Who's the team accountable to? Right? And the manager's responsible for a lot of the social and interpersonal dynamics of the group. Obviously, if there's disruptive things occurring, if there's toxicity occurring, if there's basically anything that violates the team's psychological safety, that also causes people to not be as productive as they possibly could be. Those are some very high-level things.
Rebecca: Yeah. And you mentioned, I wanna come back to the org level too. And maybe we'll get there right here, but you mentioned a hundred percent utilization is actively bad. Right? And there's lots of reasons, and you named some of them. But one thing that often comes up with this is just KTLO work, keeping the lights on work, and how that fits in.
Again, I have a hypothesis that companies would invest more in KTLO that they would go faster, but I don't wanna make an umbrella “tech debt is bad” argument because some tech debt should just be left alone. But yeah, how do you see KTLO work in this universe where ideally you're being rewarded for customer impact, right? And you're aligned around delivering customer impact. And KTLO doesn't really feel like that, but it also needs to get done. So, how do you think about that in the context of a team, but also let's zoom out and think about it in the context of an organization?
Max: Sure. I'm gonna lay this down. As a software engineer or as a software engineering team or a software engineering organization, the most important thing you need to optimize for is the amount of effort you're going to be spending on your software in the future. That is the most important thing that you need to optimize for as a business or as a team.
So there is bad KTLO and there is good KTLO, okay? Bad, sometimes necessary, by the way, sometimes necessary as a stopgap. But bad is, “I have two engineers allocated full-time every week to sit in Slack channels and answer questions of my engineers,” okay? And we don't do any retrospectives. We don't ever figure out what the root causes are of why people are asking all these questions. We just have two engineers full-time allocated to answering questions from people.
So good KTLO looks like, “Hey, we did a retrospective. We figured out that there's a bunch of stuff we haven't automated. There's a bunch of bugs we haven't fixed. A bunch of our error messages are unclear” – by the way, especially if you work on developer productivity tools. Super common. “Our error messages are unclear.” Really common problem.
And that is the lifeblood of your team. You must either allocate time to it or you need to estimate every project you have as the amount of time it would take to do it right – not the amount of time it would take to do it at all – the amount of time it would take to do it right. That's the only estimate you should ever provide. Almost no engineering manager in the world is asking you for the amount of time it would take to do it wrong.
And I find mostly that engineers feel anxious about delivery. The individual IC engineers feel anxious. They feel like maybe they're being judged, they feel like maybe it's gonna reflect on their performance, and so they give numbers that are shorter than they should be. They give numbers that are a week or two shorter than they would require to do a good job.
And so one of the ways that you can do this is if you have technical debt that involves refactoring, the number one best way to solve that is every time you're about to touch a feature in that area, you should refactor to make it easier to write that feature. This is a very simply stated fact that would solve probably 75% of all the technical that I've ever seen, and that's how we refactored the Bugzilla project. In fact, I would say that refactoring the Bugzilla project was the basis for our success, in fact. If you think about KTLO, we spent years. We were doing incremental delivery, right? We were building a feature, but every time, right before we build that feature, we'd refactor the area that we were about to touch to build the feature. We'd take things out of being long functions and move them to objects. Like the code base didn't have objects, right?
And so if you do that, you both keep delivering incremental value, but you also speed yourself up over time. And it should be the same with all of your other KTLO. You should always be thinking, “How can I eliminate this effort in the future?”
Rebecca: Your description of KTLO projects just brings back a lot of memories. But yeah, I think some of the most successful that I have been part of, just like you said, you had to refactor first before you built a feature. And at one of my jobs, we built a whole new kind of framework for building UI components in our flagship product. And then just migrated, you know, block by block by block the entire application over to that new framework without stopping active development on the flagship product. I haven't seen it work any other way.
Max: I don't think it does work any other way. But it's funny how many people try to make it work some other way.
Rebecca: Yes, yes. I have also been around some exceptionalism on this topic of “but we know what we're doing. We'll get it right.” But let’s zoom to the larger organization and what behaviors in a larger org in the engineering organization as a whole. What behaviors are, again, drivers and detractors from effective software delivery?
Max: So, at a high level across a company, I think I mentioned standardization as a key driver, so standardization with flexibility to make the technical decisions that need to be made. So, if I work at a company with thousands of software engineers and I say, everybody must write everything in Python all the time, no other languages are approved, you will run into situations where Python is not the right tool.
So if you are the standard setter, you need to understand all of the technical requirements that could possibly arise, which is quite a task. It is not just wave your hand and say, “thou all shalt use Python.” It is “What kind of software do we build? If you need to change our minds, the standards body, what's the process for changing that? What, what kind of standards do we set?”
For example, I very often encourage companies or teams not to standardize or require people to use a particular editor. Because if you watch software engineers, their editor is very ingrained into the way they work and the way they think, and they sometimes – especially if they're Emacs or Vim users – they have years, decades of muscle memory of plugins built up around their ecosystem, ways of working that you would just destroy if you put them into another IDE.
And most of the time, there's not a lot of value. But on the other hand, you might say, “hey, there's a few IDEs that we're gonna provide plugins for, or higher levels of support for.” But if you still want to use the other ones, we're not gonna block you from doing that. And we're gonna make sure that the rest of our ecosystem still functions if you choose to use another editor. So that's standardization.
Goals are also important, although usually at a company level, the goals are understood. If you don't understand what your goal is as a company, maybe you're a startup and early product market fit phase, right? You're just trying to figure that out. So that's fine. That's your goal. Your goal is to figure out a product that works for our market and find a market that fits that product. But I think at a higher level across the organization, there's a lot more cultural things that come into play. So that culture of independence versus that culture of ticket-taking. “Do you have a culture of fear or a culture of empowerment?” Is another big one. Are engineers doing their work because they're afraid that something bad is going to happen, or are they doing their work because they love being here and they believe in the goal?
Now, look, I'm not saying everybody at the company needs to be like a banner-waving zealot for your company's mission. That's not required, right? Most people and most companies wouldn't come to work if they weren't getting paid. That's fine. Okay? But are they frantically coding, just trying to defend themselves? Or are they genuinely trying to do their best because they care about doing their best, and they're in an environment that encourages them to do their best?
The “we all need to defend ourselves from each other” leads to an organization in which people don't want to do anything because doing things is the way to get in trouble. That's the Ne Plus Ultra of that pathology, right? Whereas, empowering people, giving them a certain amount of freedom, along with clarity and the right guardrails so that they don't go off the sides of the path. I'll talk about that in a second, actually, I'll give more clarity on guardrails. That leads to much more productive people.
And what do I mean by guardrails? So, ideally, it should be impossible to do the wrong thing. In your developer ecosystem, your tools need to make sure, as much as possible, that if something is dead wrong, that people cannot do it at all. That's the best. I mean, the best is that there's not even a tool that does it; it's that you've deleted all the things that could possibly cause that problem.
Internally at Google, a lot of cross-site scripting vulnerabilities were solved by creating a templating language in which they were basically impossible. We had a lot of scanning infrastructure for cross-site scripting vulnerabilities and stuff, and we still kept running it for all the tools that weren't using those templating languages, but in the tools that use those templating languages, cross-site scripting errors basically vanished. I mean, I'm talking like something like 99 plus percent of them went away, maybe higher. You just started getting no hits on the scanner all the time. No hits. No hits. No hits. No hits.
But that's a lot of work. Believe it or not, making these things vanish is a lot of work. And then even putting the guardrails in is a lot of work, you know? If people must run tests before they merge their PRs, that has to just happen, and it has to be mandatory. It has to not be a choice. You can allow overrides in an emergency. Although that's a whole other subject in itself, because you will discover that if you don't set that up the right way, then it will get abused because of incentives. People have incentives, they will do the thing that they're incentivized to do. They're not bad, not evil. They're not like trying to trick you. So your guardrails set the incentives. If your guardrails make it impossible to do the wrong thing, it changes the calculus of doing the wrong thing.
And people often try to solve this by human mandates. They try to say “everybody do the right thing,” but that does not work nearly as well as actually encoding it somehow in the technology. So, look, there's a million things that I could talk about, about how to make orgs productive, but there's a few of them.
Rebecca: Yeah, the standardization thing is so interesting. Everything that you said is true, and I went through the process of trying to introduce a new language to an organization that was almost entirely Java. And that language was JavaScript because it made a lot of sense to be doing things with JavaScript instead of with Java in certain cases. So I've been through that, and it was painful 'cause we didn't have that standard. Everybody wrote in Java. And so that was the standard. And there wasn't really a mechanism to even open that conversation. We spent months figuring out who to even talk to to get that conversation started. But yeah, I think everything you said really, really resonates with me.
And you talked a little bit about goals and understanding what you're trying to do as a business or what you're trying to do as a team. I wanna talk about goals around productivity and experience and effectiveness itself. A team might set these kinds of goals, or a group might set these goals, or the whole organization might set goals around, “we want to be more productive.” In a sentence or two, what is a good manifestation of that kind of goal?
Max: Okay. So first of all, the generic goal setting process looks like: say what you want to do, and then ask why a lot, and then define all the words in the goal. Until you have something where you're like, “Oh, I understand how I could know if I was accomplishing that or not.”
If your goal is “I want to improve productivity,” I would have no idea how you're accomplishing that or not. So you have to first say, “What do you mean?” You have to say, “I want to ship quality software faster to my users,” let's say. Okay, what does “quality” mean? What does “ship” mean? What does “faster” mean? How fast? And, eventually, you might get into multiple goals. So you go through that process. Okay, so that's your goal-setting process. But then, very often, you need something more specific for an individual tea,m and you also need to understand the drivers of the metrics.
So let's first talk about the specific goal settin,g and then we'll talk about the drivers with the metrics. So I mostly have worked in developer experience, I'm gonna use developer experience examples, right? If you are like, “Hey, I want to improve the productivity of my web developers.” And so what does that mean? Well, first of all, you have to find out what's blocking them, and then you need to measure that thing.
And that thing is gonna be something totally specific to your business. It's not gonna be some door metric. It's not gonna be anything that anybody else has ever said. It's gonna be something like, “how long does it take to run startup” or you're gonna be like, “oh, the problem is the edit refresh cycle for my web developers. They make a change, and then they try to hit refresh on the browser, and then it takes a minute.” And you're like, “Wow, that's horrible. We need to fix that. But then what's in that minute? What's taking that time?” So you might be, “okay, if we're gonna have a metric, it's gonna be that whole edit to refresh time.” But then you discover that 50 seconds of it are build and you're like, “oh, okay, so we need to optimize build. So that's actually the metric we're gonna focus on. And that's the metric we're gonna make our target for the next three months or something like that. And our goal is gonna be, we're gonna analyze it, we're gonna work on it.”
So at the team level, it gets very specific like that. And that's also sort of what I mean when I talk about drivers. So the drivers are, you know, your goal is, “hey, we want our web developers to ship quality software faster to our users” – with more specific wording than that. But your driver is build time. It turns out it's build time at the bottom, right?
If your goal is “we want to deploy more frequently, we wanna be able to experiment more rapidly,” then one of the things that you often find is batch size. You're shipping too many PRs in a single deployment. Or you find there's issues with your tests. The tests are not sufficiently high quality. There's not enough of them. And so you can't deploy with confidence. So you spend a long time doing manual testing. Those are the drivers. And those are the things you need to fix, and you don't really get anywhere by sort of just wagging your finger at a team and going, “now you be more productive, or you ship faster, or you ship more frequently, release more often.” Because they're not failing to release more often because they're lazy. I mean, with rare, rare, rare exception. Most of the time, something is in the way.
One of the things that I've seen is “We have two engineers and we own 20 binaries. How are we gonna ship 20 binaries every week with two engineers? Even if they're all on automation, what if three of them fail a week? Then we're not gonna be doing anything.” So then you're like, “Oh, well that's an organizational issue.” It's something different all the time. You have to dig into it and figure it out.
Rebecca: I have seen cases where there is a desire for the fix to exist at the individual or the team level. Just like you just said, “go faster.” There's a desire that that's where the problem is because if that's not where the problem is, then you have to spend money on the problem, right? If you can't solve the problem with the people you already have by yelling at them, then now you have to spend money to solve this problem. And it's not gonna be quick, cheap, or pretty, probably.
So I found that to be a challenge often, or at least that organizations go too long before they realize, “we need somebody to own developer experience here.” And yeah, this is just commentary. I don't know how to convince leaders to spend money on this stuff, but maybe you do.
Max: I think the only thing that I would say is, as far as the timing goes, remember all that stuff we talked about KTLO and fixing the things in advance? It is much harder and more expensive to fix it later than earlier. So if you wait a long time and let all that stuff build up, then you have engineers who are all caught up in doing all of that stuff all the time, and they don't have any time to do the developer experience work. So now, all of a sudden, you do need to hire more people if you want to be able to both deliver your current software and have any hope of a future for your business. But if you had just done it as you went, you might have been able to get by with fewer engineers, actually.
Rebecca: Yeah. I talk about the work of engineering effectiveness is just bending the curve because, by default, your utility from every additional engineer, if you just point them all, like “do work faster,” your utility per engineer will decline over time. Hiring one engineer won't get you one engineer worth of work because there's 20% worth of friction, and so you only get 80% of an engineer.
So yeah, I think I love to talk about that just because I've seen the power of investing in broad productivity improvements, and I just kinda wanna tell people, “you don't have to wait. It can be just like that person who's really good at working on the build. There's always somebody.”
Max: There's always somebody. Once you get large enough, there's always somebody who cares.
Rebecca: So then there are two or three, and then put them on a team.
I've had a lot of interesting conversations, like Nathen Harvey and I had an interesting conversation about how the existence of AI as this elephant in the room has been one of the best things for clarifying goals in a long time. Companies are suddenly very clear: “We're all in on AI.” Whether they should be or not, suddenly there's clarity and everybody's pointed in the same direction in a way that maybe they haven't been previously. And I thought that was a really interesting observation.
We don't need to go deep into AI, but, obviously, AI makes it faster to type code, right? Cause it types the code for you and it mostly works… sometimes. But we're thinking at Swarmia a lot about, with all of the code that is getting written with AI, what does this look like in a year or even in six months or two years? What is the state of software development if so much of the clickity clacking has moved to AI?
Max: Yeah, that's a great question. I mean, look, I can't predict the future. I can tell you what I've observed in the present and where I think that might go. You know, I think that there's two points here. One of them is that you should not abandon supporting traditional software development because you think AI will eliminate traditional software development. Every time I have seen a company abandon their support for their traditional software engineers, it has gone very poorly for that company.
And I understand that people don't wanna be left behind. And I think that's important. There is a transition happening in the world today, and it's worthwhile to understand how those tools can be the most effective for you. But they are tools. Right now, they are tools. I don't anticipate them ever being anything other than tools that human beings use to solve problems. There are definitely possibilities for them to expand. The ability of the systems has grown incredibly. I've been playing around on my own time with some of the tools, and I'm really impressed.
But I have noticed something, and that is that my skills as a senior engineer have never been more important than with AI, because you need to be able to do things like describe extremely clear requirements in English. Right? That's my job. It's been my job for a long time, right? Understand how to provide very clear instructions for what needs to be done in English. Know what scope and size of instructions you can safely give to an AI, just like you would give to a junior developer, like how much can you tell 'em to do all at once without coming back to you for a review?
You need to be able to look at 500 lines of code and instantly know, within a minute, what the next step is to refactor those 500 lines of code for clarity. You need to know when to add tests. You need to know what good tests look like with rapid speed. You can't be sitting there for 20 minutes looking at that test file. You will get very bored and you will not be able to keep doing your job. You'll just give up and be like, “I'm gonna ship bad code to the world,” right?
And so I can imagine them getting dramatically better at instruction following, I can imagine them being able to follow more and more complex instructions at a higher level, given just requirements. I can imagine a future in which the reasoning capabilities allows them to reason better from first principles. One of the problems today is that the internet is full of bad code. And the LLMs are trained on bad code. And so what they write for you is mostly bad code that you need to then tell them how to fix. And people might think, well, I don't need to understand that because the LLM will just fix it for me when I tell it to fix it.
The problem is this. Code is still the thing you're checking into your repository. And it is still a thing that's running in production, and it's still a thing that produces stack traces. So when it breaks, the code is the thing you will need to understand in order to know how to fix it, and when you wanna modify it and understand that it's been modified correctly, the code is the thing that you will need to look at.
Maybe there is some future far down the road that I cannot currently imagine where we can compile English to binary. But in order to do that, we'll have to never need to read code again, or at least very rarely. If you can think about the transition from assembly to high-level languages, you know that happened most thoroughly when developers no longer needed to read assembly. The vast majority of developers cannot read assembly code and do not need to know how to do that to do their jobs. So if there ever comes a day when the vast majority of developers don't need to read code, then that is when you will have pure English AI coding. I do not currently foresee that occurring.
Rebecca: What does this mean for the junior engineer? What does this mean for the person coming out of a great CS program? What does this evolution mean for the junior engineer?
Max: My opinion is that it is more important than ever to understand the fundamentals of software engineering because you need to be able to do them very quickly, and you need to be able to do them with confidence in response to the LLM. The great news is the LLM is the greatest learning tool mankind has ever developed. It can answer all of your questions, and usually it's answering them accurately. Especially when you just wanna learn about something? Take the time. If you are a junior engineer, take all the time that you need to learn about every piece of software that you touch, every technology you get involved with, because the thing that's gonna matter is your ability to understand technology broadly.
You wanna be able to give an LLM an instruction and just have it do it with any language or any technology that it thinks is the best or it is the most comfortable with. You need to get fast at learning new technologies, and the way you do that is by getting really good at a few basic ones. So you understand the basics of how software engineering works and the basics of how things like data stores and other parts of technology work. And then expanding out into a few other ones, learn some mor,e and then you'll eventually learn the similarities, learn the patterns.
And if you always ask yourself, not just “how do I do this,” but “what's the best way to do this?” Not just “does this work at all,” but “what's the best practice?” And really seek out every single time you learn something. What is the best practice in that space? And be comfortable. Chill out. Take some time to learn. Take days, take weeks sometimes, and make sure you really, really understand. And your ability to become more and more senior and better as a software engineer has never been better than it is today.
Rebecca: Max, you've just, in the last two minutes, you've said things that I could have kept talking to you for another hour about, including you used the word “best,” and I have so many questions. But, for now, thank you so much for chatting. I can't wait to keep following you and stay in touch. So yeah, thanks so much for your insight and for being willing to join us today.
Max: Thanks, Rebecca. I love this so much. It was great.
Rebecca: And that's the show. Engineering Unblocked is brought to you by Swarmia, where we are rethinking developer productivity. See you next time.