Kathryn:  I think that AI is a tool like any other tool that engineering has at their disposal. Hearing folks say, “Oh my God, it's so disruptive. It's gonna be the step function. It's the biggest technology change in my lifetime.” I'm like, “Well, I'm kind of getting up there and this is one of many that I have.” And the important thing is to keep a level head and explore and train up the workforce, not to be cats with laser pointers, but more thoughtful about how can this apply in our space? Because AI ultimately is a productivity tool.

Rebecca: I'm Rebecca Murphey, and this is Engineering Unblocked. Engineering Unblocked is brought to you by Swarmia, the engineering intelligence platform that's trusted by some of the best software companies in the world, including startups like Superhuman, scale-ups like Miro, and companies from the Fortune 500.

On today's show, I'm talking to Kathryn Koehler, the director of Developer Productivity at Netflix. Welcome, Kathryn.

Kathryn: Thank you. Thank you for having me.

Rebecca: Yes. So excited to talk to you finally after meeting you a couple years ago. So really looking forward to this. But, yeah, let's start by, introduce yourself. Tell me a little bit about you and your journey to developer productivity.

Kathryn: Oh, boy. I can buckle up, right? Grab some popcorn. It's a long journey. So I'll just try to hit the highlights.

Let's see, mechanical engineering undergrad. Wanted to design race cars when I grew up. Graduated, started working at an automotive tooling company. So with all of the hubris of a new college grad, raised my hand when they had a software project that needed doing, 'cause I took an intro to CS and I was like, “I can code a chess game in C, how hard could it be?” And quickly got my ass handed to me because it was pretty hard. It was pretty hard. But realized that all of the itches that I had around engineering and problem solving and creativity were scratched by computer science problems, with the benefit of not hopefully killing anybody and faster cycle times.

So I diligently started taking some classes after graduation. Learned C++, learned COM – I’m dating myself – and quickly outran the capabilities of the company where I was working in terms of my own engineering journey and tried to find a legit job somewhere else. Legit software engineering job.

I told this story before, but I was able to land a tried-and-true software engineering role, not based on experience, but based on enthusiasm. And I learned more during my interview than I had in the previous two and a half years. And they hired me! I know. Fool me once kind of thing.

And from there, my software career meandered. I wouldn't say it took off, but it meandered. I was in and out of different industries, more just taking opportunities as they came rather than having a prescribed “must be in C-suite by X amount of time” approach, and worked in entertainment, worked in virtual reality when that was a thing. I've worked in TV a few times. I worked in philanthropy in my last role and I thought that was gonna be my forever job.

But ended up at Netflix because it's a place that really lines up with my values and how I like to work and see the world and the people I like to surround myself with. And the product's also great. Yeah, there’s something for everybody.

Rebecca: So tell me, had you been working in developer productivity prior to Netflix, or were you just like, “That sounds fun?”

Kathryn: So, at my Evernote role, I had run all of the application platform teams. So web, mobile, desktop. And that wasn't so low-level and infrastructure-y, but the leverage was pretty incredible. And when I moved on to Chan Zuckerberg, I worked in their science initiative, which was really building tools, platforms, and infrastructure for scientists. Completely separate customer group, but the principles are roughly the same in terms of how do you standardize, how do you make things more effective? How are you an enablement platform? And I think that's what got my foot in the door at Netflix. They took a bet on me as well.

I'm very energetic about interviews. So it's a whiff, but never at the scale and complexity that I have at Netflix.

Rebecca: Yeah. Yeah. I talk a lot about how productivity at an organization is a function of size, age, and culture. And age is a really interesting aspect for Netflix, right? So I wanna talk about some of the problems that you get to solve with a how old–?

Kathryn: Yeah, like 20, over 20 years. Yeah.

Rebecca: Wow. Okay. Goodness.

Kathryn: So you're exactly right. Productivity is an interesting space, right? There are only a few of us who do it formally. There's some companies that do it really formally and they've got big teams, right? Netflix is sort of a hybrid where we've got really tricky problems in terms of scale and latency. But we're not super big, and so we run very lean, and we have to be judicious about what we build and what we buy. But we led the industry for so long that we had to build a lot, right?

And so I kind of think of an accordion, right? So early days you're just getting prolific and all of the things that you have to craft that are bespoke, right? And then industry catches up and then you have to pause and reflect and look about you and say, “Hey, could this be something that is open source? Is this something that we buy off the shelf? Is there an enterprise solution that does what we do so that we can get outta that business?“

Rebecca: But you had to be in that business in 2005.

Kathryn: Oh, yeah, yeah, absolutely. We still have a lot of things that outpace the industry. Netflix is the only streamer where you don't run into a spinner. So we have a very bespoke delivery of content. Great UI, but the backbone of how we deliver content. You could be in some bizarro place with basically one bar, and you're really covered.

I'm not allowed to talk about using other services, but sometimes we're using something else and my kids or my husband will be like, “this just doesn't work. Why is this not loading?” I'm like, “oh!” Because yeah, we've had our hands on basically every part of that pipeline.

Rebecca: Turns out that kind of engineering is really hard. Yeah. So many learnings from Netflix in my days as a front end engineer. Oh, so much prolific, great content. Amazing. Yeah.

Kathryn: But letting go of your in-house solutions is tricky, right? I mean, there's a rubric you can follow with build versus buy. And it’s not no-cost from a resource perspective, from a person perspective, when you buy and integrate and then you are kind of beholden to that other company's roadmap. But we do need to let go of more things and be really thoughtful about that.

Rebecca: It's not just a build versus buy, it’s you already built it.

Kathryn: Yeah. Are we now beyond the capabilities of what we have built? Are we limiting ourselves in terms of other external solutions because we've got this ecosystem that is just ours?

Rebecca: So we've both used the words productivity and platform as though those are very well-defined words.

Kathryn: Oh, boy. Yes. Yeah. Right.

Rebecca: Let's start with platform, because I think that is just a term that means whatever it wants to mean to whoever's saying it. But what's your take on platform and infrastructure and developer productivity and the intersection of those?

Kathryn: This is my webster’s dictionary defines… moment?

Rebecca: Yeah.

Kathryn: Platform for me is anything that you can build on top of, and will hide the particular ‘isms’ or efforts that are specific to your development environment. So, really taking away security concerns, efficiency concerns, observability, deployment, even language concerns. Not to the point of polyglot but basically allowing our customers – who are internal developers at Netflix – to build on top of and do their business logic. So if we have a growth engineer, we want them focused on growth. Those products.

Rebecca: Not on platform.

Kathryn: Exactly. We don't want them worried about how do I deploy this? How do I deploy it safely? How do I debug it? How do I test it? How do I even discover the tools that I need to do my job? So the platform is basically the very solid, stable, resilient, reliable, observable foundation on which our developers can build.

Rebecca: How did that platform emerge? And I know you probably weren't there for the beginning, but how did that platform emerge for Netflix? Yeah, because I see, often, platforms are an accident at first.

Kathryn: Yeah. It does predate me a little bit, but I do know there was coalescing around runtime and building out. One of our more mature areas is our Java platform, which is developed on top of Spring Boot. So, making a decision early, like we're gonna separate language concerns and develop out that way. Most of our services are written in Java, so this makes sense for us. It's how we evolved. We're beefing up our JavaScript platform space and beefing up our Python space as well.

But it largely comes out of what is everybody using, or most people? Taking a look around and saying, “Hey, can we consolidate some of this into a team or teams to support it?” And that decision still happens to this day, about maybe some customer teams have something brewing, and can we graduate that to a platform team so it will be higher leverage across all of our customer verticals?

So how we test, creating more bespoke IDE integrations and things like that. In any number of areas, we're thinking about how do we graduate them in a program or– we're gonna stop there. We're gonna back up. We're gonna try that again.

Yeah, it's like a hub and spoke model, right? So we've got this concept of a local central team, which is effectively a smaller platform team that builds on top of our central platform offerings so that they can tailor it for our specific customer groups. But for us, we onboard and maintain and develop the higher leverage pieces.

Rebecca: There's so much benefit in standardization from an efficiency perspective, right? But developers often get grumpy about it, about the lack of independent decision-making or whatever. How have you navigated that?

Kathryn: That's really tricky. We're still navigating it to this day. So what we call our platforms and infrastructure and tools, we call those the paved road or paved path. I've heard it used interchangeably. So our paved road are what we would encourage people to use when building out in their specific domains. And the way that we navigate that is by building things that are compelling and take away a lot of the pain, right?

Hey, if you're developing on the paved road, again, you don't have to worry about all those ‘ilities’ or ‘isms’ around performance, around security, safety, compliance, around safe– I already said deployment, but you know, safe deployment, incident remediation. You will get these things not for free, but it'll be a lot cheaper than if you had built it yourself. And it really depends on what people are building. Are these Tier 0 or Tier 1 level services? Obviously you wanna be very rigorous and you want to have the backing of your central engineering organization to help you throughout the life cycle of those services. And so that's a compelling argument, right? For leveraging what we do.

But we have a high freedom and responsibility culture where developers can choose to go off the paved path. But a big component of freedom and responsibility is responsibility. And so you have to be very cognizant of those trade-offs and why. Maybe there's a use case that you just can't do leveraging our pave path tooling. So hopefully there's some conversations happening with the central engineering team just so we're aware, “Hey, maybe we should be building in that direction in the future or supporting that longer term based on criticality.”

But other languages spring up that are better for different use cases that we don't officially have full paved path support for and wanna make sure that they are still safe and rigorous. So having some agreements with our customers that there are baselines of requirements they need to meet in order to potentially deploy some of these things. But if you're experimenting and having fun and playing around and you feel more comfortable leveraging Go or Erlang or COBOL, Pascal, Basic? Sure.

Rebecca: Whatever strikes your fancy. How about Fortran? Can we write in Fortran 77?

Kathryn: Empires were built on Fortran, so why not? That falls under more of our community support. We're asking you to do the needful to make sure that you're not breaking things. Yeah.

Rebecca: Great, I'll look out for that pull request any day now.

Kathryn: It's nice being in a non-top-down-type culture for most cases, just because the decisions are pushed to the folks who are at the ground floor and they have great opinions and they're customers of our own stuff. But it's also sometimes challenging, right? Like, “Hey, everybody needs to get on this latest version.”

Rebecca: How do you handle migrations like that? Whether it's “everybody needs to get on this version,” or “everybody should use this new tool,” or whatever. How do you approach this?

Kathryn: We have done a lot of work to improve our migration tooling, where we make it not hands-free, seamless, but have reduced a lot of our developer toil. Imagine an application that basically has a list of all the migrations that need to happen, and the people who are driving those migrations have done the best of their ability to make them fully automatable. In some cases, we can't. It's too challenging or sometimes the ROI isn't there. But within this tooling, those migrations can happen. And teams’ leadership can take a look and say, “Hey, how is the overall health of my team in terms of drift or complying with latest and greatest versions of platforms?”

And with that visibility, with this accounting, with a prioritization of what migrations need to happen on teams and the automation around migrations being in place, we've really lowered the friction, because we always be migrating, right? Like, just as soon as your hands touch the keyboard and your hands are off the keyboard, things get out of date.

So being strategic about when those migrations happen and what really needs to happen. Will the world end if we don't track immediately to the latest release of something? 'cause we also have to do our due diligence if there are third party releases that happen. So I wouldn't say it's perfect, but having a more consolidated, unified, standardized, centralized view, so people know what to expect, see it coming, and then help them along the way.

And then we do have a team that helps us with those white-glove migrations. So we don't just leave it to the customer and be like ‘best of luck.’ We will send in some folks who are highly skilled and are able to get people across the finish line for those really critical migrations.

Rebecca: Are you in a monorepo universe or a multimono repo or just multirepo?

Kathryn: Multi and multimono repo. So that makes that a lot harder. A lot harder, right? I mean, the future is managed. And not necessarily in a monorepo environment, like, in my opinion, for everybody, not Netflix, but having things that you don't really have to be a part of those updates and we just move the infrastructure underneath you. And it works.

Rebecca: Yeah. I worked with a monorepo at Stripe, which I had not done before. I had been around people saying, we need a monorepo, and I was like, okay.

Kathryn: But then there are a whole host of problems that come with that.

Rebecca: Oh my. Oh yes, there are.

Kathryn: Like you're just making trade-offs.

Rebecca: Yes. A large portion of the developer productivity team at Stripe was focused on making sure that repo stayed fast. And there were times when it was not at all.

Kathryn: We like the flexibility, obviously, the nimbleness. But we within platform do not pay the price for it, but we have to be very thoughtful in how we develop against that.

Rebecca: Yeah. And your solution space is different than in monorepo context as well. So, how do you decide what to work on?

Kathryn: We have a really big dartboard with a bunch of things listed on– No. So we have a couple of mechanisms. One is obviously business need, right? So, as Netflix moves into different areas like games or ads or live, these all have to be things that are ultimately supported by a central engineering organization. So, we consider those more of the top-down requests – top priorities for the company. How do we make sure that we can support them?

Then we have the bottom-up team decisions – our domain experts, if you will, who are not only developers in the space, but customers of the space and KTLO or keeping the lights on, running the business work that needs to happen. So there's a budget for that. But also they wanna get ahead of or keep a pace with evolving technologies, integrations that we mentioned earlier with open source or third-party opportunities. So I call that the bottoms-up roadmap. And then we got the tops-downs roadmap.

And we also have, I think, kind of rare, we have dedicated product managers. Which is great. Some productivity orgs, it's YOLO and there's no PM. Some leverage TPM. We have TPM, we have PM and we have design. I know, I know. But we still run really lean, don't get me wrong. So they're also scanning the horizon in terms of our customer needs, in terms of industry trends. We leverage a database for customer feedback that we review frequently to make sure that we're closing gaps and reducing toil for folks.

And then, you know, we'll generate some big cohesive end-to-end product ideas, like our campaign tooling that I mentioned for migrations and things like that. So it’s kind of, I call it a hug sandwich. You got a bunch of things coming in from multiple directions and then really relying on the teams to figure out how best to sequence and prioritize that work with their tech debt, bug fixes, keeping the lights on, support running the business work.

Rebecca: So I have a hypothesis and I'm curious to hear whether you agree. And that's without intervention, without the existence of a platform team or a productivity team, your “utility per engineer” that you're getting – for some definition of utility – It's an econ word, right? Your utility per engineer will decline as team size and code size increases. Again, without intervention.

Kathryn: I agree. I agree. And I think of the productivity or even any platform org, 'cause we didn't really define “what does productivity mean,” right? Our central infrastructure org is security, it is data platform, it is cloud infrastructure, and it is productivity. And productivity in that space means we make your day today easier. The two words that I like to latch onto are ‘we simplify and we accelerate development at Netflix.’ And there's a much longer thing that I could say, but I can't remember it.

So the productivity org scales sub-linearly with the growth of our customer organizations. And so this is work that is standardized, repeatable, ensures higher quality, ensures rollbacks. I mean, I've used the word ‘safe deployments’ a bunch of times. It's really prevention and remediation in terms of events that happen, and for a company that's a streaming and a live streaming company, is really important. Imagine if you had federated all of those concerns to our individual developers. You would have a lower quality, higher risk. Just bad experience.

Rebecca: Well, and you would be limiting your opportunity space too, because you'd have so many people spending–

Kathryn: Exactly. Right? So we want them focused on their domain and their specialty. And we wanna take all of that other stuff away from them in a way because it scales. We can keep our org pretty small and do these things.

And one of the anecdotes I like to describe is when you have a very solid paved road and you have a new hire, they can check in code day two. Day one, they're setting up their laptop, right? Day two.

If you had to do all that from scratch, you may have had some stuff that had been locally developed on the team to help out, but now you have, what, a thousand different instances of all that stuff and it's inconsistent? You wouldn't be productive until like month three or four.

Rebecca: Do you try to measure that? Do you try to measure the effectiveness of new hires?

Kathryn: We are looking at time to 10th check-in type situation because ‘hello world’ doesn't count. Because our teams are so specialized, it's hard to standardize across the board. But we do look at that time to 10th check-in. And we also get more qualitative feedback from teams from their onboarding experience and things like that. We have a good developer education team that handles onboarding and getting people familiarized with our platforms so that they can be productive on day two, which is really nice.

So I would say it's a mix of qualitative and quantitative measurement. And we have really exacting standards from our new hires, particularly our L3s and L4s, because Netflix has been predominantly a very senior and staff-level developer company.

Rebecca: Developer productivity is part of this larger, kind of infra-group. This was something that I've experienced in the past. How are your peers in the infra group thinking about “the developers at Netflix are my users?” Are they approaching with the same mindset? Are they thinking about that too?

Kathryn: We hire and nurture customer-obsessed individuals, which is great. Sometimes you gotta hold 'em back and be like, “Hey, you just spent three days working on this support call,” and it ended up being that it was just that person's pet peeve. Because we're engineers and we wanna solve problems, and we hire folks who are really excited about the productivity space. So that one person on the team who's fixing the build and making everything reproducible, and fixing out. You know, that is usually the person that we hire for this team. So it's a collection of people that really think deeply about the customer and the impact on the customer.

And what I mentioned earlier about migrations. We spend so much time and energy taking the bumps and speeding up and improving the experience for somebody on the receiving end of a migration request or need. And that's that customer obsession and how it shines through. We will do the undifferentiated heavy lifting is the phrase that we like to use for our customers. And the introduction of product management was actually maybe four and a half years ago? Before that, we interface directly with customers and there's still a lot of that vibe on the teams.

And we are customers of our own things. You know, not in every case. You know, some things we build, we're not necessarily customers of. But very highly opinionated about what would be easier, what would be cool, what should we be going after, how do we reduce toil, that type of thing. So we select for it. We encourage it. We incentivize it. Occasionally, we won't get the balance right about what is the work on me versus the work on our customer. But having that customer empathy is critical to being on the team.

Rebecca: Well, even just realizing that you have customers and that you're making a product. That has not always been widely understood.

Kathryn: So, yeah, that brings up a really great point. One of the reasons I love this space is that our customers are in the building, right? Even in a hybrid world, they are a Slack message away. “Hey, I just shipped this thing, or I'm working on this thing. What do you think?” And Netflix has a very strong culture of feedback. It will be shared with you all the time. And so our customers, thankfully, are not shy about what they need, what they want, and what they don't like. And that feedback loop with internal customers is very tight. Which means we are lining up our deliverables with what they need and being able to really figure out if it makes an impact or not.

It's hard to do that, statistically speaking and from a measurement perspective, just given our customer base is three or 4,000 people big. But there's not a huge gap because we get a lot of anecdotal and or sentiment information as well from them.

Rebecca: As the director of developer productivity at Netflix, how much are you spending time thinking about the productivity at an individual team level? Looking at metrics like DORA metrics, like cycle time or something like that? At the individual team level.

Kathryn: Oh, not at all. If our tooling is standing in the way of a team's productivity, if we are hindering or not helping, then that's something that we look at from our perspective, we could be doing better. But we are not a company that measures lines of code or team. We're looking at impact. We're looking at: are our teams doing what they need to be doing to deliver customer impact, irrespective of who that customer is?

And so for us, we hire good people who have good judgment, who are all very driven. And we measure the things that we really wanna more incentivize and manage. Things like how much manual intervention was there in using this tool? What are cycle times, not from the perspective of “how is the team doing,” but how is our tooling serving the team? And then for any sort of performance-related, like big P performance-related measurements, we rely on really strong leadership teams to evaluate by observing.

Rebecca: You mentioned that Netflix has tended to be pretty senior, I've worked with a similar, similar-ish sized company, but they hired a lot of new grads, a lot of new grads. And so that showed in the team discipline. A lot of what you had to educate people on was how to work together on a team. They didn't learn that in college, right? And so, yeah, I think that you are at an advantage just by tending towards the senior end. I think that eliminates a lot of the team-level chaos that can happen.

Kathryn: What's been interesting for us by hiring new grads – and we have an exceptional cohort. An exceptional cohort. We wanna hire people that we know can thrive in a very senior environment, 'cause it's not easy being that pioneering group. And it really has shown a spotlight on how effective and easy to use is our tooling, how well documented is it? How discoverable is it? How much institutional knowledge do you have to have? Because if that barrier to entry is high, it doesn't matter if it's a new grad or an L6 who's joining the company, they're going to be wasting time trying to get to productive unless we can meet them where they are.

So it has actually introduced a fair amount more of technical rigor on our side to be a truly effective productivity organization. And I would say from a team health, organizational learning how to work with people, we have one or two on each team, right? It's not as if we opened floodgates. But those folks, I mean, they've been phenomenal. They've been phenomenal. Like really tight buddy-buddy mentorship situation. And I haven't found any of it to be disruptive. Quite the contrary. Yeah, it's been really nice.

Rebecca: We've talked about what you do. What are you accountable for? What does it look like if you're doing a good job?

Kathryn: Yeah. I am developing the next line of leaders. I am creating healthy, productive environments where people can do their best work. I am empowering people to make decisions, and they don't have to be the right decisions, but they need to make them, and then they need to remake them if they were the wrong decisions.

Most of how I work is evaluated by the people above me in terms of outcomes and impact to our customers. Are customers happy? Are they, not necessarily singing our praises, but not gnashing teeth and wailing, you know? And can I have teams that are delivering impact in a sustainable way? So not bursts of little P productivity, because I'm cracking “everybody get to work” or chaining you to your seat.

So we have a pretty routine health surveys that go out just to see: “I feel empowered to do my best work. I'm adding value. I understand the business,” and all of that is a direct reflection on me, even if they're reporting to managers under me. Because I set the tone for the org and it's really important that the buck stops with me on a lot of these things.

So yeah, and can my peers work with me? Am I collaborative? Am I supportive? Am I making hard choices? Am I initiating tough conversations? Am I developing high-performing teams that feel good about what they do? That is my ultimate accountability. Yeah. Not the technical choices here, there, and everywhere. Those are largely up to my managers and my talented individual contributors.

Rebecca: I've got one last question for you before we wrap up. It wouldn't be a podcast in 2025 if I didn't ask you about AI. So is it your nemesis? What are you seeing as far as how it's changing software engineering?

Kathryn: I worry about AI FOMO driving hasty decisions.

Rebecca: Mm-hmm.

Kathryn: And I think that AI is a tool like any other tool that engineering has at their disposal. Hearing folks say, “Oh my God, it's so disruptive. It's gonna be the step function. It's the biggest technology change in my lifetime.” I'm like, “Well, I'm kind of getting up there and this is one of many that I have.” And the important thing is to keep a level head and explore and train up the workforce, not to be cats with laser pointers, but more thoughtful about how can this apply in our space? Because AI ultimately is a productivity tool, right? And if we can't think about how to leverage AI in productivity, then what are we doing?

Rebecca: Yeah. I like to compare it to because we get all sorts of people who want to measure the impact of the AI tools that they're using. And I get it, especially 'cause I've been reading lately that many companies are realizing people aren't using all the tools that they're paying for. But I also look at it in terms of nobody questioned whether we should start using IDEs, right? Which were also transformational to the co-development process.

Kathryn: I think where it gets scary is there's a content generation piece. And by content, I mean code tests, documentation insights and things like that. I'm not talking about Netflix content. So, where it gets interesting is, do you trust it? And it can be somewhat– like say you're doing code gen and folks are not reviewing the output.

I think the sweet spot is… You have a senior engineer. They're coding in a traditional way. You have somebody who's just blindly accepting whatever is being, you know… And that's relying very heavily on, you've trained well on models that are specific to your ecosystem, et cetera.

That sweet spot is somebody that's reviewing the output. And can you get that productivity boost with a ‘trust but verify’ approach? And that's where I want people to say, “Hey, yeah, all of this is pretty, it's great, but I've seen things hallucinate pretty crazily.” Are you going to be setting yourself up for failure if you're very reliant on this to generate tests, because then how are you validating inputs in terms of code changes?

I can tell when people are leveraging Gen AI to write their docs. I can tell, right? Can we ground ourselves in more specifics and things like that? So I think it's interesting and it's one to watch and I wanna leave space for my teams to innovate and explore, but there's no mandate.

Rebecca: That's good because I've heard of that, and it's very puzzling to me. All for standardization, but maybe not that kind.

Kathryn: We hire so many smart people, right? And we wanna keep leveraging smart people.

Rebecca: Yeah, I do think that there are real challenges ahead of us with, like you said, code gen. Writing code isn't the slowest part of software development. But we just made it faster.

Kathryn: Yeah, it is gonna be interesting. And, I mean, we're all learning, right?

Rebecca: Yes. Well, it's been really great to chat with you, finally. Really great. Thank you so much for taking the time, and I look forward to running into you somewhere again sometime soon.

Kathryn: Yes, Rebecca, it was a pleasure. Thank you.

Rebecca: And that's the show. Engineering Unblocked is brought to you by Swarmia, the engineering intelligence platform trusted by some of the best software companies in the world. See you next time.