Adapting to AI: Leadership

I have been writing a series of essays on adapting software engineering to the widespread use of AI. My last essay focused on adapting organizations, designing them to be their most effective with AI. Because leadership is so critically important to the success of an organization, it might seem like an omission that I didn’t include a section on leadership, but the topic of leadership deserved an essay of its own.^[1]

Transformational Leadership

Transformational times require transformational leadership. We admire leaders who can inspire, lead by example, define excellence, connect the work to its purpose and its positive impact on the world, and deeply understand humans and human relationships, in part because these leaders are so rare.^[2] With AI rapidly changing the way we all work, exceptional leaders will be more important than ever, helping organizations navigate these generational changes to become highly effective, while not losing their humanity.

You have to be run by ideas, not hierarchy. The best ideas have to win. Otherwise good people don’t stay.
—Steve Jobs

It doesn’t get talked about as much, but bureaucratic, routine management—management that never inspired anyone—can be just as easily replaced by AI as the work performed by programmers. An AI is probably better. While an AI can liberally and tirelessly apply bureaucracy and nuts-and-bolts management, it also holds a broader perspective, is less biased, and comes with less baggage. Unless you make it part of the context, the AI is not looking to get promoted, it is not competing with peers, it doesn’t favour its old team, it doesn’t get emotional, it is not worried about getting fired, and it is not self-conscious about its ability to lead—it is just exploring the vector space. AIs are also great tools for inviting different perspectives, weighing pros and cons, and being sensitive to a broad set of needs.^[3]

Don Norman, in his book The Design of Everyday Things, notes most engineers struggle with design thinking because of their limited understanding of people and an over-emphasis on logical explanations being all that matters:

Why this deficiency? Because much of the design is done by engineers who are experts in technology but limited in their understanding of people. “We are people ourselves,” they think, “so we understand people.” But in fact, we humans are amazingly complex. Those who have not studied human behavior often think it is pretty simple. Engineers, moreover, make the mistake of thinking that logical explanation is sufficient.
—Don Norman, The Design of Everyday Things

This could also be said of engineering managers, most of whom were promoted based on their technical achievements not their exceptional understanding of people or organizations. It is natural for managers to think, just as naively, “I have worked for a number of managers, both good and bad, I have delivered successful projects, navigated organizational politics, I’ve been through a few reorganizations, hired and fired, so I understand people.” But a deep understanding of people only comes after years of intentional study and practice, most likely guided by a skilled mentor, therapist, or psychologist, and it starts with the long, never-ending journey of building your own self-awareness.^[4] This awareness includes understanding how you perceive the world, how your perceptions are shaped by your own experience, and how your perceptions are interpreted by others who perceive the world differently from you. It includes appreciating what people say and how they act, but also understanding the motivations behind those words and actions. Finally, it includes being aware of how you impact others through your role, the values you demonstrate, your body language, and the words and actions you choose, or fail to choose.

Leaders are usually rewarded for being decisive, having a plan, and acting with certitude.^[5] One of the most difficult things a leader must do is sit with uncertainty. This capacity must be balanced with clear direction, a bias for action, a sense of urgency, and strong execution—otherwise it slides into indecision. Yet the ability to sit with uncertainty is what allows leaders to stay engaged with the most difficult, unsolved problems while they are still being solved, and to connect meaningfully with the people working on them across the organization. It also enables leaders to remain open to many perspectives, using those perspectives to fuel their own creativity and to inspire others. Exceptional leaders are always taking on new challenges. By sitting with uncertainty and vulnerably demonstrating this to others, they lead by example, critically evaluate their own work, and continually improve.

Sitting with uncertainty is uncomfortable, and few leaders feel at ease right now. Transformational leaders won’t have all the answers, but by embracing this uncertainty with openness, self-awareness, and trust, they can inspire their organizations through a generational change.^[6]

Walking the Walk

Leaders can’t just encourage their organizations to use AI, they need to demonstrate how they are using it themselves, showing how it is changing their own work. Leaders need to figure out how to use AI to handle the routine management tasks—the management 101 stuff—freeing up more time to be directly engaged with the people and the work.

More layers of management can result in slower information flow. Most organizations likely need fewer managers and more exceptional leaders.^[7] Organizations probably become flatter, finding ways to organize around their most critical leaders, putting more attention on the actual work itself, and less on managing the people or the work. Organizations will demand superior executive management and those executives will be expected to work more directly with engineers and customers.^[8]

AI will create new challenges for leadership, but right now, AI is going to magnify the existing challenges in any organization. If a leader is already struggling to address those challenges, AI could make them overwhelming.

On my way to work on the Amsterdam Metro last week, I overheard two people talking. The one person laughed: “People are using AI to build their kingdoms all over the company. It is so dysfunctional.”

Customers experiencing the constraints of an organization reflected in the product has long been a problem—it is the origin of Conway’s Law. But with AI, people and teams are able to move more quickly and more independently. It is easy to expand scope and create many local minima. A critical focus of leadership needs to be actively shaping the human organization to avoid multiplying existing dysfunctions with AI, bringing people together to achieve higher-order objectives.

I have worked on Amazon S3 for ~12 years and if there is one thing that I have learned, it is that when you run complex systems at scale, you must think deeply about how teams work. It’s not enough to be get into the details about what you build. You have to spend lots of time engineering, iterating, and improving how you and your team operate.
—Mai-Lan Tomsen Bukovec, Principal Engineer Roles Framework^[9]

I used to work for an executive who paid special attention to software projects because, as he said, it is easy to write software all day long. That’s now more true than ever. He focused on lost efforts in software—projects that didn’t yield enough value—consolidating projects and teams to focus on a smaller number of things but doing them well. He believed it was better to consciously balance the attention of people in the organization rather than let this happen unconsciously. It forced explicit trade-offs. When you see a team writing software to solve a problem, especially with how easy it is to write code with AI, ask why that problem exists in the first place.

I also worked with a very well-respected software VP who I still turn to for advice. He believed little islands of software development supervised by people who don’t understand software create real organizational problems, something AI has made even more prevalent. Humbly, he knew he didn’t have all the answers to the most difficult organizational challenges—no one does—but he recommended at least avoiding what you don’t want. He insisted on not “shipping the org chart” and avoiding pathologies. He viewed software teams who don’t feel the pain of operations as an example of both. He believed the second generation of any product or platform should never be owned by a separate team, otherwise people would not be operationally responsible for the current generation, or think pragmatically and systematically—if you own the new thing, you must own the old thing.^[10] He viewed systems thinking as a critical skill and that only comes from owning the whole system. Most important to him, accountability and authority—who is responsible and who gets to make decisions—had to be co-located in the same person or team.

These leadership principles have always been important, and often lacking in many organizations. With AI, they seem more critical than ever.

Stratification

AI is going to empower the most valuable leaders in any organization and make them even more valuable. Your best systems engineers who have the agency to design, communicate, build, test, secure, deploy, operate, debug, and scale systems will be even more productive and empowered when combined with AI. There will be less demand for a systems engineer who can program, but doesn’t communicate well, or own the full lifecycle of the software from design, through testing, deployment, and operations. The same will happen in management. Professional managers will become an obvious bottleneck, while leaders who can inspire, connect with people, design human systems, engage directly with the work, both the problems and the solutions, and connect the work to a greater purpose, will become even more valuable.

One of the first times I concretely appreciated the transformative nature of AI and how much it would multiply the impact of leaders was watching Marc Brooker’s talk Try again: The tools and techniques behind resilient systems at AWS re:Invent in 2024. Marc uses AI to build simulations that help him quantitatively explore the dynamic behaviour of systems. It was amazing that the AI could produce such valuable simulations so quickly, but I also realized most people could not have done what Marc did. It was because he understood how the systems worked, how they failed, the business impact of failures, the economics of the infrastructure, and even the mathematics and statistics behind the system dynamics, that allowed him to build such valuable simulations so quickly, and then evaluate the quality of them. He knew the questions he wanted to explore, and AI helped him explore more questions more quickly. I realized amazing leaders like Marc would be even more productive and even more valuable with AI at their fingertips.

What does this mean for leaders earlier on in their careers, without this breadth of experience? I highly recommend reading Marc’s essay What about juniors?, and not just from the perspective of junior engineers, but to think about how we all adapt as leaders.^[11] I think Marc would agree with what I wrote in my essay Adapting to AI: Adapting Organizations that the social aspects of engineering will be increasingly important, requiring a broad set of skills:

What does a new engineer do on my team? They build. And, much earlier in their career than you might have been comfortable with before, they own. Own a project. Talk to a customer. Own a deadline. Get exposed to all the messy customer, economics, business, people, and other constraints that they need to understand in order to optimize for. Set high expectations for using deep knowledge of technology to solve problems in new ways. That’s going to be hard for some, especially those who came into the field expecting a purely technical (or barely technical) career.
—Marc Brooker, What about juniors?

AI is a tremendous tool for learning and democratizing knowledge that might otherwise take years to accumulate. The leaders in our organizations aren’t necessarily going to be the ones with the most years of experience or the most prominent roles in the current organizational chart. They are going to be the people who can adapt, work across disciplines, communicate, write, build relationships, collaborate, mentor, think in systems, and execute.

It’s a job that spans customers, economics, people, and technology. Failure to engage with one or more of the parts of that complex job, or handle the complexity that comes with optimizing for all of them at once, is the most common reason that I see for engineers’ careers stalling out.
—Marc Brooker, What about juniors?

Conclusions

Most organizations need to rapidly evolve with AI in order to survive—patience doesn’t work with AI. At the same time, many people have an underlying anxiety about how AI is changing their jobs, questioning their purpose and ability to adapt. Organizational leaders are certainly not immune to these same feelings. To navigate these extraordinary times, we need extraordinary leaders.

I define a leader as anyone who takes responsibility for finding the potential in people and processes, and who has the courage to develop that potential.
—Brené Brown^[12]

Transformational leadership isn’t going to emerge from carefully managing direct reports, skip-level meetings, business plans, quarterly roadmaps, and PowerPoint summaries. We need leaders who are good writers. Leaders who are curious and ask good questions. Leaders who have the courage to act—letting others see that they aren’t perfect and don’t have all the answers—while instilling the confidence that we will figure things out together. Leaders who produce and consume original work, intimately understanding the details, from first principles, actively involved in making better decisions, not just living in summary.^[13] We need leaders who can zoom out and place the work in a broader context, connecting the work to its purpose, and protecting the conceptual integrity of technical and human systems. These leaders need to be exceptional designers and systems thinkers, and they need to deeply understand people, starting with themselves.