Engineering Management: Three Books and Three Videos

I declined a number of opportunities to lead a team because I knew the role demanded an enormous change from that of an engineer. I still wanted to be solving technical challenges, in a state of flow, for hours a day. Equally important, I wanted to build a team with a specific character: a team dedicated to engineering excellence, collaboration, and high-quality work, but also a team that is connected and inclusive, where people feel at home to do their best work, develop their self-awareness, and be vulnerable. This demanded the right environment.^[1]

Like many people, I have suffered the consequences of a team lead assuming the role without embracing the change in role, or without developing an intentional leadership philosophy. I am often asked for the books that have shaped my management philosophies. Most of my ideas came from studying why people and organizations behave the way they behave. I was fortunate to do much of this exploration under the guidance of a close friend, mentor, and colleague who, in addition to being a software engineer, is a practising therapist with a PhD in clinical psychology.^[2] We would regularly discuss the books I was reading, many at his suggestion, and explore the interpersonal and organizational dynamics that were unfolding at work, in our personal lives, and the world at large.

To appreciate my management philosophy, I know people are not going to digest dozens of books, some of which are academic or deeply personal. However, I can distill my management philosophy to three accessible books, all of which have an accompanying video lecture. If you are not a reader, or if you just want to quickly digest the high-level ideas, you can do so in under a couple of hours by watching the videos. This essay is a collection of the most influential ideas from these books and videos on my thinking. The team I lead is an experiment largely based on these ideas.

Peopleware

Peopleware: Productive Projects and Teams is a software industry classic. It is a book for engineers, by engineers. Everyone should read it. I read it for the first time 15 years ago after encountering many references to it on Joel Spolsky’s blog. Despite the ideas being more relevant than ever, almost 35 years after it was originally published, I don’t encounter many people who have read it.

Peopleware is best summed up by quote from Joel on the back cover:

Peopleware is the one book that everyone who runs a software team needs to read and reread once a year.

Every time I read this timeless book, it reinforces something new. It is the single most influential book on my thinking regarding management, teams, and organizations.^[3]

The focus of Peopleware is on quality: the quality of the work, the process, the product, and the environment. Some organizations do much better than others—and not just a little better, but often an order of magnitude better—and this tends to sustain over time. The best organizations attract, nurture, and retain talent. The worst organizations fail to attract or keep good people, or make it impossible for even good people to work effectively. The difference is in how the organization treats people.^[4]

A good engineer has a higher internal definition of quality than anything you can impose. It is always a mistake to pressure someone into delivering work that fails to meet their internal standards of quality. Management assumes projects are constrained by cost, scope, and time—the classic “good, fast, or cheap: choose two”. But you cannot negotiate quality with a team. If you don’t choose quality, the best engineers leave. You are left with a poor product and a poor culture. Many people will identify that something is wrong, but you won’t have the people with the ability to correct it.^[5]

Many managers are compelled to micromanage, weary of engineers endlessly tinkering in the name of quality. However, most people are eager to get the job done, provided they don’t have to sacrifice quality, because it is the only means to experience the satisfaction of completing their work. Interestingly, the same people chided for tinkering in the name of quality are held responsible when quality is low.^[6] In the day and age of always-on, you-build-it-you-run-it software services, where teams are responsible around the clock, often without additional compensation, forcing a team to deliver low quality is unethical. In reality, quality brings cost reductions. Quality far beyond that required by the end user is a means to higher productivity and innovation.

Peopleware presents evidence that projects with no time estimate whatsoever result in the highest productivity. I understand that organizations need to staff and plan—or at least trick themselves into thinking they can plan—but this is the reason I detest burdensome and prescriptive processes, like Scrum or OKRs, that have invaded the industry. Peopleware also warns that measures of productivity should only be used for self-assessment and never available to management to compare or manipulate.

The environment the organization has provided to help you work has a big impact on quality and productivity. Almost without exception, the modern workspace is noisy, interruptive, un-private, and sterile. No one can get much real work done: “As long as workers are crowded into noisy, sterile, disruptive space, it’s not worth improving anything but the workplace.”^[7] To work in peace, behaviours like staying late, arriving early, hiding in the cafeteria, or working from home, are a damning indictment of the office environment. Working extra time is not so much a means to increase the quantity of work time as to improve the quality. I have never worked in an office I liked, with the exception of a brief stint working in a small, converted closet alongside the colleague that I described in the introduction. Noise and interruptions are directly proportional to density, and all of the offices I’ve worked in got worse as we crammed in more people, or focused on open-plan seating to encourage the appearance of collaboration.^[8]

While I’m responsible for a large team, now upward of 20 people, I don’t like referring to myself as a manager. I haven’t done any serious programming in a couple of years, and my role is certainly more administrative, but too often a manager is associated with directing or controlling. This is not what I do. Peopleware says a manager’s function is not to make people work, but to make it possible for people to work. A manager should provide organizational objectives, priorities, and context, and then get out of the way. I don’t even like referring to myself as the leader of the team, because it is a team full of leaders. I reject the traditional idea that managers “lead” and that it happens down the hierarchy. Peopleware notes, “the best leadership—the kind that people can mention only with evident emotion and deep respect—is most often exercised by people without positional power.” A leader is a catalyst. My goal is to create an environment where a team of leaders can do excellent engineering.

The organization must be accepting of this bottom-up leadership. Most organizations talk a good game, but bottom-up innovation can be uncomfortable, or distinctly unwelcome, because it requires accommodating change. Managers and executives can be “uncomfortable with any kind of behavior that is different from average. They need to impose safely homogenized mores on those beneath them to demonstrate that they are in charge.” To be successful, management must give up control, “since nobody knows how the next innovation may alter the organization, nobody knows enough to give permission to the key instigators to do what needs to be done.”

In any organization, there are people who are sensitive to organizational dysfunction. They can often be seen as difficult. Engineers are paid to identify and solve problems and they will do so within the organization, too. Provide attention to these people, listen to them, and help them constructively channel their discomfort. They are often some of the most driven and creative people in the organization, and the challenges they are drawing attention to are important.

Companies pay great attention to diversity and inclusion in hiring and career development. Imagine where the industry would be if we heeded Peopleware’s advice on bias over 30 years ago:

That almost unsensed pressure is pushing toward the company average, encouraging you to hire people that look like, sound like, and think like everybody else. When the culture is unhealthy, it’s difficult or impossible to hire the one person who matters most, the one who doesn’t think like all the rest.

Uniformity is even seen as “professional”, since you will be viewed as unsurprising: “You will be considered professional to the extent you look, act, and think like everyone else.” Or consider Peopleware's observations on aptitude tests, like the programming tests uniformly applied in the hiring process:

The aptitude tests we’ve seen are mostly left-brain oriented. That’s because the typical things new hires do are performed largely in the left brain. The things they do later on in their career, however, are to a much greater degree right-brain activities. Management, in particular, requires holistic thinking, heuristic judgement, and intuition based upon experience. So the aptitude test may give you people who perform better in the short term, but are less likely to succeed later on. Maybe you should use an aptitude test but hire only those who fail it.^[9]

Ultimately, we must relish and make space for the different ways people work, think, and communicate.

As a complement to the book, I love the talk We’re on a Mission From God: The Return of Peopleware, by one of the authors, Tim Lister.

I had the audio recorded on a CD and I used to listen to it on the drive to and from work as a regular refresher. The talk expands on the ideas in Peopleware:

• Don’t view the team as a sports team where the person on the bench is competing with the starter to get their opportunity. View the team as a musical group or glee club where everyone has their voice and part to play.
• From an American perspective, managers are Canadians: they are friendly; they (roughly) speak the same language; they share some similar cuisine and culture; they enjoy visiting us, and we enjoy visiting them; but when the team says “we”, they don’t mean the manager, at least a traditional top-down or positional manager, they mean the team.
• When a team gels, it’s turnover rate goes to near zero, and it’s not because of management, it’s because of the peer group.
• Good teams are collegial.
• The team needs to own all the critical technical decisions.
• Abolish the annual merit rating and management by objectives.
• When the team focuses, they really focus. They become excellent problem solvers. All the manager needs to do is basic care and feeding and get out of the way.
• On a good team there’s a cult of quality, but there is almost no talk of quality—it’s concrete.
• A good engineer will not do some things.
• People cannot sustain working faster or slower than their natural pace.
• Software is not built nine-to-five, it’s built in enormous, focused bursts.
• If the workspace constantly pulls you out of flow concentration, you get really frustrated. It takes 10 to 20 minutes to resume flow after an interruption. If you can’t see a window of time to do focused work, you won’t even start.^[10]
• Show appreciation in private. Tell the person what you saw and why you appreciated it. A blatant compliment is about as good as it gets.
• To keep people, you need to challenge them with interesting projects.
• You cannot be a member of more than one team.
• For every hour over of overtime, you get an hour over undertime.
• People immediately understand real deadlines and they need no explanation. Phony deadlines are just that: phony. With deadlines, teams need to perceive the possibility of success.
• If a project is really important, why can’t we start on it and then evaluate how much work it will be?

One point of caution: reading Peopleware can be frustrating. I have a close friend who started reading Peopleware, on my recommendation, when we were both frustrated by the many dysfunctions of our employer. Peopleware drew his attention to even more dysfunctions, things he knew were not going to change, and it made him even more agitated about his situation. He didn’t even finish the book. You may need to read Peopleware with kind attention to yourself and distance from your current frustrations, but it will provide greater awareness for whatever you do next.

Drive

Building on themes in Peopleware, Dan Pink’s Drive: The Surprising Truth About What Motivates Us helped me appreciate what motivates me in my work. Understanding our motivations, along with what motivates others, is essential for building a great team. Pink’s thesis is that people are intrinsically motivated by three things: autonomy, mastery, and purpose. Autonomy is our desire to be self-directed; mastery, the urge to improve our skills; and purpose, our desire to contribute to something meaningful. Pink has given a very popular TED Talk, but I prefer his more in-depth talk at the RSA as a complement to the book.

People are bright, talented, and naturally motivated. A manager can either embrace this, or destroy it. Two factors determine how people fare on the job. The first are “hygiene” factors: extrinsic rewards like fairness in compensation, working conditions, and job security. Their absence creates dissatisfaction, but their presence will not lead to job satisfaction. Real motivators are things like the enjoyment of the work itself, genuine achievement, quality, learning, personal development, and how creative one feels. Not all of our work is exciting, but if-then rewards only work for routine, mechanical tasks that lack cognitive skill, challenge, and variety. For work that isn’t exciting, show empathy, acknowledge when a task is boring, connect it to a large purpose by offering a rationale for why it is necessary, and allow people to express creativity by completing the task in their own way.

Like Peopleware, Drive is critical of process. Many organizations obsess about the “how” but not the “why”. Policies, standards, and guidelines become nothing more than a checklist, and the goal becomes meeting the minimum requirements, rather than focusing on the larger purpose. Goals people set for themselves are usually healthy, but goals imposed by others—sales targets, standardized test scores, defect rates, lines of code, “story points”, “sprint velocity”—have unintended consequences.^[11] When incentives are high, creativity can be worse due to “functional fixedness”, a cognitive bias where we can only see an object or process in the way it is traditionally used. Extrinsic rewards should be unexpected, not continent on a goal, and offered only after the task is completed. Providing a sense of urgency toward a goal is fine, as long as you then get out of the way and trust. Superior performance is achieved with a process that tolerates early failure, rewards long-term success, and provides freedom to experiment.

I like working on applications for industrial automation and IoT because they provide real value and they must function reliably. I like working on infrastructure because I like working on things that last, evolve, and improve over time, and I like thinking in systems. The only way to enjoy working on these systems is to deliver quality over time and this can be difficult to do in product-focused organizations that don’t understand the latent investments required for scalability, performance, security, backward compatibility, and high-availability.^[12] Drive explains the best indicators of long-term success are perseverance and a passion for long-term goals. Performance improves as understanding deepens: “Greatness and nearsightedness are incompatible. Meaningful achievement depends on lifting one’s sights and pushing toward the horizon.”

To deliver quality work, people need autonomy. It is important to figure out what is important to each member of the team. Pink draws a distinction between empowerment and autonomy. Empowerment presumes the organization has the control and benevolently gives some of it to the grateful employee. But this is not autonomy, this is a slightly more civilized form of control. Empowerment, a prominent feature of the modern workplace, is notable for its lack of engagement and disregard for mastery. Autonomous motivation involves behaving with a full sense of volition and choice.^[13]

Our greatest satisfaction comes from making progress in our work, therefore, it is strange that organizations don’t offer people more focus. Days are disrupted by the noise of open-plan offices, meetings, chat clients, and notifications.^[14] Drive presents evidence that depriving people of flow activities actually promotes anxiety, health problems, and disordered sleep. For highly motivated people in a supportive environment, the boundary between work and play is artificial in terms of their enjoyment. In fact, people are likely to achieve a state of flow at work more often than in leisure due to the autotelic experience: clear goals, immediate feedback, and challenges that just stretch our abilities. We don’t just enjoy our work more, we do it better.

Similar to Peopleware, Drive says the most meaningful recognition comes without tangible rewards, like just reaching out to someone and saying thanks. Feedback should be useful and never used to manipulate behaviours. Focus feedback on specifics and praise the effort and the strategy rather than a particular outcome. In other words, appreciate the thinking, ingenuity, and quality behind the work, not just the end result.

The most successful people, the evidence shows, often aren’t directly pursuing conventional notions of success. They’re working hard and persisting through difficulties because of their internal desire to control their lives, learn about their world, and accomplish something that endures.

We are motivated by innate psychological needs for competence, autonomy, and relatedness. When those needs are satisfied, we are motivated, productive, and happy. When they’re thwarted, our motivation, productivity, and happiness plummet. We are not destined to be passive and compliant, we’re destined to be active and engaged. Control leads to compliance, but autonomy leads to engagement, and engagement leads to mastery: the desire to get better and better at something that matters. Look for leaders, rather than competent managers. Let them set a clear direction, state the outcomes needed, and provide context. Allow people to align their work with the greater purpose of the organization. Then leave them alone to do great work.

Daring Greatly

The final book is not a typical management book. Brené Brown is a research professor studying courage, vulnerability, shame, and empathy. She has written a number of best-selling books and gained prominence by giving one of the most watched TED Talks of all time.^[15]

Shame and vulnerability are ever-present in our work, relationships, and organizations, but usually ignored, or even forsaken. While guilt is the notion that “I did something bad”, shame is the fundamental idea that “I am bad”.^[16] Shame creates a feeling of disconnection, worthlessness, and judgement. When we feel shame, we may rationalize it, withdraw, avoid, attack ourselves, or attack others, anything to diffuse the uncomfortable feeling. Shame is also a powerful tool for manipulating people, especially in the workplace, keeping them in line through fear, ridicule, belittling, cynicism, or blame. There are also subtler expressions of shame like self-worth tied to achievement, productivity, compliance, perfectionism, or overwork. Shame thrives when we keep secrets. At work, if people are afraid to take risks or try new things, or stay quiet, struggling to be seen and heard, almost certainly there is a culture of shame.

We have an automatic, physical response to shame that requires no thinking.^[17] If we want to change our response before we default to withdrawing, avoiding, or attacking, we need to develop the self-awareness to recognize our response. Then we need to start thinking. By doing so, we can write the ending to the story. The antidote to shame is connection: empathy and self-compassion. Empathy is connecting with the emotions someone experiences, not the events, and requires withholding judgement.^[18] Identifying and connecting with our own emotions and withholding self-judgement is just as important. After bringing this awareness, we can reach out, connect, communicate how we feel, and ask for what we need.^[19] We are all going to experience shame, but if we can have a self-aware relationship with it, we can come out the other side with more courage, compassion, connection, and authenticity than we had going into the experience.

To avoid feeling shame, we avoid being vulnerable, but in the workplace, this is incompatible with the fact that creativity, learning, developing relationships, and seizing new opportunities, are all vulnerable experiences. Brené describes how shame experiences at work can often be gendered and quite primal.^[20] Men generally don’t want to be seen as weak, and vulnerability is weakness—men need to be seen as powerful and having the right answers. Women can be driven by perfection, hiding their flaws, trying to do it all, but feeling like they are never good enough, or that they never have an equal seat at the table. But all of us fear being criticized, ridiculed, or embarrassed, and we don’t want to fail, show fear, or not be tough enough. Employees disengage from vulnerability long before other forms of betrayal, like being openly hostile, or finding a new job. It starts by not caring or investing, letting the connection go, and not being willing to devote time and effort to the relationships or the work.

There is no vulnerability without intimacy.^[21] This is why a team needs to feel like a team, not just a collection of individuals and why we need to build trust, incrementally, over time. Interestingly, Brené observes that vulnerability is often the last thing I want you to see in me, but the first thing I look for in you. We need to learn to sit with vulnerability, without discharging the shame and discomfort. “Numbing vulnerability is especially debilitating because it doesn’t just deaden the pain of our difficult experiences; numbing vulnerability also dulls our experiences of love, joy, belonging, creativity and empathy.” In other words, we can’t selectively numb emotion: if we numb the dark, we also numb the light. Therefore, beware of teams that have a hard time expressing joy, compassion, or appreciation.

When you are vulnerable with someone who doesn’t reciprocate, this compounds the sense of disconnection and shame. The relationship must be able to bear the weight through trust, empathy, and asking for what you need. Signs that a person can engage with vulnerability:

• They say, “I don’t know.”
• They ask for help.
• They identify and ask for what they need.
• They ask, “Can you teach me how to do this?”
• They offer help.
• They give difficult things a shot, stretching themselves beyond where they are comfortable.
• They say thank you and show appreciation.

To do our best work, a team must be connected and caring. “Connection is the energy that is created between people when they feel seen, heard, and valued; when they can give and receive without judgement.” Belonging, tying into Drive’s theme of purpose, “is the innate human desire to be part of something larger than us.” Furthermore, “because true belonging only happens when we present our authentic, imperfect selves to the world, our sense of belonging can never be greater than our level of self-acceptance.” There is a dynamic interaction between belonging and self-acceptance: if my community helps me feel like I belong, I’m more likely to feel self-acceptance. I have to be like you to fit in, I get to be like me if I belong.

Giving and receiving feedback, evaluating, and judging are unavoidable aspects of any workplace since they are essential to interviewing, hiring, promoting, structuring organizations, assigning work, one-on-one conversations, and adjusting compensation. Brené believes feedback thrives in cultures where the goal is not “getting comfortable with hard conversations” but normalizing discomfort. Learning new things, critical thinking, change, are all somewhat uncomfortable. Leaders must normalize these feelings, modelling the behaviours they want to see, to connect, and put others at ease. Vulnerability is also essential to the feedback process. Armoring up to give or receive feedback will thwart the process. Once triggered, we will not be accepting and our hardwiring will take over to self-protect.

Brené doesn’t accept feedback from just anyone:

I only accept and pay attention to feedback from people who are also in the arena. If you’re occasionally getting your butt kicked as you respond, and if you’re also figuring out how to stay open to feedback without getting pummeled by insults, I’m more likely to pay attention to your thoughts about my work. If, on the other hand, you’re not helping, contributing, or wrestling with your own gremlins, I’m not at all interested in your commentary.

People whose opinions matter appreciate you for your strengths and your struggles. Rather than holding people to narrow standards, I like recognizing people for their unique gifts and contributions. Instead of framing constructive feedback in the usual strengths and weaknesses, I prefer strengths and preferences: styles of working, career ambitions, values.^[22] It is easy to identify “weaknesses”, but I don’t want many of my “weaknesses” to be my strengths—they just aren’t my preferred ways of working, interacting, or thinking—and I assume the same for others.^[23] Together, we find opportunities to take advantage of strengths and align with preferences. I also try and appreciate how the person works—the quality of their work and interactions—not just the outcomes.

All of us want to feel love, belonging, and acceptance, a sense of worthiness. Below is Brené’s checklist on giving engaged feedback. My goal is to build organizations where this is how we engage with each other so that we can be our authentic self and do our best work.

• I’m ready to sit next to you rather than across from you.^[24]
• I’m willing to put the problem in front of us rather than between us (or sliding it toward you).
• I’m ready to listen, ask questions, and accept that I may not fully understand the issue.
• I want to acknowledge what you do well instead of picking apart your mistakes.
• I recognize your strengths and how you can use them to address your challenges.
• I can hold you accountable without shaming or blaming you.
• I’m willing to own my part.
• I can genuinely thank you for your efforts rather than criticize you for your failings.
• I can talk about how resolving these challenges will lead to your growth and opportunity.
• I can model the vulnerability and openness that I expect to see from you.

Conclusion

All three of these books and accompanying videos have been very influential on my approach to interpersonal relationships, teams, and organizations. They all share common themes, like the importance of connection, purpose, quality, and authenticity. As I mentioned in the introduction, the team I lead is largely an experiment in these ideas. This experiment has turned out pretty much exactly as I expected. It is a team of very talented, creative, dedicated, and motivated engineers, delivering very high-quality work in a safe, supportive, and cohesive environment. We are all very protective of the team and its culture. The nature of the team has evolved as each of us has evolved and as we have added new people, but our core values of quality work, self-awareness, vulnerability, and being motivated by our larger purpose, have not changed. I hope these books and videos provide insight into my thinking and the culture of our team, along with what I think are the most fundamental and universal aspects of engineering management.