Since starting this blog over six years ago, my cadence has been to publish one article a month. I enjoy long-form writing and it would be near impossible for me to publish essays more frequently. As I write this essay, the world is gripped by the SARS-CoV-2 pandemic. All over the world, people are staying home to slow the spread of the virus and avoid overwhelming our healthcare systems, attempting to arrest the spread of the virus, or buy time for the development of effective treatments.
This pandemic has already had a tremendous impact on how most people live and work. Teachers are adapting to wrangling web technologies and video conferencing to continue instructing students. Most people in high technology industries, including engineers and software developers, are working from home. These are essential measures for slowing the transmission of the virus. Unfortunately, this is juxtaposed with many other people who cannot isolate at home putting themselves at increased risk, including people working in health care, delivery services, grocery stores, and emergency services. The stories of doctors, nurses, paramedics, and hospital staff are heroic. The stories of people with the disease, COVID-19, are heartbreaking and devastating.
Given this backdrop, I am not inspired to write this month. I have been working on essays on the topics of stateful serverless; streaming APIs for time-series and events; communicating complex state without using strings; and stereotype threat as it relates to hiring, career advancement, and meritocracy in the workplace. But none of these topics seem that important right now. Like you, I am finding it harder to concentrate. I am finding it harder to balance work. I am finding it harder to attend to my emotions and anxiety.
For those of us who are privileged to work from home during this time, it is not that I think we should take a three-month sabbatical and fail to make progress on the most important challenges of our generation: climate change and renewable energy, food and agriculture, cancer and infectious disease research, mental health and social well-being, to name a few. This crisis should refocus our attention on the most pressing problems in the world. I do feel like reading, researching, experimenting, and learning, but I don't feel like writing or sharing my ideas at the moment, especially on technical topics. Maybe I will take a break from writing, maybe I will just write more erratically, or maybe I will feel more invigorated next month. Time will tell.
For many of us, this pandemic has been a time of reflection, a time to consider what we hold most important in life. I hope this crisis will have a lasting impact on the way we work and a transformational impact on many cultures and organizations. With it being unclear if I will write more in the coming months, for this month, I will reflect on the future of work through the lens of this crisis. I will conclude with a call to action. If this blog goes quiet for a while after this, you will understand why.
The ubiquity of the Internet and the development of tools that support remote work have made remote work increasingly popular over the past two decades. The many benefits of remote work include: eliminating the commute; reducing carbon emissions; living in more affordable areas; improved focus with fewer distractions; more time for hobbies, friends, and family; and the ability to hire a more diverse workforce. In some industries, like manufacturing, transportation, and healthcare, remote work is not always possible. But with innovations in technology and automation, even these industries are benefiting from more remote work. For example, the ability to video conference with a doctor is not only convenient, but it helps bring specialized services to new populations.
Despite the benefits of remote work, many companies insist on building campuses in some of the most expensive cities in the world, in order to co-locate teams. There are many benefits of co-located teams. Even just the incidental conversations that come from having coffee or lunch together can have very positive effects on connection, communication, creativity, and problem solving. But what are the thousands of people at places like Facebook headquarters doing, other than enjoying the "free lunch"?
When I think about why tech often fails to serve our interests, I think about rooms like this. So long as software requires large rooms of people staring at computers all day, every day, forever—I think there will often be a mismatch between how we wish tech worked and how it does.
I have never worked entirely remotely. My team works from home two days a week and I have worked remotely for weeks or months at a time to spend time home in Canada. I have been working exclusive from home for the past four weeks. For many people adjusting to this new situation, it is as much living-at-work as it is working-from-home. With schools closed, people are balancing work with children who are also home and adapting to remote learning. I wouldn't be surprised if there are some people learning how to cook at home or clean their apartment for the very first time. Most demanding of all, some people working from home are managing symptoms, caring for a sick family member, supporting someone in the health-care industry, managing the stress of continuing cancer treatment or dialysis, deferring routine medical treatment, or managing a pregnancy during this pandemic. This is not normal remote working.
Despite the challenging circumstances, as many companies have a large fraction of their employees work remotely for months through this crisis, I think they will come to appreciate the benefits of remote work. This forced experiment will make organizations that were not open to remote work re-evaluate the costs and the benefits. They will see that people continue to be productive. They will appreciate the benefits to business continuity of not having employees concentrated in one geographic area. They will turn to remote work for talent acquisition, cost reduction, and the ability to hire a more diverse workforce that can appreciate and serve the needs of customers more effectively. Even if an organization does not transition entirely to remote work, having people work remotely a day or two a week will be seen as a way to continuously exercise remote work as a core competency of the business for times like this.
Most countries identify critical infrastructure as some combination of the following industries: oil and gas, food and agriculture, energy, financial services, government, healthcare, manufacturing, communications, utilities, transportation, defence, and emergency services. To varying degrees, these mature industries are prepared for exceptional circumstances and have responded to incidents before, ranging from natural disasters, to systemic failures, cyberattacks, epidemics, pandemics, and wars.
SARS-CoV-2 is the third pandemic caused by a coronavirus in the past two decades, with SARS-CoV-1 and MERS-CoV being the other two. In the past two decades there have also been pandemics resulting from the Ebola virus and Influenza A subtype H1N1, commonly called Swine Flu. The continuing HIV/AIDS pandemic, now close to four decades old, demonstrates the challenge of developing effective vaccines and antiviral drugs. Pandemics are hardly a thing of the past and are perhaps made worse by globalization and the ease of international travel.
I believe this pandemic will lead to an expansion of what we consider critical infrastructure and it will change how we value many jobs and industries. With the shift in many economies to providing essential goods and services online, should the people working in online fulfilment, distribution, and delivery not be considered critical infrastructure? For many people isolating at home, these are the people who are allowing them to isolate in the first place, while still enjoying the delivery of food, alcohol, toiletries, and other essentials. Should the person working at the grocery store, hardware store, carry-out restaurant, or farmers market, making a relatively low hourly-wage with few, if any, benefits, putting themselves at much greater exposure to SARS-CoV-2 than any tech worker isolating at home, not be considered equally part of our critical infrastructure?
The cloud infrastructure providers—Amazon, Microsoft, Google, Alibaba, and others—must also be considered critical infrastructure, since they are running workloads and applications for healthcare, government, defence, energy, manufacturing, transportation, and emergency response, not to mention keeping the online fulfilment and distribution networks, that I just mentioned, functioning. Microsoft has reported a 775 percent increase in video conferencing traffic in Italy after social distancing and shelter-in-place orders were enforced. The cloud is known for its elasticity, but for the first time, the cloud providers may be faced with decisions like segregating and protecting critical workloads for healthcare, energy, government, and emergency services, from people trying to stream some mediocre television series, numb the mind with a video game, or video chat with Grandma.
I trust this pandemic will strengthen people's focus on energy reliability and independence. For many people isolating at home, life isn't too bad, as long as there is electricity, food, and the Internet. For critical services like hospitals, manufacturing, and food distribution, reliable energy is of supreme importance. It is impossible to manufacture personal protective equipment, medicine, or ventilators, let alone operate ventilators or other Intensive Care Unit (ICU) equipment, without reliable energy. The ability for homes and business to be more energy independent will become more appealing. In a crisis like this, it is much more appealing to recharge an electric vehicle at home, rather than refuel at a gas station, or rely on a national, or global, distribution network for fuel. The appeal of generating renewable energy, locally, through solar, wind, or hydro, and storing it in batteries that can operate flexibly and independently, will become increasingly important to home owners and businesses. There is some concern that oil prices falling to near historic inflation-adjusted lows will curb investment in renewable energy and electric vehicles. For example, low fuel prices could make electric vehicles less cost-competitive with internal-combustion-engine vehicles. But it may also have the opposite effect: low prices may curb investment in oil and gas exploration and development. Should prices increase, they could increase sharply due to the lack of investment, making renewable sources less risky and more attractive long-term investments.
In addition to the focus on energy independence, I think we will also see a renewed focus on local manufacturing. The pandemic has highlighted the challenges of manufacturing and supply chains concentrated in certain parts of the world, some with only a single manufacturer or supplier. Even if we wanted to manufacture and distribute a lot of tests, masks, ventilators, antiviral drugs, or vaccines, most countries do not have the manufacturing capacity or expertise. This pandemic is also putting strains on the manufacturing and supply chains for antibiotics and reagents for other routine testing. We are in a global market for all of these products. We typically plan for disasters in a single place, the result of a hurricane, a fire, a power blackout, or something similar. This is arguably the first time our consolidated, interconnected, and globalized economy has faced a worldwide disaster. Flexible and reliable local manufacturing, decoupled from the rest of the world, will receive renewed attention and be the source of new jobs.
I am not a political person and this is not a political blog, but I want to say a few words about government and how it relates to the future of work in light of this crisis.
In software and engineering incidents, we like to remind people that the systems we work with are complex and dynamic. Looking to assign blame or find a "root cause" is considered counter-productive, or even harmful. When many of us turn to politics, however, we become consumed by emotions and our mindset shifts to that of group and identity warfare. We oversimplify problems and assign blame without acknowledging that government is an equally complex beast.
From my perspective, the majority of governments around the world have responded similarly to SARS-CoV-2. There is very little evidence that larger governments, for example, the European Union, have provided a more effective response than local ones. There is a possibility that the variation in outcomes, even within countries, may be largely attributed to chance. Hindsight bias makes it easy to say that governments, or leaders of individual organizations, should have acted differently, but balancing an overreaction to every outbreak of disease with the potential for exponential growth, is a tough balance to strike. In software or engineering incident response, assigning blame or politicizing issues is never helpful, particularly during the incident, so let us avoid doing this with government and focus on more constructive outlets.
All together now: “multiple contributing factors”.
Coordinating a pandemic response across different levels of government—testing, travel restrictions, emergency response, essential services—is challenging, to say the least. Adding to the challenge, many governments around the world are polarized, dysfunctional, and highly leveraged. But responding to an epidemic or pandemic needs to be a core competency of governments. While private or market-based institutions can also play a role, they do not have the same capacity to coordinate a response. We have had crises in the recent past that could have refocused governments on their critical functions, but that failed to happen. Perhaps the magnitude of the current crisis will lead to an improved functioning of governments around the world, or even a change in the people who participate in government institutions that are desperately lacking leadership.
Many governments have promised large economic stimulus plans, or direct support of citizens. Putting aside the unintended consequences that may result from the moral hazard, monetization of debt, and inflation, I generally do not trust governments to invest in the best new technologies or industries, as governments will tend to cater to established industries, ripe with voters. While this short-term and highly-politicised economic support will be essential for many people around the world, I also think this crisis will make programs like Universal Basic Income (UBI) appeal to many. Rather than UBI being a fringe idea struggling for mainstream appeal, it may become viewed as a more just and humane way of supporting the basic needs of everyone, through good times and bad, rather than crafting social assistance programs that discourage work, encourage specific behaviours, and are complicated to administer. If we see hyperinflation, this crisis may lead to a renewed focus on sound money. And I think it is fair to say that having healthcare coverage and benefits tied to employment is also something that this crisis may finally change.
You are probably wondering what the image at the top of this article has to do with the future of work. That picture was taken ten minutes before I gave a presentation at QCon London 2020, with my friend and colleague Percy Link, as people were filing into the room. We put a lot of work into that presentation and we were excited to share it, since it was our first opportunity to share a small part of what we work on. At the time, it was still unclear how SARS-CoV-2 was spreading around the world, in no small part due to the lack of testing. However, people were starting to take precautions, including the conference organizers. With that said, the pubs, restaurants, museums, and the London Underground were still chock-a-block. London is one of my favourite places in the world and reflecting on this experience—travelling internationally, presenting at a major conference, meeting new people, catching up with friends, walking all over London, eating in great restaurants, and enjoying cask ales in my favourite pubs—just before the whole world came to a standstill, is surreal.
The concluding message of our talk was twofold. First, software is key to solving many of the most important problems in the world, from providing clean and reliable energy, to managing and curing infectious diseases. Second, the key to solving these problems is not the hardware, or the algorithms, or the data, or the software, but the system itself. We should take our talents as software developers and focus on these most pressing problems.
When I look at technical solutions or proposals in the grid space, they typically discuss "algorithms" for solving various problems. But the key part of modern distributed computing systems is the system itself—connecting all the pieces, managing data flows, correcting errors, creating models, visualizing system behavior, managing distributed state and decision-making, reconciling place and time (this is harder than it sounds), managing reliability, distributing work...once you've done all that, algorithms begin to matter.
I often hear people say that they are motivated and care deeply about the quality and the impact of their work, but they yearn to get closer to the customer. I have spent two decades working on infrastructural software. I enjoy laying the foundation for others to build products and solutions, like manufacturing more efficiently, providing data in near real-time, rather than in batch, or connecting systems and providing integrations that were not possible before. Equally important are the people working on a programming language, an open-source library, a cloud infrastructure, a communications tool, or a hardware component that becomes the foundation for others. Remember, it is about the whole system.
As an industry, software is not as mature as the traditional engineering disciplines, or many of the industries considered critical infrastructure that I mentioned earlier. However, I find this kind of message dangerous:
There is no such thing as a "best practice" in software development. There are "good ideas," but that's far different from a "best practice." To imagine there are is a fundamental misunderstanding of how software development works. A structural engineer can be jailed for not using best practices.
Those who have worked on software for critical infrastructures in industries like transportation, energy, manufacturing, and healthcare will appreciate the fallacies in this statement. Software systems in these industries are imperfect and best practices are continually evolving, but things like testing, peer review, authentication, authorization, encryption, least-privilege, threat modelling, source control, change management, auditing, monitoring, alerting, and off-site backups, among other conventions, are considered industry-standard best practices for the development and operation of software, not just good ideas. In certain regulated industries, like the ones that fall under NERC CIP, HIPPA, or financial regulation, some of these best practices are codified in law.
The software industry is dominated by too many people just hacking code. Perhaps even more troublesome, most software developers implement and operate software without the legal, ethical, or professional backing established for professionals like engineers, doctors, and lawyers. Imagine a software engineer with the duty and agency to delay a software release because it is unsafe or insecure without fear of reprisal. As we realize how important software is to the functioning of our critical infrastructures, and how important software is to solving the most important problems of our time, it is a chance to accelerate the maturation of software development and operations into a true engineering discipline.
Let's Work on Problems that Matter
This pandemic will end one day. I hope the lessons we learn from it will transcend and endure. Many people will want to go back to normal, but for many others, there will be no normal to go back to. With what undoubtedly started with one individual, in one corner of the world, contracting the virus, has ended up with the entire planet on lock-down, with a very uncertain future. It demonstrates how truly interconnected we all are, even if being interconnected is not our cup of tea.
This crisis should have a transformational impact on the future of work, from remote work, to how we value work, to what we view as critical infrastructure. I hope it also impacts work by bringing a renewed focus on local manufacturing, energy independence, and effective government. In the past, innovation has thrived in times of crisis. This time should be no different. Let's use this opportunity to refocus on what is most important in our lives. As software developers and engineers, let's choose to work on the most important and lasting problems that we can find.
I usually publish my blog on the first or second Sunday of each month at about 10 AM Pacific Time. I usually skip January. I find it hard to have enough time to write over the holidays and I also enjoy the break from writing, starting the year with more time to read and explore. It also gives me some enjoyment that, on an annual basis, my blog goes "up to eleven" essays. ↩︎
An approach that I find helpful: rather than suppressing emotions and anxiety, bring awareness to them, but treat them like house guests. You can welcome them in and learn what they are bringing attention to, but you make the house rules, and you can ask them to be on their best behaviour. The first time I came across this idea was the reference to Stephanie Noble's essay Emotions as Honored Guests in Indi Young's book Practical Empathy. See also the Poem entitled The Guest House by Rumi. ↩︎
For those of us able to isolate and work from home, we should remember what a privilege it is. Many people have already lost their jobs. Many others must still go to work and cannot work from home. Even the privilege of using soap and water to regularly wash our hands and protect ourselves from the spread of the virus is not the same in communities that do not have running water. Depending on where in the world you live, even "social distancing" is a privilege. ↩︎
Many of us have been relying heavily on tools for video conferencing. Some platforms are distinctly better than others. In terms of working, these are things I really appreciate: a gallery view where I can see all participants, even in a large meeting with tens of people; good filtering of background noise, so that I only hear people talking and not whatever else is going on in the background; helping the next person jump into the conversion without people talking over each other—the gallery view helps with this, but some platforms are still better at this than others, perhaps through audio filtering or participant prioritization; a clear indication that my screen is being shared, so that I don't need to always ask, "Can you see my screen?"; the efficient use of CPU, so that my computer remains functional for other tasks; a whiteboard tool; and meetings that are easy to share and have people join and leave without a cumbersome invitation process. ↩︎
Part of my desire to work remotely in the summer was to spend time with friends and family and enjoy the wonderful Ontario summers. Some years it also coincided with development and installation work I had to do at some customer sites. But the biggest reason for working remotely in the summer was for me to play in my band at the time, the 78th Fraser Highlanders Pipe Band, and compete annually at the World Pipe Band Championships. The tools that support remote work have improved tremendously in the past two decades, as have the tools for capturing and sharing live performances. ↩︎
The California Independent System Operator (CAISO) has isolated critical employees at its control centres. The New York Independent System Operator (NYISO) has taken the even more extreme measure of sequestering grid operators, having them live on-site to ensure the continuity of operations. This gives new meaning to the term "working from home". The energy industry is probably one of the most prepared for these emergencies, performing exercises and drills on an annual basis. There are even contingencies for National Grid and Consolidated Edison, two of NYISO’s member utilities, to operate parts of the regional grid should that become necessary. ↩︎
See my article On Embracing Error in Distributed Software Systems. ↩︎
Some people are misinterpreting this statement. The issue seems to be my use of the word blame. I am not suggesting we suppress strong emotions, or avoid holding people accountable. I am using blame in this context: Brené Brown on Blame. ↩︎
Even in countries with socialized medicine, like Canada, people with benefits through their employer have significantly better access to treatments for diabetes, dental care, and mental health. The reason health coverage for many people in the United States is tied to employment is a strange result of trying to control wages during World War II, combined with the discovery of penicillin. One of my favourite books is Economics in One Lesson: The Shortest and Surest Way to Understand Basic Economics by Henry Hazlitt. It is based on Frédéric Bastiat's essay from 1850 "Ce Qu'on Voit Et Ce Qu'on Ne Voit Pas". In English, "What is Seen and What is Not Seen". The simple lesson is that in economics, we must look not just at the immediate effect of an act or a policy, but also at the long-term, latent and systemic effects. Healthcare being tied to employment is a great example of this and we should remember this as governments around the world develop new acts and policies in response to this pandemic. ↩︎
Please do not interpret this as a general criticism of Allen. Much of what he has to say on software development practices is pure gold and you should follow him on Twitter. ↩︎
Exploring the evolution of software development to become a true engineering discipline deserves an essay of its own, but I describe some of the reasons I think software development and operations will look more like traditional engineering in my article Observations on Observability. ↩︎