Why Autonomy needs the "Safety Software Engineer"

public domain

Michael Diaz

In the near future when autonomy dominates the technology in our lives, safety engineering skills will be a prerequisite for the majority of software jobs.  Every university and software training program will include it throughout its curriculum.  Students won’t have zero or maybe one project where they’re evaluated on formal consideration for safety - it’ll be expected for every software project.  Because graduating without this experience will leave them unsellable in the job market due to autonomy’s ubiquity.  The only companies willing to take a chance on such ‘traditional’ software engineers will be the future ‘traditional’ dinosaur software companies who didn’t appreciate- or didn’t make - the jump to autonomy.  These stagnant software companies of the future will have the appeal of any automotive companies today ignoring EVs, the role of software in cars, or the entirety of the China market.  Yes - they’ll be that far down the path to extinction.

What the world needs now (besides Love, Sweet Love) with its Rise of the Robots, are agile fast-moving software engineers with extensive training in, and ultimate loyalty to safety.  I call them “safety software engineers”.  In the future they’ll simply be known as “software engineers” because the world will need all software engineers to be “safety software engineers”. So where is the autonomy industry now?  There are a few paradigms today for how safety and software engineering come together.  Let me describe them in order of increasing efficiency:

0. Safety is not considered

I label this zero because in the near future these will all be dead or outdated companies dealing in old heavily-commoditized software that either missed the leap into automation or did so and got litigated to death for doing it without safety considerations.  Software from them will be akin to the fart apps Steve Jobs spoke against when the App Store got serious.  The result was Apple’s App Store guidelines speaking out against “quality Apps surrounded by amateur hour.” There won’t be room in autonomy for amateur hour.

1. Safety is considered late / after initial release

This is where many companies are today - especially startups.  They just can’t justify spending the time, effort and money up front.  They feel the need to reach an MVP or first-customer and then plan to bring in safety.  This isn’t entirely these companies’ fault.  Their investors or shareholders don’t appreciate safety or economic impact and want a return on investment.  And perhaps the consumers of these products are in the same boat.
This is the most difficult archetype to bring safety to for the same reason Level 2 ADAS drivers quickly become erroneously inattentive - ”it seems to be working fine!”  Worse yet when safety is brought in there is the chance for cultural civil war within the company that must be fought and won before safety is integrated into the processes and product.  All the “traditional” software engineers (may be aged 1 to 100) will show their inability to adapt to changing times.  Safety will come at a very high cost because of the amount of culture shock, process re-definition,late documentation, and software rework that will have to be done.  With enough leadership, commitment, money, consulting or outside help, and adaptable software engineers - you’ll make it..  

If a safety event occurs, these companies may not have the runway to come back.  Remember what happened to Uber’s autonomy arm.  Do you think your org can weather it if Uber couldn’t?

2. Safety is considered early

This represents a growing number of players.  These companies have “safety organizations” in the company structure efficiently evangelizing and peddling safety throughout the company’s culture, processes, and products.  Just as important, these safety organizations don’t just hit the software engineers over the head with the good books.  Instead will have the right charisma, teaching skills, and software backgrounds to bring the company’s nascent teams aboard the safety train.  

I’ve seen this done at companies where functional safety training was done early and across all levels (from software    engineers to admins to C-suite executives).

This is close to the ideal (see #3 below) but does require a certain degree of vision and leadership by higher-ups in order to shape and guide company culture.  But your chances of making it are much higher since it’s done early in the process.  Line item costs for achieving safety are spread out from the beginning and the need for a late spike in risk and cost are eliminated.  

3.  Safety has forever been there

This is the ideal and not yet the standard only because enough of the competition is in buckets #0-2.  Today, you aren’t definitely failing if you’re not in #3… yet.  The only way to achieve this level is for your software team to be made of what I termed “safety software engineers” above.  You never had to convince the engineers of the importance of safety engineering and practices any more than you had to convince them to use a compiler -- they know no other practical way to do it!  In this bucket safety is not a line item you have to sell your investors or middle managers on.  It’s like having internet access or a company messaging system.  No one needs convincing - so you just have it from day one. 

Best of all, safety is an open book. It isn’t a blame game or used in political jockeying. A safety issue is like an issue in a basic utility. It’s like a mass server outage. It’s not covered up or downplayed. It’s like a mass server or internet outage - it’s all hands on deck to bring it back NOW.

You may think it sounds expensive or slow or you may see your team in a less-than-ideal group in the list above and don’t want to admit to yourself you need to improve safety in your teams. Meanwhile, software engineers have forever been brought up with one fundamental principle: “Move fast and break things.”  This principle, and the effects it produces in consumer products, is now felt almost by everyone everyday through software updates and the improvement cycles that quickly lead to refined functioning products.  It’s how software “ate the world” (paywalled, but Marc Andreeson said it over a decade ago).  Now technology in the form of autonomy is coming close to deployment at a time when the world has come to take the benefits of this quintessential software development principle for granted.  And autonomy is built first and foremost on software.  

With safety software engineers, "Move fast and break things" becomes "Move faster and break things less"

The surprise here is that safety software engineering, and safety engineering overall, is the greatest way to optimize this principle! With safety software engineers “Move fast and break things” becomes “Move faster and break things less” of all technologies, autonomy is the most inappropriate use of this old software development principle because autonomy so quietly requires extreme safety considerations. Safety in autonomy is a silent, unseen, internal metric. It has no features to show off or impressive customer quality scores. The result in the autonomy industry is a tremendous demand for software engineers along with an underplayed, underrepresented critical need for safety.  Until the universe supply catches up with the demand (market forces will make this happen and intelligent leaders across the industry and academia will hopefully make it happen sooner) - I’m happy to offer my time and energy as a contract engineer to move automotive autonomy forward through safety software engineering.  

I was lucky enough to ‘accidentally’ become a “safety software engineer” through a series of unexpected career moves.  I worked in the automotive industry doing controls software in propulsion systems and autonomous vehicles.  Along the way I happened to work as a systems and requirements engineer which taught me the importance of processes and documentation.  But it was only after I worked at a startup whose leadership had the foresight to prioritize a safety culture and was willing to invest in the right outside help to come teach it to every, and I mean every, employee that I realized two things: 1) Safety is the holdup in autonomous vehicle deployment, profitability and return on investment and 2) Safety helps - never hurts - development efficiency.

How We want to help

Think honestly about which bucket your team is in and how disruptive an outside inquiry into your organization’s safety would be This isn’t about shaming anyone - it’s about being open about where you are and it’s about offering a way to help you move up. You’re not doomed if you’re not in #3. And you’re certainly not alone if you’re in #0. If you disagree with anything I said I want to hear from you all the more either in comments or by speaking directly.

If you need safety software engineering, functional safety, or otherwise want to bring your group to a higher safety bucket - reach out.  Maybe you just need to get started by adding one temporary humble safety software engineer to your team who can code but also share thoughts on safety. Maybe you want a project manager with ultimate loyalty to system safety. Maybe you want functional safety consulting. Maybe you need some more details on safety practices and standards in autonomy today. We provide all of the above and would love to discuss solutions.

Then check out our thoughts on the value of safety, get a little deeper technically with details on possible AV verification frameworks, or run your actual autonomous vehicle data through our RiskEngine risk analyzer. We let you try it on web for free.