We are suffering more, and more severe, cyberattacks on our critical infrastructure because it relies on commercial-grade software such as Windows, Linux, macOS, Android, iOS, and OpenSSL. A safety-critical system is a device which, were it to malfunction or be hacked, could kill or seriously injure people or cause great economic or social damage.
Examples include the power grid, hospitals, cars, airplanes, medical devices, nuclear power plants, water treatment plants, chemical factories, trains, elevators, dams, and voting machines. Attackers can easily purchase one of the thousands of security defects available for sale by hackers. The attackers then exploit the defect to get inside the system, take control, and wreak havoc. As they attack larger and larger targets and are paid larger and larger ransoms without getting caught, their finances, staffing, and confidence grow. As a result, they increase the frequency and severity of their attacks.
The technology giants need to be convinced
The Dawn Project is launching an initiative to stop the use of commercial-grade software in safety-critical systems. Our first initiative is to explain to the technology giants such as Microsoft, Apple and Google, and Open Source advocates why they urgently need to adopt the principle that commercial-grade software not originally developed as safety-critical grade software must not be used in safety-critical systems. Their leadership on this issue will serve as a model for others and pave the way for lawmakers and regulators to incorporate this basic principle into legislation and permanent safety regulations.
Ever since companies began hooking up safety-critical devices to the Internet with commercial-grade software never intended to be safety-critical, computers have been effectively ‘weaponised’. They now pose a grave threat to humanity. Any tin-pot dictator, well-heeled terrorist group, or malign billionaire could destroy the electric power grids that families, cities and nations depend upon.
The programmers who write safety-critical grade software are driven by an acute awareness that one mistake in their software could maim and kill people.
The development process involves painstaking design by highly skilled and experienced experts. Implementation proceeds with a carefully monitored adherence to safety and security directives. Debugging and testing is extraordinarily thorough. All changes are subject to extensive review. The knowledge that any mistake could cause untold fatalities gives pause to the programmers to exercise extreme diligence and care.
Contrast that with programmers who write commercial-grade software for your phone or computer, and know that the worst consequence of an error is that you might have to reboot your device. These programmers are directed to produce the greatest number of the coolest features in the shortest time, as long as they work pretty well. Mark Zuckerberg had “Move Fast and Break Things” painted on the wall. To put it mildly, this is not a dictum that puts safety at its heart.
The reality is that commercial programmers know that their software fails sometimes—and they know that it can be hacked. It was not designed, developed, documented, or tested adequately to be safety-critical grade software, because the developers never imagined that a single error they made could put millions of lives at risk. The result is that commercial-grade software not intended for use in safety-critical systems always contains large numbers of bugs and security defects which can easily be exploited by hackers.
Isn’t it obvious that safety-critical systems must not use commercial-grade software?
Apparently not. The desperately worrying reality is that the makers of most safety-critical systems are putting commercial-grade software, never intended for safety-critical uses, into many safety-critical systems.
Aircraft and hospitals use safety-critical software
Everyone agrees that commercial airliners must only use components designed, developed, and tested as safety-critical devices. No one would use common hardware store bolts to attach cabin doors to a commercial airliner. The cabin door bolts on a Boeing jet must be tested to maintain their strength over extreme temperature and pressure ranges. There is nothing wrong with hardware store bolts, but the people who designed and manufactured them intended they would hold the screen doors onto your house. They could not imagine that anyone would ever be so reckless as to use them to hold cabin doors on a commercial airliner. If they did so, no one would be surprised if the bolts broke and the cabin doors blew off the airliner at 35,000 feet, 600 MPH, and 75 degrees below zero causing hundreds of people to be injured or die. All sensible people would agree that the aviation ‘experts’ who fitted the hardware store bolts on an airliner should be prosecuted and if convicted, sent to jail.
Similarly, everyone would agree that the software in a hospital management system is safety-critical and must be designed, developed, and tested as such. No one should use software developed for the office or phones in a hospital management system. The developers of that software probably did not imagine that anyone would be reckless enough to use their software in a safety-critical system, where one mistake could cost countless lives. If that were to happen it would not be surprising if the software was hacked and disabled. If the hacking resulted in doctors and nurses being unable to determine what treatment or medication a patient needed, lives would be endangered or lost.
How we get started
We need to make computers safe for humanity by requesting that the CEOs and open-source managers of the commercial-grade software not developed from the start for safety-critical systems, including Windows, MacOS, Linux, Android, and OpenSSL, support our cause by:
- Signing up to The Dawn Methodology principle that safety-critical systems must not use commercial-grade software that was not originally and continuously designed, implemented, documented, and tested for safety-critical applications.
- Clearly stating in public forums including their websites, marketing literature, legal disclosures, SEC disclosures, that their commercial-grade software must not be used in safety-critical systems.
- Ensuring that their license agreements prohibit the use of their commercial-grade software in safety-critical systems.
- Sanctioning any of their salespeople who sell their commercial-grade software to customers to use in safety-critical systems.
Safety-critical applications are unlikely to constitute 1% of these companies’ sales. The Dawn Project would urge and hope that these companies will happily sacrifice less than 1% of their sales so that:
- Their software never harms anyone.
- They will avoid the legal liability and the bad press that will attach to them when something goes catastrophically wrong due to a failure of their software in a safety-critical system.
- All of the tech giants and the Open Source advocates can celebrate doing something positive and essential for humanity by endorsing the simple, obvious, and uncontroversial principle that safety-critical systems should only use software that was originally and continuously designed and developed for use in safety-critical systems.
Will the tech giants agree to endorse this principle? The Dawn Project’s first task is to convince them to do so.