Tesla has been misusing the term “beta” to justify selling defective Full Self-Driving software to consumers for use on public roads.

I have developed safety-critical software for decades. I designed and wrote the core security system for our nuclear forces, military and commercial aircraft, Top Secret encryptors, laptops for FBI field agents, and military cell phones and radios. The operating system I designed and wrote is certified by the FAA to the highest level of safety and reliability: DO-178C Level A.

Here is a basic outline of how to test software, and why FSD is not beta software.

Developer Testing

As the developers write software they write tests to help them debug the software. These tests are usually not comprehensive enough to ensure that the software will consistently perform in the way it is expected to.

Test Engineering

Test engineers are engineers who are dedicated to testing software. They should not be development engineers. Their job is not to build the software, their job is to break the software. They develop software simulated environments in which to test. They develop closed courses to test the software. They must write tests for all of the functionality of the software.

The bugs are sent to the developers to fix them. This process continues until the developers can fix the bugs faster than the test engineers can report them. Only then does the process advance to alpha testing.

This process ensures that basic functionality is achieved without putting the public at risk.

Alpha Testing

For alpha testing a complete product in the form it is expected to be delivered to customers is given to employees who use the product as a customer is expected to use it. The employees report all of the bugs they encounter back to the developers to be fixed.

Only after the developers can fix bugs faster than the alpha testers can report them, does the process move to beta testing.

Beta Testing

A company’s employees may not match the diversity of a company’s customers. To avoid embarrassing problems with a full customer release, a small group of long-term loyal customers from diverse groups that represent the full customer base are recruited to use the software as they normally would. They report their problems back to the developers.

When the developers can fix bugs faster than the beta testers can report them, more beta customers are brought in. When the beta testers are satisfied that the product is ready for full customer use, the product is released.

What is Tesla doing wrong?

Tesla is doing completely inadequate test engineering. The Dawn Project has developed many tests that demonstrate severe safety defects in FSD, like showing that FSD doesn’t understand common traffic signs like Do Not Enter and Road Closed. It will blow past a stopped school bus with its red lights flashing and stop sign extended and run down a child in the road. These bugs have been reported to Tesla, but they haven’t been fixed in over a year.

FSD YouTubers have also reported many additional safety-critical defects that still haven’t been fixed:

Drivers Have Reported Thousands Of Critical Safety Defects In Tesla’s Full Self-Driving Software

We can find severe safety defects in FSD faster than Tesla developers can fix them, so FSD should be in the test engineering phase.

FSD is not ready for alpha or beta testing. There is absolutely no reason to endanger the public by putting 400,000 defective Full Self-Driving cars on the road.