“Software updates” Does Not Fix Systemic Risks

Self driving cars are not as trust worthy as many may think and here’s another incident of that. Waymo, a company known for its massive fleet of robotaxis, one of their self-driving cars went around a stopped school bus with its red light flashing and kids stepping off. This is a clear traffic rules violation. 

The road laws require every vehicle to remain stationary around an off-boarding school bus. Obviously the card didn’t get a chance to read the road laws in depth. When the car’s fifth generation AI can’t recognize a school bus stop arm as a hard stop, it exposes how safety validation has taken a back seat in this rapid deployment of self-driving cars. 

In their defense, the company responded, “We have updated the software”, as if nearly hitting children was just another data point for improvement. This is a classic example of Auto industry’s dangerous normalization of learning through near-mis incidents. 

Accountability on Auto-pilot

The National Highway Traffic Safety Administration has launched an investigation of the incident and moved into analyzing the company’s over 2,000 robotaxis and their operational softwares. According to the regulatory authority, this does not look like a single such incident as the failure of software at one point increases the likelihood of it happening on other locations as well. That is why the probe is set to cover over 2,000 cars, because the issue might be built into the entire system and not just one car. 

Quite interestingly, this is Waymo’s third safety investigation. The company has faced 2 investigations owing to the similar crashes prior to this current investigation. Which means that regulators are unable to keep up with the constantly changing software. Every time a crash or violation happens, companies release an update and move on, making it impossible to hold anyone accountable for past failures. 

The result? A regulatory system chasing an evolving target, while unsafe vehicles keep rolling out under the label of “innovation”.

Scale First, Safety Second

Amidst such failures and dangerous incidents, Waymo isn’t expanding, it’s racing ahead with the production of autonomous vehicles. It now operates over 2,000 robotaxis in major U.S. cities. The company reports more than 250,000 paid rides per week and the number is constantly growing.  

Waymo’s business model hinges on expanding rapidly to capture the future of mobility, putting millions of miles driven before proper, life-like, and reliable testing. This speed-first model prioritizes market dominance and data collection growth over secure, fully-validated deployment. 

The message is clear that  the industry believes scale before safety, and the burden of failures may fall on the public long before tech maturity arrives. One should be aware that this isn’t just fender-bender risk assessment, it’s life-threatening safety failures that involve a population that has become a testing subject without giving any consent.  

The incident exposes how the autonomous vehicle industry treats public roads as testing grounds where safety validation occurs through incident response rather than comprehensive pre-deployment verification.