A pedestrian was killed in Tempe, Arizona by an Uber autonomous car.  In 2015, Governor Doug Ducey enticed the self-driving car industry to Arizona by executive order clearing the way for testing in the state.  Last month, he updated this order touting Arizona’s “business friendly and low regulatory environment”.  Following the crash, Uber has stopped all real-word testing of its autonomous cars, which were happening in San Francisco, Phoenix, Pittsburg and Toronto.  The accident is now in the crosshairs of both the U.S. National Highway Traffic Safety Administration and the National Transportation Safety Board.

The recent Cambridge Analytica revelations on Facebook data to help Donald Trump’s campaign is ill-timed for autonomous car companies.  And is forcing regulators to increase scrutiny on the level of self-policing that has so far been granted to tech companies generally. The fatality and recent privacy breach revelations will almost certainly adversely impact the pace of autonomous car technology advancement in the U.S.

There are at least two broad black box areas regulators will want to examine and ultimately address.  One is conceptually straightforward, while being technically bedeviling.  Autonomous cars are trained using AI methods such as deep learning on massive amounts of data to interpret and react to driving conditions.  However, unlike traditional statistical predictive methods such as regression analysis, deep learning does not easily lend itself to transparency of decision making, which leaves it with an air of magic about it.  This reality will make it difficult for regulators to communicate with an increasingly skeptical public.

The other issue is philosophically much more challenging.  Inevitably, autonomous cars are going to be in situations requiring them to make an instantaneous choice between a set of bad outcomes.  For example, the decision when an autonomous car is faced with a choice of modest damage to itself versus more material damage to its surroundings.  Even more fundamentally, what happens when lives are at stake?  How will the car measure tradeoffs and react to them?  At some level the processes for making these ethical decisions must be programmed into the car.  The public at large is unlikely to accept a Google, BMW, Ford, or an Uber unilaterally making such decisions.  Recent headlines on Cambridge Analytica that erode public trust in tech companies, and, now, a self-driving car fatality will force a bright spotlight at the core of autonomous vehicle systems.

The rule of law and consumer protection is a strength of the U.S.  At the same time, these strengths could prove to be an impediment in the race for global leadership in the development of autonomous cars, and AI more broadly.

AphaGo, a computer program developed by DeepMind, beat Lee Sedol, a leading exponent of Go, by four games to one in 2016. It has been improving rapidly. In May Alpha Go beat Ke Jie, the number one ranked player, by a score of 3 – 0. DeepMind has since unveiled a new program called AlphaGo Zero. It took just two days of training for AlphaGo Zero to beat the version of Alpha Go used against Lee Sedol.

The difference in programming is that the first program began its training on thousands of actual games played by human experts. The resulting potential winning strategies were then refined using millions of simulated matches played against itself. AlphaGo Zero, however, skipped the initial training phase and just started by randomly playing against itself, before establishing chosen strategies.

The latter method can be of significant advantage in a situation with a lot of structure and an enormous level of possibilities. It avoids the potential inefficiency of having to supply the initial set of “training” data. It also avoids potential human biases in solving problems.

That’s the good news. On the other hand, the real world sometimes thumbs its nose at orderliness and structure. A leading area of research in machine learning is image recognition (cat pictures …). The applications are vast, including driverless cars. Recently, researchers from Kyushu University showed they could consistently get incorrect results by just a one-pixel change in test images. This vulnerability was true of all the state-of-the-art systems the researchers tested.

The timing was unfortunate. Shortly after, the city of Las Vegas debuted a self-driving shuttle bus in November this year. The shuttle had an accident on its first day out when it was unable to avoid a delivery truck that backed into it. No one was hurt. The issue was probably not one of image recognition, since the shuttle stopped when the delivery truck started backing up. However, the shuttle was unable to successfully address a common traffic event.

The future remains bright. There has been more than $20B of M&A activity in AI related fields this year. But, some caution is in order. Richard Branson committed to taking his family on the first flight of Virgin Galactic. Any similar takers on driverless cars?

Sources: The Economist; BBC