Driverless Cars and the Myths of Autonomy

Every robot, like every person, is autonomous within a social setting, and must navigate its relationships to others. Working in isolation is conceivable, indeed appealing to a certain mindset, but unrealistic in the human world.
This post was published on the now-closed HuffPost Contributor platform. Contributors control their own work and posted freely to our site. If you need to flag this entry as abusive, send us an email.


We hear a lot these days about the goal of "full autonomy." In a dawning robotic era, we're told by Google's automobile engineers and others, machines will act completely on their own, collecting data about the world, making informed, rapid decisions and learning from their experiences. On a commonly used scale of levels of autonomy, level one is fully manual control and level 10 is full autonomy. With its familiar rising numbers, the scale itself seems to imply full autonomy as the inevitable, ultimate goal of a natural trajectory of progress in robotics and automation.

History and experience show, however, that the most difficult, challenging and worthwhile problem is not full autonomy but the perfect five -- a mix of human and machine and the optimal amount of automation to offer trusted, transparent collaboration, situated within human environments.

Consider the Apollo lunar landings. In the early 1960s, engineers began to build the computer that would guide NASA's astronauts to the moon. At first they thought that the computer's interface should have only two buttons. One would be say "go to moon," and the other, "take me home" -- nearly level 10 on the autonomy scale.

As the Apollo program progressed, with human lives and national prestige on the line, the engineers gradually added human approvals and inputs. The machine that eventually landed Neil Armstrong and Buzz Aldrin on the moon (see picture above) did indeed include an advanced digital computer running complex software (the first digital fly-by-wire system). But it worked as a perfect five. With a rich set of controls (and real-time collaboration from Houston), the astronauts could vary the level of autonomy, adding or disengaging aspects of the computer in response to alarms and distractions. All six attempts at landing succeeded.

The futures of driverless cars today promised by Google and others have a utopian quality similar to the early Apollo level 10 visions: thanks to full autonomy, computers control our transport, leaving us to read, sleep, or generally laze about while cruising down the highway or sitting in traffic. Human frailty, inattention, and error behind the wheel will fade into the past; horrific highway fatalities will be known to future generations only as the sacrificial toll of a primitive mechanical era. But that's not how it works.

No robot is fully autonomous. Even the unmanned spacecraft that travel to Mars, Pluto, and beyond carry human instructions and report back to scientists on the ground who programmed them and interpret their data. A 2012 report by senior scientists in the Department of Defense, an agency itself burned by expensive, utopian visions of full autonomy, put it thus: "There are no fully autonomous systems just as there are no fully autonomous soldiers, sailors, airmen or Marines." Even expensive, highly-automated airliners can fall out of the sky when they surprise their human pilots.

Every robot, like every person, is autonomous within a social setting, and must navigate its relationships to others. Working in isolation is conceivable, indeed appealing to a certain mindset, but unrealistic in the human world. As robots leave the laboratory to play important roles in social and economic life, we must engineer them to work safely within webs of interdependence, not to act with full autonomy.

In fact, the idea that machines are somehow moving toward level 10 full autonomy according to some internal logic is a myth. As robots and automated systems move from the lab into the field, we see from the Apollo example and countless others like it that when the stakes are high, we gradually, but indelibly, insert human approvals and intention back into the loop, seeking some version of a perfect five.

In aviation, the full autonomy goal of an unmanned aircraft that can takeoff, fly to a destination, and land on its own was attained in research settings at least a decade ago. By contrast, allowing drones and unmanned aircraft in national airspace that has to be shared with airliners and other users is tremendously difficult. This situated autonomy requires new technology, and new approaches to policy, reliability, certification, cultural perceptions, and trust which level 10 full autonomy cannot deliver.

Google has aimed for full autonomy with its latest, bug-like driverless cars. The challenge, however, is to make these machines work in diverse human settings, where pieces fail, strange things happen, and the people around may or may not be cooperating. (The New York Times recently reported Google engineers' frustrated surprise at learning that people don't always follow the rules when they drive).

It takes more sophisticated technology to keep the humans in the loop than it does to automate them out. Rather than the weak links, people offer the strong links, constantly making small corrections for failures of design, communication, or execution, and often doing so through social relationships. Fully autonomous cars push those corrections into the cubicles of computer programmers, at our peril.

What we ought to seek then, is not the level 10 but the perfect five of autonomy: true collaborations between humans and machines, where the autonomy is trusted, transparent, and available but never dominates, always subject to our instructions and aims. Give me all the laser scanners, virtual maps, and computer-aided vision I see on driverless cars, but make them my tools as a driver. Relieve the tedium of driving, allow me minor distractions, enhance my family's safety. But keep my life in my hands, and use the technology to deepen my engagement with the world around me. Just ask a Volkswagen diesel customer what it's like to discover that the values embedded in your car's software are not your own.

Perfection will of course be unattainable, but the goal of the perfect five enables us to rethink our notion of progress, away from a mythical trend toward full autonomy, and toward a trend of deep, trusted collaboration.

None of this is easy. The perfect five is an elusive goal that requires us to address transfers of control, human distraction, and rapid responses to system failures. But we will face these problems anyway, because full autonomy will never be perfect, and will require us to intervene on a moment's notice.

Popular in the Community


What's Hot