Five years to the day after I criticized Uber for testing its self-proclaimed “self-driving” vehicles on California roads without complying with the testing requirements of California’s automated driving law, I find myself criticizing Tesla for testing its self-proclaimed “full self-driving” vehicles on California roads without complying with the testing requirements of California’s automated driving law.

As I emphasized in 2016, California’s rules for “autonomous technology” necessarily apply to inchoate automated driving systems that, in the interest of safety, still use human drivers during on-road testing. “Autonomous vehicles testing with a driver” may be an oxymoron, but as a matter of legislative intent it cannot be a null set.

There is even a way to mortar the longstanding linguistic loophole in California’s legislation: Automated driving systems undergoing development arguably have the “capability to drive a vehicle without the active physical control or monitoring by a human operator” even though they do not yet have the demonstrated capability to do so safely. Hence the human driver.

(An imperfect analogy: Some kids can drive vehicles, but it’s less clear they can do so safely.)

When supervised by that (adult) human driver, these nascent systems function like the advanced driver assistance features available in many vehicles today: They merely work unless and until they don’t. This is why I distinguish between the aspirational level (what the developer hopes its system can eventually achieve) and the functional level (what the developer assumes its system can currently achieve).

(SAE J3016, the source for the (in)famous levels of driving automation, similarly notes that “it is incorrect to classify” an automated driving feature as a driver assistance feature “simply because on-road testing requires” driver supervision. The version of J3016 referenced in regulations issued by the California Department of Motor Vehicles does not contain this language, but subsequent versions do.)

The second part of my analysis has developed as Tesla’s engineering and marketing have become more aggressive.

Back in 2016, I distinguished Uber’s AVs from Tesla’s Autopilot. While Uber’s AVs were clearly on the automated-driving side of a blurry line, the same was not necessarily true of Tesla’s Autopilot:

In some ways, the two are similar: In both cases, a human driver is (supposed to be) closely supervising the performance of the driving automation system and intervening when appropriate, and in both cases the developer is collecting data to further develop its system with a view toward a higher level of automation.

In other ways, however, Uber and Tesla diverge. Uber calls its vehicles self-driving; Tesla does not. Uber’s test vehicles are on roads for the express purpose of developing and demonstrating its technologies; Tesla’s production vehicles are on roads principally because their occupants want to go somewhere.

Like Uber then, Tesla now uses the term “self-driving.” And not just self-driving: full self-driving. (This may have pushed Waymo to call its vehicles “fully driverless“—a term that is questionable and yet still far more defensible. Perhaps “fully” is the English language’s new “very.”)

Tesla’s use of “FSD” is, shall we say, very misleading. After all, its “full self-driving” cars still need human drivers. In a letter to the California DMV, the company characterized “FSD” as a level two driver assistance feature. And I agree, to a point: “FSD” is functionally a driver assistance system. For safety reasons, it clearly requires supervision by an attentive human driver.

At the same time, “FSD” is aspirationally an automated driving system. The name unequivocally communicates Tesla’s goal for development, and the company’s “beta” qualifier communicates the stage of that development. Tesla intends for its “full self-driving” to become, well, full self-driving, and its limited beta release is a key step in that process.

And so while Tesla’s vehicles are still on roads principally because their occupants want to go somewhere, “FSD” is on a select few of those vehicles because Tesla wants to further develop—we might say “test”—it. In the words of Tesla’s CEO: “It is impossible to test all hardware configs in all conditions with internal QA, hence public beta.”

Tesla’s instructions to its select beta testers show that Tesla is enlisting them in this testing. Since the beta software “may do the wrong thing at the worst time,” drivers should “always keep your hands on the wheel and pay extra attention to the road. Do not become complacent…. Use Full Self-Driving in limited Beta only if you will pay constant attention to the road, and be prepared to act immediately….”

California’s legislature envisions a similar role for the test drivers of “autonomous vehicles”: They “shall be seated in the driver’s seat, monitoring the safe operation of the autonomous vehicle, and capable of taking over immediate manual control of the autonomous vehicle in the event of an autonomous technology failure or other emergency.” These drivers, by the way, can be “employees, contractors, or other persons designated by the manufacturer of the autonomous technology.”

Putting this all together:

1. Tesla is developing an automated driving system that it calls “full self-driving.”
2. Tesla’s development process involves testing “beta” versions of “FSD” on public roads.
3. Tesla carries out this testing at least in part through a select group of designated customers.
4. Tesla instructs these customers to carefully supervise the operation of “FSD.”

Tesla’s “FSD” has the “capability to drive a vehicle without the active physical control or monitoring by a human operator,” but it does not yet have the capability to do so safely. Hence the human drivers. And the testing. On public roads. In California. For which the state has a specific law. That Tesla is not following.

As I’ve repeatedly noted, the line between testing and deployment is not clear—and is only getting fuzzier in light of over-the-air updates, beta releases, pilot projects, and commercial demonstrations. Over the last decade, California’s DMV has performed admirably in fashioning rules, and even refashioning itself, to do what the state’s legislature told it to do. The issues that it now faces with Tesla’s “FSD” are especially challenging and unavoidably contentious.

But what is increasingly clear is that Tesla is testing its inchoate automated driving system on California roads. And so it is reasonable—and indeed prudent—for California’s DMV to require Tesla to follow the same rules that apply to every other company testing an automated driving system in the state.

]]> https://robohub.org/ubers-fatal-crash/ Wed, 21 Mar 2018 13:27:48 +0000 http://robohub.org/ubers-fatal-crash/

Credit: Uber

By Bryant Walker Smith

An automated vehicle in Uber’s fleet fatally struck a woman crossing a street in Arizona. A few points pending more information:

1. This sad incident will test whether Uber is becoming a trustworthy company. Uber needs to be unflinchingly candid and unfailingly helpful in the multiple investigations that are likely to result. It shouldn’t even touch its onboard and offboard systems unless credible observers are present. In this crash, a multitude of data will likely be available to help understand what happened—but only if those data can be believed.
2. The circumstances of this crash certainly suggest that something went wrong. Was the vehicle traveling at a speed appropriate for the conditions? Did the automated driving system and the safety driver recognize the victim, predict her path, and respond appropriately? The lawfulness of the victim’s actions is only marginally relevant to the technical performance of Uber’s testing system (which includes both vehicle and driver).
3. Regardless of whether this crash was unavoidable, serious developers and regulators of automated driving systems understand that tragedies will occur. Automated driving is a challenging work in progress that may never be perfected, and I would be skeptical of anyone who claims that automated driving is a panacea—or who expresses shock that it is not.
4. However, this incident was uncomfortably soon in the history of automated driving. In the United States, there’s about one fatality for every 100 million vehicle miles traveled, and automated vehicles are nowhere close to reaching this many real-world miles. This arguably first fatality may not tell us much statistically, but neither is it reassuring.
5. On the same day that this tragic crash happened, about 100 other people died in crashes in the United States alone. Although they won’t make international news, their deaths are also tragedies. And most of them will have died because of human recklessness, including speeding, drinking, aggression, and distraction. This is a public health crisis, and automated driving may play an important role (though by no means the only role) in addressing it. In short: We should remain concerned about automated driving but terrified about conventional driving.
6. Technologies are understood through stories—both good and bad. I don’t know how this tragic story will play out in the fickle public. Surprisingly, Tesla’s fatal 2016 crash doesn’t seem to have dramatically shifted attitudes toward driving technologies. But that was Tesla, and this is Uber. And whereas few people use Autopilot, almost everyone is a pedestrian.
7. The current tragedy includes a long prologue that does not look good. In 2016, Uber refused to comply with California’s automated vehicle law, the state revoked the company’s vehicle registrations, Arizona’s governor tweeted “This is what OVER-regulation looks like! #ditchcalifornia,” and Uber trucked its vehicles down to his state.
8. Developers need to show that they are worthy of the tremendous trust that regulators and the public necessarily place in them. They need to explain what they’re doing, why they believe it is reasonably safe, and why we should believe them. They need to candidly acknowledge their challenges and failures, and they need to readily mitigate the harms caused by those failures. I expand on these principles in a paper (“The Trustworthy Company”) forthcoming at newlypossible.org.

]]> https://robohub.org/what-the-past-of-robotics-law-says-about-its-future-2/ Tue, 15 Mar 2016 11:00:00 +0000 http://robohub.org/what-the-past-of-robotics-law-says-about-its-future-2/

One day, robots will present difficult legal challenges. This seems to be the consensus among commentators. And who am I to disagree? I have myself argued, right here on the digital pages of Slate, that robotics will generate no fewer puzzles for the lawthan the last transformative technology of our time—the Internet. Future courts will have to decide, for instance, whether a home robot manufacturer is responsible for the apps that run on it and whether to hold anyone accountable for robot behavior no one intended or foresaw.

So I’m in agreement with the many scholars, journalists, and others that see interesting times ahead for robotics law and policy. It turns out, however, that there are just as interesting times behind.

Robots have been in American society for half a century. And, like most technologies, they have occasioned legal disputes. A small team of research assistants and I went back and looked at hundreds of cases involving robots in some way or another over the past six years. The cases span a wide variety of legal contexts, including criminal, maritime, tort, immigration, import, tax, and other law. Together they tell a fascinating story about the way courts think about an increasingly important technology. (You can read the full paper, “Robots in American Law,” here.)

In many of the cases I came across, the role of the robot was incidental: The case would likely have come out just the same way were it not a robot at issue. Some of these incidental cases were fascinating. Nannuzzi v. King et al. (1987) involved an injury on a movie set where a robotic lawnmower malfunctioned and injured a cameraman. The film, written and directed by Stephen King, was Maximum Overdrive—a film about machines coming alive and attacking people. Nevertheless, the legal issue presented by a falling stage light would have been basically the same.

In other cases, however, it really seemed to matter that a robot was at issue. In White v. Samsung (1993), for example, a federal appellate court had to decide whether a robot version of Vanna White in a Samsung print ad “represented” the game show hostess for purposes of the right to publicity. The majority thought it did. The dissent was adamant it did not. “One is Vanna White,” said the dissent, “The other is a robot. No one could reasonably confuse the two.” Just a few years later the same court encountered a second case of robots emulating people—in this instance, Cliff and Norm from the television shows Cheers. Judge Alex Kozinski’s eventual dissent from a decision not to rehear the case began with the words, “Robots again.”

In Comptroller of the Treasury v. Family Entertainment Centers (1987), a Maryland court had to decide whether life-size, animatronic puppets that dance and sing at Chuck E. Cheese restaurants trigger a state tax on food “where there is furnished a performance.” The court went on at length about the nature of the term performanceand why a robot could not display the requisite spontaneity. Whereas in Louis Marx & Co. and Gehrig Hoban & Co., Inc. v. United States (1958), a customs court had to decide whether a “mechanical walking robot” being imported represented an animate object (and therefore a doll), which is taxed at a lower rate. The court went on to draw a distinction between a robot—which represents a human—and the toy in question—which only represents a robot.

Read the full piece at Slate. 

Three dimensional printing turns bits into atoms. The technology is simply amazing. These machines draw on programming, art and engineering to enable people to design and build intricate, beautiful, functional jewelry, machine parts, toys and even shoes. In the commercial sector, 3D printing can revolutionize supply chains as well. As the public interest group Public Knowledge wrote once, “It will be awesome if they don’t screw it up.”
The age of publicly accessible 3D printers and printing services is finally here, but are our legal doctrines up to the task of protecting the public while not screwing up a fantastic new tool? Stanford Law Professors Mark Lemley and Nora Freeman Engstrom, the CEO of Shapeways, the founder of Printrbot, the president of Airwolf 3D, and CIS Director of Civil Liberties Jennifer Granick discuss the product liablity and intellectual property issues surrounding this innovation.
Panelists:

Mark Lemley – William H. Neukom Professor of Law at Stanford Law School
Nora Freeman Engstrom – Associate Professor of Law at Stanford Law School
Brook Drumm – Founder of Printrbot, a desktop 3D printer you can build in a couple hours
Erick Wolf – Intellectual Property Attorney and President of Airwolf 3D, an affordable, durable, and easy-to-use 3D printer

Moderator: Jennifer Granick – Director of Civil Liberties at Stanford CIS

Ryan Calo discusses how researchers at Oxford, Geneva, and Berkeley have created a proof of concept for using commercially available brain-computer interfaces to discover private facts about today’s gamers.