Source: commaai

There have been few postings this month, as I took the time to enjoy a holiday in New Zealand around speaking at the SingularityU New Zealand summit in Christchurch. The night before the summit, we enjoyed a 7.8 earthquake not so far from Christchurch, whose downtown was 2/3 demolished after quakes in 2010 and 2011. On the 11th floor of the hotel it was a disturbing nailbiter of swaying back and forth for over 2 minutes — but of course, swaying is what the building is supposed to do, that means it’s working. The shocks were rolling, not violent, and in fact, we got more violent jolts from aftershocks a week later when we went to Picton.

While driving around that region, we encountered this classic earthquake scene on the road:

There were many like this and in fact the main highway of the South Island was destroyed long-term not too far away, cutting off several towns. A scene like this makes you wonder just what a robocar would do in these situations. I already answered this question in a blog post on how to handle a tsunami. Fortunately, there was only a mild tsunami for this quake. A tsunami will result in a warning in the rich world and the car will know the elevation map of the roads and know how to get to high ground. In some places, like Japan, there is also an advanced earthquake warning system that tells you quakes are coming well before they hit since electrons go much faster than seismic waves. With such a system, robocars should receive a warning and come to a stop unless they need to evacuate a tsunami zone. Without such a warning, we still could imagine the road cracking and collapsing in front of you as might have happened on this road. Of course, the cones and signs that warned me days later would not be present.

The answer again lies in the fact that pictures (like mine) will be used to create situations like this in a simulator and car developers will be able to test their systems with simulated quake damage. I’ve spoken since 2010 on the value of a shared simulator environment and I think if government agencies like NHTSA want to really help development, providing funding and tools for such an environment would be a good step. NHTSA’s proposal that all developers share their logs of all incidents would clearly make such a simulator better, but there is pushback because of the proprietary value of those logs. When it comes to strange situations like earthquakes, I doubt there would be much pushback on having an open and shared simulator environment.

New Zealand’s government is taking a very welcoming approach to robocars. They are not regulating for a while and have invited developers to come and test. They have even said it’s OK to test unmanned vehicles under some fairly simple rules. NZ does not have any auto industry and it’s quite remote, but we’ll see if they can attract developers to come test. Their roads feature something you don’t see much in the USA — tons and tons of one-lane bridges and other one-lane stretches of highway. Turns out that robocars, with a little bit of communication, can make efficient use of one-lane two-way roads, so it might be worth exploring.

[Editor’s note: Christchurch will host New Zealand’s first trial of a fully autonomous electric vehicle next year. The trials will largely be conducted on the Christchurch Airport campus, starting on private roads with no public present, with the long-term aim of moving to public roads once the safety case has been made and all regulatory approvals are in place.]

Open Source Comma One box

Speaking of Open, today Comma.ai, which previously had declared they were giving up on their neural network autopilot due to NHTSA threats today announced they have open sourced their software, along with hardware designs and case designs. NHTSA did not want them making an autopilot and said they could not simply rely on the fact that drivers were told they must be diligent, It will be very interesting to see how NHTSA reacts to the release of open designs that anybody can then install on their car.

The automotive industry has had a long history of valuing the tinkerer. All the big car companies had their beginnings with small tinkerers and inventors. Some even died in the very machines they were inventing. These beginnings have allowed people to do all sorts of playing around in their garages with new car ideas, without government oversight, in spite of the risk to themselves and even others on the road. If a mechanic wants to charge you for working on your car they must be licenced, but you are free to work on it yourself with no licence, and even build experimental cars. You just can’t sell them. And even those rights have been eroded.

Clearly far fewer people will have the inclination to build an autopilot using the comma.ai tools by themselves. But it won’t be that hard to do, and they can make it easier with time, too. One could even imagine a car which already had the necessary hardware so that you only needed to download software to make it happen.

In recent times, there has been a strong effort to prevent people with tinkering with their cars, even in software. One common area of controversy has been around engine tuning. Engine tuning is regulated by the EPA to keep emissions low. Car vendors have to show they have done this — and they can’t program their car to give good emissions only on the test while getting better performance off the test as VW did. But owners have been known to want to make such modifications. Now we will see modifications that affect not just emissions but safety. Car companies don’t want to be responsible if you modify the code in your car and there is an accident involving both their code and yours. As such, they will try to secure their car systems so you can’t change them, and the government may help them or even insist on it. When you add computer security risks to the mix — who can certify the modified car can’t be taken over and used as a weapon? — it will get even more fun.

I will also point out that I suspect that comma’s approach would not know what to do about the collapsed road because it would never have been trained in that situation. It might, however, simply sound an alert and kick out, not being able to find the lane anymore.

Regulatory pushback

Regular readers will have seen my strong critique of the NHTSA rules. The other major news during my break was the pushback from major players in the public comment on the regulations. In some ways, the regulations didn’t do enough to give vendors the certainty they need to make their plans. At the same time they were criticized for not giving enough flexibility to vendors. In addition, as expected, they resist giving up their proprietary data in the proposed forced sharing. I predict continued ambivalence on the regulations. Big players actually like having lots of regulations, because big players know how to deal with that and small players don’t.

This post originally appeared on robocars.com.

George “Geohot” Hotz Presents the Comma One at Disrupt SF. Source: TechCrunch/YouTube

Silicon Valley and other technology centers have their share of brilliant minds. Some of them have similarly outstanding egos. A few of those have very short fuses. Such is the story of George Hotz.

Source: YouTube

Hotz is certainly brilliant. He was the first person to unlock the iPhone and hack the PlayStation3. Recently his 2,000-line deep learning code, plus car hacks to capture the car’s digital systems, and his driving model, has learned how and written code to drive a car. He demonstrated this to a Bloomberg reporter earlier this year after which he got some funding, hired a team of engineers and turned that software into an inexpensive product – the green thing in his hand – which he promoted at TechCrunch Disrupt SF in September. The green device, the Comma One, is a $999 aftermarket add-on kit with a $24 monthly subscription fee for select Honda and Acura cars. The device replaces the rear-view mirror and captures the car’s radar, steering, braking system and gas pedal. It also adds a camera.

Hotz said that there were 750 of these devices at work in Beta test vehicles around the area, state and country.

The TechCrunch video is in two parts: Hotz’s 7-minute product presentation and an in-depth Q&A with Darrell Etherington which is also informative and worth watching.

Part of Hotz’s talk was a review and critique of other companies making self-driving kits and self-driving cars. “Shippability” was his criteria and by that token, only Tesla has shipped and delivered a self-driving car. He was particularly hostile to the Cruise Automation people saying that they had sold out their original intention to build and sell an aftermarket kit. [For another spin on Cruise, see below.] The most politically correct thing I can say about that portion of his presentation was that his remarks were youthful.

Enter the NHTSA (National Highway Transportation Safety Agency) with a strongly worded query and special order about the safety of Comma One.

“Recently, NHTSA has become aware that your company intends to sell a product, the “comma one,” that you claim allows certain Honda vehicles (and potentially certain Acura vehicles) to operate in a semi-autonomous mode. In a blog post dated October 20, 2016, you described your product as an “advanced driver assist system,” or “in traditional auto manufacturer terms: as “lane keeping assist” and “adaptive cruise control.” You assert that the product “should be legal.”

“This “aftermarket upgrade,” as you describe it, is replacement motor vehicle equipment and your company is a manufacturer of motor vehicle equipment subject to the requirements of the National Traffic and Motor Vehicle Safety Act (Safety Act), 49 U.S.C. Chapter 301, under NHTSA’s oversight. It is essential that you are aware of and in full compliance with your legal responsibilities to ensure vehicle safety before introducing this product into commerce.”

The NHTSA Special Order (SO) said that Comma.ai must respond under oath and accompanied by an affidavit, to 15 “requests,” or be fined up to $21,000 per day and/or become the subject of a civil action to compel responses. The SO asked Hotz to describe, in detail, how the Comma One is installed, what features it offers, how the driver can use the product, what conditions constitute safe operations of the device, what types of operator controls, roads, areas, speeds and environmental conditions such as weather or time of day may a car equipped with the Comma One operate safely and what testing procedures have been implemented to attest to those claims. Each of the questions requires a detailed description of the basis to determine safe operation conditions.  Finally, by replacing the rearview mirror, how will the Comma One block or obstruct the drivers’ ability to have sufficient rear view.

[I don’t know if the NHTSA’s letter and SO are particularly harsh but, certainly, their concern for safety is evident as it should be; and as it should be for Hotz and the Comma.ai team. Governments and their regulations are an integral part of the process of bringing a new product like the Comma One to market. Certainly, it was in their domain to request testing and safety methods from Comma. But Comma’s response was definitely not predictable.]

A day later Hotz Tweeted the bottom three messages you see (read from bottom to top). Thus Hotz cancelled Comma’s first official product even though an admirer says it was the best autonomous driving experience (top Tweet).

Hotz said that Comma would turn its attention to “other products and markets.” It’s unclear what those new products will be. But it is clear that this is a major change in direction at Comma, perhaps even the beginning of its demise.

There’s a saying that at some point every startup needs to bring in an adult to handle the difficult, distasteful, and often diplomatic tasks that happen in business and with strong-willed inventors.

Perhaps now would be a good time for Comma.ai to hire that adult (although it might be a little late).

Cruise Automation

For those of you who thought the billion dollar acquisition of Cruise was a fluke, read on. One billion dollars! That’s what GM paid earlier this year. $300 million in cash; $300 million in GM stock and the remainder in incentives to retain key employees.

Up until news about the acquisition, Cruise was known as the developer of an aftermarket kit to turn select cars into self-driving cars. Actually Cruise had abandoned the kits almost a year earlier to build software instead. That’s what attracted GM.

A new article in Fortune by Erin Griffith sheds some insight and details into Cruise, its two founders, Kyle Vogt and Daniel Kan, and its importance to GM.

“Since the deal closed, Cruise has more than doubled its staff. The company’s office is filling up quickly. Office space in San Francisco is already scarce and pricey, and Cruise has a unique challenge: It needs to drive cars in and out of the office. Beneath Cruise’s barnlike warehouse is a fleet of Chevy Bolts, each outfitted with LiDAR laser sensors. There’s only one ramp in and out of the basement. The cars are starting to feel crowded too.

Cruise could solve its problem by moving its operations to the suburban sprawl of Silicon Valley proper. Like all the automakers, GM has an innovation outpost in Palo Alto. But no one at Cruise has even visited it. Staying independent is too important.”

Read the complete Fortune article here.

If you liked this article, you may also want to read these articles about autonomous cars:

• Comma.ai cancels comma-one add-on box after threats from NHTSA
• What to expect from autonomous cars
• How much can customers test robocars?
• The inevitability of the global march toward self-driving vehicles
• Two New Year’s resolutions for developers of automated vehicles
• comma.ai’s neural network car and new technology in robocars

See all the latest robotics news on Robohub, or sign up for our weekly newsletter

Source: commaai

Comma.ai, the brash startup attempting to make a self-driving system entirely from a neural network has announced it will cancel the “comma one” add-on box it has planned to sell to owners of certain Honda vehicles. The box stuck on the rear-view mirror and used the car’s own bus commands to provide an autopilot similar to those offered by car makers, with lane-keeping and adaptive cruise control.

Of particular importance is the letter from NHTSA to comma.ai which I suggest you read. This letter creates several big issues:

1. There are many elements of this letter that would also apply to Tesla and other automakers which have built supervised autopilot functions.
2. Of particular interest is the paragraph that says: “it is insufficient to assert, as you do, that the product does not remove any of the driver’s responsibilities” and “there is a high likelihood that some drivers will use your product in a manner that exceeds its intended purpose.” That must be very scary for Tesla.
3. I noted before that the new NHTSA regulations appear to forbid the use of “black box” neural network approaches to the car’s path planning and decision making. I wondered if this made illegal the approach being done by Comma, NVIDIA and many other labs and players. This may suggest that.
4. We now have a taste of the new regulatory regime and had it existed before systems like Tesla’s Autopilot, Mercedes Traffic Jam Assist, and Cruise’s original aftermarket autopilot would never have been able to get off the ground.
5. George Hotz of comma declares, “Would much rather spend my life building amazing tech than dealing with regulators and lawyers. It isn’t worth it. The comma one is cancelled. comma.ai will be exploring other products and markets. Hello from Shenzhen, China.”

To be clear, comma is a tiny company taking a radical approach, so it is not a given that what NHTSA has applied to them would have been or will be unanswerable by the big guys. Because Tesla’s Autopilot is not a pure machine learning system, they can answer many of the questions in the NHTSA letter that comma can’t. They can do much more extensive testing that a tiny startup can’t. But even so, a letter like this sends a huge chill through the industry.

It should also be noted that in Comma’s photos the box replaced the rear-view mirror, and NHTSA had reason to ask about that.

Source: commaai

George’s declaration that he’s in Shenzen gives us the first sign of the new regulatory regime pushing innovation away from the United States and California. I will presume the regulators will say, “We only want to scare away dangerous innovation” but the hard truth is that is a very difficult thing to judge. All innovation in this space is going to be a bit dangerous. It’s all there trying to take the car — the 2nd most dangerous legal consumer product — and make it safer, but it starts from a place of danger. We are not going to get to safety without taking risks along the way.

I sometimes ask, “Why do we let 16-year-olds drive?” They are clearly a major danger to themselves and others. Driver testing is grossly inadequate. They are not adults so they don’t have the legal rights of adults. We let them drive because they are going to start out dangerous and then get better. It is the only practical way for them to get better, and we all went through it. Today’s early companies are teenagers. They are going to take risks. But this is the fastest and only practical way to let them get better and save millions.

“…some drivers will use your product in a manner that exceeds its intended purpose”

This sentence, though in the cover letter and not the actual legal demand, looks at the question asked so much after the Tesla fatal crash. The question which caused Consumer Reports to ask Tesla to turn off the feature. The question which caused MobilEye, they say, to sever their relationship with Tesla.

The paradox of the Autopilot is this: The better it gets, the more likely it is to make drivers over-depend on it. The more likely they will get complacent and look away from the road. And thus, the more likely you will see a horrible crash like the Tesla fatality. How do you deal with a system which adds more danger the better you make it? Customers don’t want annoying countermeasures. This may be another reason that “Level 2,” as I wrote yesterday is not really a meaningful thing.

NHTSA has put a line in the sand. It is no longer going to be enough to say that drivers are told to still pay attention.

Black box

Comma is not the only company trying to build a system with pure neural networks doing the actual steering decisions (known as “path planning”.) NVIDIA’s teams have been actively working on this, as have several others. They plan to make commentary to NHTSA about these elements of the regulations, which should not be forbidding this approach until we know it to be dangerous.

It is challenging. It’s hard to do QA on neural networks. You can examine any single state of them, but not really understand them. You can fix the errors they make, but not know how you fixed it or whether your fix is going to work in other cases. On their own that sounds too scary, but the problem is they are outperforming other algorithms at many of the problems they are being applied to.

If you have two systems:

1. A conventionally coded system which you fully understand, that has one safety incident per 100,000 miles,
2. A black box machine learning system that has one safety incident per 150,000 miles, but you have minimal understanding of.

Which is the one that is better to put on the road? It’s not a no-brainer.

Scaring people away

George Hotz is about as brash as they come, so some might discount his bluster about moving his projects to China. But this is a real thing. When regulations come, they tend to favour the big players. The big players have large QA teams and compliance departments. They can afford them, and they move conservatively because of them. Big players help write the regulations to favour their way of doing things — sometimes selfishly but sometimes just because it is the only thing they know.

The USA and California led the way in robocars in part because it was unregulated. In the USA, everything is permitted unless it was explicitly forbidden and nobody thought to write “no robots” in the laws. Progress in other countries where everything is forbidden unless it is permitted was much slower. The USA is moving in the wrong direction.

Countries like China are another animal altogether. There, everything is forbidden unless a smart autocrat allows it but once the smart (or stupid) ruling official makes it happen, the other red tape can vanish. It is not the best system overall, but it can win at things they put their minds to.

Tesla

Has Tesla received such a letter? What would they have done if they received it in their early days? While Tesla and Elon Musk love the Autopilot project, the hard truth is Tesla’s biggest problem is going to be making enough Model 3s to meet demand and doing it profitably. Taking out Autopilot isn’t going to cut their sales at all — they have too many sales and not enough cars. Only their forward vision and guts will keep them going if NHTSA does the same thing to them. I do believe Tesla can answer questions like those in the demand letter more easily than comma.ai. But not trivially, and they almost surely will need to bump the countermeasures.