In episode four of season three Neil introduces us to the ideas behind the bias variance dilemma (and how how we can think about it in our daily lives). Plus, we answer a listener question about how to make sure your neural networks don’t get fooled. Our guest for this episode is Jeff Dean,  Google Senior Fellow in the Research Group, where he leads the Google Brain project. We talk about a closet full of robot arms (the arm farm!), image recognition for diabetic retinopathy, and equality in data and the community.

Fun Fact: Geoff Hinton’s distant relative invented the word tesseract. (How cool is that. Seriously.)

• Talking Machines: Overfitting and asking ecological questions, with Tom Dietterich
• Talking Machines: Restricted Boltzmann Machines, with Eric Lander
• Talking Machines: Automatic Translation and t-SNE, with Hal Daume
• Talking Machines: Generative art and Hamiltonian Monte Carlo, with Doug Eck
• Talking Machines: Machine learning and the Flint water crisis, with Jake Abernethy
• Talking Machines: Gaussian processes and OpenAI, with IIya Sutskever

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

It is unclear if Masayoshi Son, Chairman of Softbank, was one of the 17 million YouTube viewers of Boston Dynamic’s Big Dog before acquiring the company for an undisclosed amount this past Thursday. What is clear is the acquisition of Boston Dynamics by Softbank is a big deal. Softbank’s humanoid robot Pepper is trading up her dainty wheels for a pair of sturdy legs.

In expressing his excitement for the acquisition, Masayoshi Son said, “Today, there are many issues we still cannot solve by ourselves with human capabilities. Smart robotics are going to be a key driver of the next stage of the Information Revolution, and Marc and his team at Boston Dynamics are the clear technology leaders in advanced dynamic robots. I am thrilled to welcome them to the SoftBank family and look forward to supporting them as they continue to advance the field of robotics and explore applications that can help make life easier, safer and more fulfilling.”

Marc Raibert, CEO of Boston Dynamics, previously sold his company to Google in 2013. Following the departure of Andy Rubin from Google, the internet company expressed buyers remorse. Raibert’s company failed to advance from being a military contractor to a commercial enterprise. It became very challenging incorporating Boston Dynamic’s zoo of robots (mechanical dogs, cheetahs, bulls, mules and complex humanoids) into Google’s autonomous strategy. Since Rubin’s exit in 2014, rumors of buyers acquiring Boston Dynamics from Google have ranged from Toyota Research to Amazon Robotics. Softbank represents a new chapter for Raibert, and possible the entire industry.

Raibert’s statement to the press gave astute readers a peek of what to expect: “We at Boston Dynamics are excited to be part of SoftBank’s bold vision and its position creating the next technology revolution, and we share SoftBank’s belief that advances in technology should be for the benefit of humanity. We look forward to working with SoftBank in our mission to push the boundaries of what advanced robots can do and to create useful applications in a smarter and more connected world.” A quick study of the assets of both companies reveals how Boston Dynamics could help Softbank in its mission to build robots to benefit humanity.

Softbank’s premier robot is Pepper, a four-foot tall social robot that has been mostly deployed in Asia as a customer service agent. Recently, Pepper, as part of Softbank’s commitment to the Trump administration to invest $50 billion in the United States has been spotted in stores in California. As an example, Pepper proved itself as a valuable asset last year to Palo Alto’s premier tech retailer B8ta, accounting for a 6 times increase in sales. To date, there are close to 10,000 Pepper robots deployed worldwide, mostly in Asian retail stores. However, Softbank is also the owner of Sprint with 4,500 cell phone stores across the USA, and a major investor in WeWork with 140 locations globally servicing 100,000 members – could Pepper be the customer service agent or receptionist of the future?

According to Softbank’s website, Pepper is designed to be a “day-to-day companion,” with its most compelling feature being the ability to perceive emotions. Softbank boasts that their humanoid is the first robot ever to recognize moods of homo sapiens and adapt its behavior accordingly. While this is extremely relevant for selling techniques, Softbank is most proud of Pepper being the first robot to be adopted into homes in Japan. It is believed that Pepper is more than a point-of-purchase display gimmick, but an example of the next generation of caregivers for the rising elderly populations in Japan and the United States. According to the Daily Good, “Pepper could do wonders for the mental engagement and continual monitoring of those in need.” Its under $2,000 price point also provides an attractive incentive to introduce the robot into new environments, however wheel-based systems are a limitation in the home with clutter floors, stairs and other unforeseen obstacles.

Boston Dynamics is almost the complete opposite of Softbank; it is a research group spun out of MIT. Its expertise is not in social robots but in military “proofs of concepts” like futuristic combat mules. The company has touted some of the most frightening mechanical beasts to ever walk the planet from metal Cheetahs that sprint at over 25 miles per hour to mechanized dogs that scale mountains with ease to one of the largest humanoids every built that has an uncanny resemblance to Cyberdyne’s T-800. In a step towards commercialization, Boston Dynamics released earlier this year its newest monster – a wheel-biped leg robot named Handle that can easily lift over a hundred pounds and jump higher than Lebron James. Many analysts pontificated that this appeared to be Boston Dynamics attempt to prove its relevance to Google with a possible last mile delivery bot.

In an IEEE interview when Handle debuted last February, Raibert exclaimed, “Wheels are a great invention. But wheels work best on flat surfaces and legs can go anywhere. By combining wheels and legs, Handle can have the best of both worlds.” IEEE writer Evan Ackerman questioned, after seeing Handle, if the next generation of Boston Dynamic’s humanoids could feature legs with roller-skate like shoes. One is certain that Boston Dynamics is the undisputed leader of dynamic control and balance systems for complex mechanical designs.

Leading roboticist Dan Kara of ABI Research confirmed that “these [Boston Dynamics] are the world’s greatest experts on legged mobility.”

If walking is the expertise of Raibert’s team and Softbank is the leader of cognitive robotics with a seemingly endless supply of capital, the combination could be the first real striding humanoid capable of human-like emotions. By 2030 there will be 70 million people over the age of 65 years in America, with a considerable smaller amount of caregivers. To answer this call researchers are already converting current versions of Pepper into sophisticated robotic assistants. Last year, Rice University unveiled a “Multi-Purpose Eldercare Robot Assistant (MERA)” which is essentially a customized version of the Softbank’s robot. MERA is specifically designed to be a home companion for seniors that “records and analyzes videos of a person’s face and calculates vital signs such as heart and breathing rates.” Rice University partnered with IBM’s Aging-in-Place Research Lab to create MERA’s speech technology. IBM’s Lab founder, Susann Keohane, explained that Pepper “has everything bundled into one adorable self.” Now with Boston Dynamic’s legs Pepper could be a friend, physical therapist, and life coach walking side by side with its human companion.

Daniel Theobald, founder of Vecna Technologies – a healthcare robotic company, summed it best last week, “I think Softbank has made a major commitment to the future of robotics. They understand that the world economy is going to be driven by robotics more and more.”

Next Tuesday we will dive further into the implications of Softbank’s purchase of Boston Dynamics with Dr. Howard Morgan/First Round Capital, Tom Ryden/MassRobotics and Dr. Eric Daimler/Obama White House at RobotLabNYC’s event on 6/13 @ 6pm WeWork Grand Central (RSVP).

Spotmini by Boston Dynamics. Source: Boston Dynamics/YouTube

In a long-awaited transaction, The New York Times Dealbook announced that SoftBank was buying Boston Dynamics from Alphabet (Google). Also included in the deal is the Japanese startup Schaft. Acquisition details were not disclosed.

Both Boston Dynamics and Schaft were acquired by Google when Andy Rubin was developing Google’s robot group through a series of acquisitions. Both companies have continued to develop innovative mobile robots. And both have been on Google’s for sale list.

Boston Dynamics, a DARPA and DoD-funded 25 year old company, designed two and four-legged robots for the military. Videos of BD’s robots WildCat, Big Dog, Cheetah, SpotMini (shown above getting into an elevator) and Handle, have been YouTube hits for years. Handle, BD’s most recent is a two-wheeled, four-legged hybrid robot that can stand, walk, hop, run and roll at up to 9 MPH.

Schaft, a Japanese startup/participant in the DARPA Robotic Challenge, recently unveiled a two-legged robot that climbed stairs, can carry 125 pounds of payload, move in tight spaces and keep its balance throughout.

SoftBank, through another acquisition (of French Aldabaran, the maker of the Nao and Romeo robots), and in a joint venture with Foxconn and Alibaba, has developed and marketed thousands of Pepper robots. Pepper is a cute, humanoid, mobile robot being marketed and used as a guide and sales assistant. The addition of Boston Dynamics and Schaft to the SoftBank stable add talent and technology to their growing robotics efforts, particularly the Tokyo-based Schaft.

Today, there are many issues we still cannot solve by ourselves with human capabilities. Smart robotics are going to be a key driver of the next stage of the information revolution,” said Masayoshi Son, chairman and chief executive of SoftBank.

https://www.youtube.com/watch?v=0x8Bc0z3Ixw

California published its summary of all the reports submitted by vendors testing robocars in the state. You can read the individual reports. They are interesting, but several other outlines have created summaries of the reports calculating things like the number of interventions per mile. On these numbers, Google’s lead is extreme. Of over 600,000 autonomous miles driven by the various teams, Google/Waymo was 97% of them — in other words, 30 times as much as everybody else put together.

Beyond that, their rate of miles between disengagements (around 5,000 — a 4x improvement over 2015) is one or two orders of magnitude better than the others, and in fact for most of the others, they have so few miles that you can’t even produce a meaningful number. Only Cruise, Nissan and Delphi can claim enough miles to really tell.

Tesla is a notable entry. In 2015 they reported driving zero miles, and in 2016 they did report a very small number of miles with tons of disengagements from software failures (one very 3 miles.) That’s because Tesla’s autopilot is not a robocar system, and so miles driven by it are not counted. Tesla’s numbers must come from small scale tests of a more experimental vehicle. This is very much not in line with Tesla’s claim that it will release full autonomy features for their cars fairly soon, and that they already have all the hardware needed for that to happen.

Unfortunately, you can’t easily compare these numbers:

• Some companies are doing most of their testing on test tracks, and they do not need to report what happens there.
• Companies have taken different interpretations of what needs to be reported. Most of Cruise’s disengagements are listed as “planned” but in theory those should not be listed in these reports. But they also don’t list the unplanned ones which should be there.
• Delphi lists real causes and Nissan is very detailed as well. Others are less so.
• Many teams test outside California, or even do most of their testing there. Waymo/Google actually tests a bunch outside California, making their numbers even bigger.
• Cars drive all sorts of different roads. Urban streets with pedestrians are much harder than highway miles. The reports do list something about conditions but it takes a lot to compare apples to apples. (Apple is not one of the companies filing a report, BTW.)

One complication is that typically safety drivers are told to disengage if they have any doubts. It thus varies from driver to driver and company to company what “doubts” are and how to deal with them.

Google has said their approach is to test any disengagement in simulator, to find out what probably would have happened if the driver did not disengage. If there would have been a “contact” (accident) then Google considers that a real incident and those are rarer than is reported here. Many of the disengagements are when software detects faults with software or sensors. There, we do indeed have a problem, but like human beings who zone out, not all such failures will cause accidents or even safety issues. You want to get rid of all of them, to be sure, but if you are are trying to compare the safety of the systems to humans, it’s not easy to do.

It’s hard to figure out a good way to get comparable numbers from all teams. The new federal guidelines, while mostly terrible, contain an interesting rule that teams must provide their sensor logs for any incident. This will allow independent parties to compare incidents in a meaningful way, and possibly even run them all in simulator at some level.

It would be worthwhile for every team to be required to report incidents that would have caused accidents. That requires a good simulator, however, and it’s hard for the law to demand this of everybody.

Source: Waymo

Google’s car project (known as “Chauffeur”) kickstarted the entire robocar revolution, and Google has put in more work, for longer, than anybody. The car was also the first project of what became Google “X” (or just “X” today under Alphabet. Inside X, a lab devoted to big audacious “moonshot” projects that affect the physical world as well as the digital, they have promoted the idea that projects should eventually “graduate,” moving from being research to real commercial efforts.

Alphabet has announced that the project will be its own subsidiary company with the new name “Waymo.” The name is not the news; what’s important is the move away from being a unit of a mega-company like Google or Alphabet. The freedoms to act that come with being a start-up (though a fairly large and well funded one) are greater than units in large corporations have. Contrast what Uber was able to do, skirting and even violating the law until it got the law changed, with what big corporations need to do.

Google also released information about how in 2015 they took Steve Mahan — the blind man who was also the first non-employee to try out a car for running errands — for the first non-employee (blind or otherwise) fully self-driving ride on public streets, in a vehicle with no steering wheel and no backup safety driver in the vehicle. (This may be an effort to counter large amount of press about public ride offerings by nuTonomy in Singapore and Uber in Pittsburgh, as well as truck deliveries by Uber/Otto in 2016.)

It took Google/Alphabet 6 years to let somebody ride on public streets in part because it is a big company. It’s an interesting contrast with how Otto did a demonstration video after just a few months of life of a truck driving a Nevada highway with nobody behind the wheel (but Otto employees inside and around it.) That’s the sort of radical step that startups do.

Waymo declared their next goal is to “let people use our vehicles to do everyday things like run errands, commute to work, or get safely home after a night on the town.” This is the brass ring, a “Mobility on Demand” service, which is able to pick people up (ie. run unmanned) and even carry a drunk person.

The last point is important. To carry a drunk is a particular challenge. In terms of improving road safety, it’s one of the most worthwhile things we could do with self-driving cars since drunks have so many of the accidents. To carry a drunk, you can’t let the human take control even if they want to. Unlike unmanned operation, you must travel at the speed impatient humans demand, and you must protect the precious cargo. To make things worse, in some legal jurisdictions, they still want to consider the person inside the car the “driver,” which could mean that since the “driver” is impaired, operation is illegal.

Waymo as leader

The importance of this project is hard to overstate. While most car companies had small backburner projects related to self-driving going back many years, and a number of worthwhile research milestones were conquered in the 90s and even earlier, the Google/Waymo project, which sprang from the Darpa Grand Challenge, energized everybody. Tiny projects at car companies all got internal funding because car companies couldn’t tolerate the press and the world thinking and writing that true future of the car was coming from a non-car company, a search engine company. Now the car companies have divisions with thousands of engineers and it’s thanks to Google. The Google/Waymo team was accomplishing tasks 5 years ago that most projects are only now just getting to, especially in non-highway driving. They were rejecting avenues (like driving with a human on standby ready to take the wheel on short notice) in 2013 that many projects are still trying to figure out.

Indeed, even in 2010, when I first joined the project and it had just over a dozen people, it had already accomplished more complex tasks that most projects, even the Tesla Autopilot that some people think is in the lead, have yet to accomplish.

Let’s see where Waymo goes.

The maker of advanced humanoid and 4-legged robots (BigDog and Atlas), Boston Dynamics, is up for sale. According to the Financial Times, “this is one of those discordant moments that makes you question easy assumptions… and the current state of Alphabet and of robots.”

Google acquired Boston Dynamics in late 2013 as part of Andy Rubin’s 9-company shopping spree. At the time the company had 80 employees and many millions of dollars of research contracts from DARPA and the DoD. Although no financial information was provided at the time, various sources have estimated that Google paid $60 million plus extensive stock options. A more recent guesstimate from The Dallas Business Journal was $500 million.

Andy Rubin left the company in October 2014. After this the robot initiative was plagued by leadership changes, failed collaborations and an unsuccessful search for a new leader. Fomenting this lack of cohesion were Boston Dynamics executives who were reluctant to work with Google’s other robot engineers, according to a person familiar with the group.

The tensions spilled into recorded meetings and emails published on a Google worker forum. The documents were made available to Bloomberg News, which reported that Aaron Edsinger, director of robotics at Google in San Francisco, said he had been trying to create a low-cost electric quadruped robot with Boston Dynamics, but there was “a bit of a brick wall around the division.”

This recently released Boston Dynamics video of a humanoid robot trudging through the snow and being pushed over has been seen by almost 15 million viewers, but not enough to convince Google that it would generate real revenue. Consequently Alphabet, Google’s parent company, has decided to sell Boston Dynamics “as it is not likely to produce a marketable product in the next few years,” Bloomberg News reports, citing two people familiar with the plans. Possible buyers include Toyota’s research institute and Amazon, which makes robots for its fulfillment centers, according to the Bloomberg report.

Earlier this year The New York Times reported that “Google’s robotics division has been plagued by low morale and a lack of leadership since the unit’s founder left abruptly in 2014.”

Alphabet, the new holding company that separated Google from its collection of speculative projects, has reframed the robots effort, moving it from a stand-alone division inside Google to a piece of the X research division. The company has also hired Hans Peter Brondmo, a technology industry veteran who last worked at Nokia, to help with management. X is using some element of robotics in numerous projects like self-driving cars; Project Wing, an effort to deliver packages with drone aircraft; and Makani, an effort to capture wind energy with high-flying kites.

The Financial Times story also said:

This is also a moment for reality in robotics. The past two years have brought an unending stream of predictions about the coming “robo-calypse”. Just about the only thing left to discuss has been whether the robots will put us all out of work first, or whether a malignant AI will wipe us out.

As people working in the field are growing tired of explaining, the current state of the technology — though advancing fast — is a long way from these science fiction scare stories. The uncertain fate of Boston Dynamics underlines that truth. Even Google didn’t have the patience to wait for a walking robot to step out of the lab and earn itself a living.

Toyota Executive Technical Advisor and Chief Executive Officer of Toyota Research Institute (TRI) Dr. Gill Pratt speaks during a press conference at the Consumer Electronics Show (CES) 2016 in Las Vegas, Jan. 5, 2016. Photo: Kathryn Rapier

At a press conference last week at CES, Toyota, led by Gill Pratt of DARPA fame and now CEO of Toyota Research Institute (TRI), informed the world what Toyota intended to do with it’s billion dollar investment in their new AI research facilities in Silicon Valley and Cambridge.

“Up to now, our industry has measured on-road reliability of autonomous vehicles in the millions of miles, which is impressive. But to achieve full autonomy, we actually need reliability that is a million times better.  We need trillion-mile reliability. Toyota has traditionally been a hardware company because in the past the most important technology for enhancing human mobility was hardware. Times have changed, and software and data are now essential components of Toyota’s future mobility strategy.”

“It is entirely possible that robots will become for today’s Toyota what the car industry was when Toyota made looms. Home robots may be even more personally prized in our future than cars have been in our past. Toyota’s goal is to move people across the room, across town, and across the country.”

“Our initial technical team directing research will include James Kuffner, CMU professor and former head of Google Robotics, who will be our area lead in cloud computing… [and also six others].”

The portion of Pratt’s announcement that James Kuffner had left Google, and was joining Toyota and Pratt, was newsworthy in the robotics world. It showcased the changes in leadership stemming from the abrupt earlier departure from Google of Andy Rubin, the original leader who left after a buying spree that brought 8 robotic startups into Google. James Kuffner became the acting CEO of the Replicant robotics group, which recently located within the Google X Lab. Conor Dougherty, in a NY Times article, wrote:

Google’s robotics division has been plagued by low morale and a lack of leadership since the unit’s founder left abruptly in 2014. Now Alphabet is cleaning it up. Over the last two months, Alphabet, the new holding company that separated Google from its collection of speculative projects, has reframed the robots effort, moving it from a stand-alone division inside Google to a piece of the X research division. The company has also hired Hans Peter Brondmo, a technology industry veteran who last worked at Nokia, to help with management.

X, which is run by Astro Teller, is using some element of robotics in numerous projects like: self-driving cars; Project Wing, an effort to deliver packages with drone aircraft; and Makani, an effort to capture wind energy with high-flying kites.

Brondmo joins Jonathan Rosenberg, who is a troubleshooter for Larry Page (the Google co-founder who is Alphabet’s chief executive) and Aaron Edsinger (previously the CEO of both Meka and Redwood Robotics) in the day-to-day management of the robotics group.

The Rubin Google acquisitions include: Schaft, a Japanese start-up developing a humanoid robot; Industrial Perception, a Silicon Valley start-up that developed a computer vision system for loading and unloading trucks; Meka Robotics, a robot developer for academia; Redwood Robotics, a start-up intended to compete with the Baxter robot (and others) entering the small and medium-sized shop and factory marketplace; Bot & Dolly, a maker of robotic camera systems used for special effects such as in the movie “Gravity”; Autofuss, a design and marketing firm and a partner in Bot & Dolly; Holomni, a maker of powered caster modules for omnidirectional vehicles; and Boston Dynamics, an engineering company specializing in building robots and software for human simulation.

image: Steve Jurvetson/flickr

Security researcher Jonathan Petit claims to be able to hack the multi-thousand-dollar laser ranging – LIDAR – systems that self-driving cars rely on. How does he do it and is it as dangerous as it sounds?

Much has been made of Petit’s recent disclosure of an attack on some of the LIDAR systems used in robocars. I saw his presentation on this in July, but he asked me for confidentiality until they released their paper in October. However, since he has decided to disclose it, there’s been a lot of press, with truth and misconceptions.

There are many security aspects to robocars. By far the greatest concern would be compromise of the control computers by malicious software, and great efforts will be taken to prevent that. Many of those efforts will involve having the cars not talk to any untrusted sources of code or data that might be malicious. The car’s sensors, however, must take in information from outside the vehicle, so they are another source of concern.

There are ways to compromise many of the sensors on a robocar. GPS can be easily spoofed and there are tools out there to do that (fortunately real robocars will only use GPS as one source to gauge their location). Radar is also very easy to compromise — far easier than LIDAR, agrees Petit — but his team’s goal was to see if LIDAR is vulnerable.

The attack was a real one, but not a particularly frightening one, in spite of the press it received. This type of attack may cause a well designed vehicle to believe there are “ghost” objects that don’t actually exist, so that it might brake for something that’s not there or even swerve around it. It might also overwhelm the sensor, so that it feels the sensor has failed and thus the car would go into failure mode, stopping or pulling off the road. This is not a good thing, of course, and it has some safety consequences, but it’s also fairly unlikely to happen. Essentially, there are far easier ways to do these things that don’t involve LIDAR, so it’s not too likely anybody would want to mount such an attack.

Indeed, to carry out this type of attack you need to be physically present either in front of the car (or perhaps to the side of the road) and you need a solid object that’s already in front of the car, such as the back of a truck that it’s following. This is a higher bar than with an attack that might be done remotely (such as a computer intrusion) or via a radio signal (such as with hypothetical vehicle-to-vehicle radio, should cars decide to use that tech).

Here’s how it works: LIDAR sends out a very short pulse of laser light and then waits for it to reflect back. The pulse is a small dot and the reflection is seen through a lens aimed tightly at the place it was sent to. The time it takes for the light to come back tells you how far away the target is and the brightness tells you how reflective it is, like a black-and-white photo.

To fool LIDAR, you must send another pulse that comes from, or appears to come from, the target spot, and it has to come in at just the right time, before the real pulse from what’s really in front of the LIDAR does.

The attack requires knowing the characteristics of the target LIDAR very well. You must know exactly when it is going to send its pulses before it does so, and thus precisely (to the nanosecond) when a return reflection will arrive from a hypothetical object in front of it. Many LIDARs are quite predictable. They scan a scene with a rotating drum and you can see the pulses coming out and know when they will be sent.

In the simplest version of the attack, the LIDAR is scanning something like a wall in front of it. There aren’t such walls on a highway, but there are things like signs, bridges and the backs of trucks and some cars.

The attack laser sends a pulse of light at the wall or other object, but does it so that it will hit the wall in the right place, earlier than the real pulse from the target LIDAR. This pulse will then bounce back, go in the lens of the LIDAR and make it appear that there is something closer than the wall.

The attack pulse does not have to be bright, but it needs to be similar to the pulse from the LIDAR so that the reflection looks the same. The attack pulse must also be at the very same wavelength and launched at just the right time, to the nanosecond.

If you send out lots of pulses without good timing, you won’t create a fake object, but you can make noise. That noise would blind the LIDAR about the wall in front of it, but it would be very obviously noise. Petit and crew did tests of this type of noise attack, and they were a success.

The fancier type of attack knows the timing of the target LIDAR perfectly and paints a careful series of pulses so it looks like a complex object is present, closer than the wall. Petit’s team were able to do this on a small scale.

This kind of attack can only make the ghost object appear in front of another object, like another vehicle or perhaps a road sign or bridge. It could also be reflected off the road itself. The ghost object, being closer than the road in question, would appear to be higher than the road surface.

There are LIDAR designs which favour the strongest or the last pulse, or even report multiple ones. The latter would probably be able to detect a spoof.

Petit also described an attack from in front of the target LIDAR, such as in a vehicle further along the road. Such an attack, if I recall correctly, was not actually produced but is possible in theory. In this attack, you would shine a laser directly at the LIDAR. This is a much brighter pulse and, in theory, it might shine on the LIDAR’s lens from an angle, but be bright enough to generate reflections inside it that would be mistaken for the otherwise much dimmer return pulse (similar to the bright spots known as “lens flare” in a typical camera shooting close to the sun.) In this case, you could create spots where there is nothing behind the ghost object, though it is unknown just how far an angle from the attack laser (which will appear as a real object) needs to be. The flare pulse would have to be aimed quite precisely to hit the lens, if you want a complex object.

image: Steve Jurvetson/flickr

As noted, all this does is create a ghost object, or noise, that temporarily blinds the LIDAR. The researchers also demonstrated a scarier (but less real) “denial of service” attack against the LIDAR processing software. This software’s job is to take the cloud of 3D points returned by the LIDAR and turn it into a list of objects perceived by it. Cars like Google’s, and most others, don’t use the standard software from the manufacturer.

The software they tested here was limited, and could only track a fixed number of objects – around 64, I believe. If there are 65 objects (which is actually quite a lot) it will miss one. This is a far more serious error, and is called a ‘false negative’ — there might be something right in front of you and your LIDAR would see it but the software would discard it and so it would be invisible to you. Thus, you might hit it, if things go badly.

The attack involves creating lots of ghost objects — so many that they overload the software and go over the limit. The researchers demonstrated this but, again, only on fairly basic software that was not designed to be robust. Even software that can only track a fixed number of objects should detect when it has reached its limit and report that, so the system can mark the result as faulty and take appropriate action.

As noted, most attacks, even with an overwhelming number of ghost objects, would only cause the car to decide to brake for no reason. This is not that hard to do. Humans do this all the time actually, braking by mistake or for lightweight objects like tumbleweeds or animals dashing over the road. You can make a perfectly functioning robocar brake by throwing a ball on the road. You can also blind humans temporarily with a bright flash of light or a cloud of dust or water. This attack is much more high tech and complex but amounts to the same thing.

It is risky to suddenly brake on the road, as the car behind you may be following too closely and hit you. This happens even if you brake for something real. Swerving is a bit more dangerous, but can normally only be done when there is a very high likelihood the path being swerved into is clear. Still, it’s always a good idea to avoid swerving. But again, you would also do this for the low-tech example of a ball thrown onto the road.

It is possible to design a LIDAR to return the distance of the first return it sees (the closest object or the last or perhaps the most solid). Some equipment may report the strongest or multiple returns. The Velodyne used by many teams reports only one return.

If a LIDAR reports the first, you can fake objects in front of the background one. More frighteningly, if it reports the last you can fake an object behind the background and possibly hide it (though that would be quite difficult and require you making your fake object very large). If it reports the strongest, you can put your fake object in either place.

Because this attack is not that dangerous, makers of LIDARs and software may not bother to protect against it. If they do take action, there are a number of potential fixes:

If their LIDAR is not predictable about when it sends pulses, the attack can only create noise, not coherent ghost objects. The noise will be seen as noise, or an attack, and responded to directly. For example, even a few nanoseconds of random variation in when pulses are sent would cause any attempted ghost object to explode into a cloud of noise.
LIDARs can detect if they get two returns from the same pulse. If they see that over a region, it is likely there is an attack going on, depending on the pattern. Two returns can happen naturally, but this pattern should be quite distinctive.

LIDARs that scan rather than spin have been designed. They could scan in an unpredictable way, again reducing any ghost objects to noise.

In theory, a LIDAR could send out pulses with encoded data and expect to see the same encoded data return. This requires fancier electronics, but would have many side benefits — a LIDAR would never see interference from other LIDARs, for instance, and might even be able to pull out information away from the background illumination (i.e. the sun) with a better signal to noise ratio, meaning more range. However, total power output in the pulses remains fixed so this may not be doable.

Of course, software should be robust against attacks and detect their patterns, should they occur. That’s a relatively cheap and easy thing to do, as it’s just software.

This topic brings up another common question about LIDAR: whether they might interfere with one another when every car has one. The answer is that they can interfere, but only minimally. A LIDAR sends out a pulse of light and waits for about a microsecond to see the bounce back (a microsecond means the target is 150 meters away). In order to interfere, another LIDAR (or attack laser) has to shine on the tight spot being looked at with the LIDAR’s return lens during the exact same microsecond. Because any given LIDAR might be sending out a million pulses every second, this will happen, but rarely and mainly in isolated spots. As such, it’s not that hard to tune it out as noise. To our eyes, the world would get painted with lots of laser light on a street full of LIDARs. But, to the LIDARs, which are looking for a small spot to be illuminated for less than a nanosecond during a specific microsecond, the interference is much smaller.

Radar is a different story. Radar today is very low resolution. It’s quite possible for somebody else’s radar beam to bounce off your very wide radar target at a similar time. But auto radars are not like the old “pinging” radars used for aviation; they send out a constantly changing frequency of radiation and look at that of the return to figure out how long the signal took to come back. They must also allow for how much that frequency changed due to the Doppler effect. This gives them some advantage, as two radars using this technique should differ in their patterns over time, but it’s a bigger problem: attack against radar is much easier because you don’t need to be nearly so accurate and you can often predict a pattern. Although radars that randomize their pattern could be robust against interference and attack.

This post originally appeared on Robocars.com.

The MIT Technology Review’s annual list of 35 Innovators Under 35 aims to illustrate the most important emerging technologies of the moment. Released this August, the 2015 list features a number of robotics and AI visionaries. Check them out.

Lars Blackmore, Visionary, 34: SpaceX engineer, working on reusable rockets that could be flown back to Earth in reverse.

Yunji Chen, Inventor, 32: designing dedicated deep-learning processors, optimized “to compute the basic blocks of machine learning.”

Adam Coates, Visionary, 34: designing artificial intelligence for Chinese search company Baidu, to make the Internet more useful to millions.

Canan Dagdeviren, Inventor, 30: working on flexible sensors and batteries for a new class of medical device.

Travis Deyle, Inventor, 32: builder of robots that can be powered wirelessly. Now at Google X working on tech to improve health care.

You may remember our Robots Podcast interview with Travis about his research on Finding objects with RFID.

Rikky Muller, Entrepreneur, 34: designing the implantable hardware intended to interact directly with the brain and help treat mental disorders.

Jaime Shotton, Inventor, 34: computer visionary at Microsoft Research. Giving computers a new way to see the world.

Ilya Sutskever, Visionary, 29: pushing the frontiers of deep-learning for the Google Brain.

You may remember the Talking Machines Podcast interview with Ilya about Machine Learning and magical thinking.

Benjamin Tee, Inventor, 33: developing medical diagnostic and treatment technologies using a synthetic sense of touch.

Conor Walsh, Inventor, 33: working on robots that are soft, lightweight, and flexible so people can wear them to enhance their abilities.

You may remember our Robots Podcast interview with Conor’s colleague Donal Holland about their Soft Robotics Toolkit.

Melonee Wise, Entrepreneur, 33: Fetch Robotics CEO wants affordable robots for the warehouse and beyond.

Check out our coverage of Melonee and Fetch Robotics on Robohub.

Credit: Illustration by Magoz; Color portraits by Matthew Hollister; black-and-white portraits by Ping Zhu. Source: MIT Technology Review.

Google’s self driving car. Source: Google

The press were all a-twitter about a report from Reuters that there had been a near miss between Delphi’s test car and one of Google’s, though it was quickly denied that anything happened.

The situation that was described — one car cutting off another — was a very unlikely one for several reasons:

• All these cars are operated by trained safety drivers who are expected to be vigilant and take control at any sign of trouble.
• In particular, special moves like a lane change would get extra vigilance. If something unusual happened (such as two cars going for the same spot) the safety drivers would be watching in advance, tracking what the car was doing, and pulling back if the car’s own displays were not telling them it was going to do the right thing.

The safety drivers are not perfect of course, but an autonomous lane change is a rare event and one that most people are still just testing, so they would be very unlikely to miss that the car was going to cut somebody else off.

Of course, situations will arise when two cars try to change into the same spot at the same time, and robocars will probably be fairly timid in these situations. The most likely situation if two robocars tried to take the same spot would be that both would back off and return to their original lane, and it will probably be that way until being so timid is not a workable strategy.

Robocars won’t be the lane-changing demons that some people (including myself) sometimes are. Many human drivers are constantly trying to find the fastest lane and we weave, only to find that the lane we just moved into has become the slowest. Part of that is our psychology.

Robocars won’t do this as much because their passengers will be occupied doing other things, and in most cases will not be in a super hurry. Those passengers will prefer a stable ride, where they can get work done, to a weaving ride with extra starts and stops. If we’re in a big hurry, we might ask the car to try to work extra hard to make the fastest trip but this will be the exception.

When we do want that, the robocar will actually have a very nice model of just how fast each lane is moving. It won’t be fooled the way we are by seeing some lanes that seem to be faster when in fact neither lane is winning by that much. If they read licence plates to identify cars, they will get excellent appraisals of what’s going on. If one lane is truly faster they will find it. On the other hand, they will be worse at the standard game of chicken needed to change lanes in heavy traffic, where you depend on the car you are moving in front of to slow down. They will know the physics though, and if a lane change is needed, they will warn the passengers of high acceleration and perfectly make a smaller spot than you might be able to make.

In other news, Google has sent two cars to Austin, Texas to expand their testing ground. I don’t have a particular insight on why they selected Austin — but I know that many towns and states regularly contact Google in the hope they might bring some cars to their area, though Texas has no modified laws yet.

Vislab

I’ve written a few times about the work of Vislab in Parma, Italy. They have a focus on doing self-driving with machine vision, and did a famous cross-continent trek from Italy to Shanghai a few years ago, using a lead car to map the way and a following car self-driving, mostly with vision.

This lab was spun out of its university but now has been acquired by Ambarella, a company that specializes in video compression chips, for $30M. One can see why Ambarella would want a computer vision lab — but it seems this might spell the end of their self-driving efforts, unless they are spun out.

Emissions

A new paper is out in Nature Climate Change on the potential for robocars to reduce emissions, in spired by some of my research in this area. Sadly, it’s behind a paywall, but the author will give a talk at Nissan’s lab in Silicon Valley on July 15th at our local self-driving car meetup.

Earlier this week Baidu announced they will be collaborating with BMW to launch an autonomous vehicle prototype later this year.

Here’s a roundup of news stories in the field of robocars.

Google begins PR campaign and talks about accidents

As the world’s most famous company, Google doesn’t need to seek press and the Chauffeur project has kept fairly quiet, but it just opened a new web site that will feature monthly reports on the status of the project. The first report gives details of all the accidents in the project’s history, which I’ve discussed previously. A new one just took place in the last month, but like the others, it did not involve the self-driving software. Google’s cars continue to not cause any accidents, though they have been at the receiving end of a modestly high number of impacts, perhaps because they are a bit unusual.

The zero at-fault accident number is both impressive, and possibly involves a bit of luck. Perhaps it even raises unrealistic expectations of perfection, because I believe there will be at-fault accidents in the future for both Google and other teams. Most teams, when they were first building their vehicles, had minor accidents where cars hit curbs or obstacles on test tracks, but the track records of almost all teams since then are surprisingly good. One way that’s not luck, of course, is the presence of safety drivers ready to take the controls if something goes wrong. They are trained and experienced, though some day, being human, some of them will make a mistake.

Baidu to build a prototype

In November I gave a “Big Talk” for Baidu in Beijing on cars. Perhaps there is something about search engines because they have made announcements about their own project. Like Google, Baidu has expertise in mapping and various AI techniques, including the advice of Andrew Ng, whose career holds many parallels to that of Sebastian Thrun, who started Google’s project. (Though based on my brief conversations with Andrew I don’t think he’s directly involved.)

Virginia opens test roads

The state of Virginia has designated 70 miles of roads for robocar testing. That’s a good start for testing by those working in that state, but it skirts what to me is a dangerous idea — the thought that there would be “special” roads for robocars designated by states or road authorities. The fantastic lesson of the DARPA Grand Challenges was the idea that the infrastructure remains stupid and the car becomes smart, so that the car can go anywhere once its builders are satisfied it can handle that road. So it’s OK to test on a limited set of roads but it’s also vital to test in as many situations as you can, so you need to get off that set of roads as soon as you can.

Zoox startup un-stealthed

Zoox is probably the first funded startup working on a real, fully automated robocar. They were recently funded by DFJ ventures and set up shop in rented space at the SLAC linear accelerator lab. Zoox was begun by Tim Kentley-Klay, a designer and entrepreneur from Australia, and he later joined forces with Jesse Levinson, a top researcher from Stanford’s self-driving car projects.

I’ve known about Zoox since it begain and had many discussions. They first got some attention a while back with Tim’s designs, which are quite different from typical car designs, and presume a fully functional robocar — the designs feature no controls for the humans, and don’t even have a windshield to see forward in some cases. (Indeed, they don’t have a “forward” since an essential part of the design is to be symmetrical and move equally well in both directions, avoiding the need for some twists and turns.) I like many elements of the Zoox vision, though in fact I think it is even more ambitious than Google’s, at least from a car design standpoint, which is quite audacious in a world where most of the players think Google is going too far.

You can see details in this report on Zoox from IEEE. I haven’t reported on Zoox under FrieNDA courtesy — in fact the early consultations with “Singularity University” described in the article are actually discussions with me.

Zoox is not the first small startup. Kyle Haight’s “Cruise” has been at it a while aiming at a much less ambitious supervised product, and truck platooning company Peleton has even simpler goals, but expect to see more startups enter the fray and fight with the big boys in the year to come.

Mercedes E Class

Speaking of supervised cruising, the report is that the 2016 Mercedes E Class will offer highway speed cruising in the USA. This has been on offer in Europe in the past. As I wrote earlier, I am less enthused about supervised cruising products and think they will not do tremendously well. Tesla’s update to offer the same in their cars will probably get the most attention.

Non-Stories

The press continue to get super excited about things that aren’t real. In spite of many reports, Uber does not yet have a car cruising the streets of Pittsburgh, though there is reality to the report that Uber has “poached” a large fraction of the robotics research crew from CMU.

In addition, many stories reported that Tesla had “solved” the liability problem of robocars through the design of their lane change system. In their system (and in several other discussed designs — they did not come up with this) the car won’t change lanes until the human signals it is OK to do so, usually by something like hitting the turn signal indicator. The Tesla plan is for a supervised car, and in a supervised car all liability is already supposed to go to the human supervisor.

Changing lanes safely is surprisingly challenging, because there is always the chance somebody is zooming up behind you at a rather rapid rate of speed. That’s common merging into a carpool lane, or on German autobahn trips. Most supervised cars have only forward sensing, but to change lanes safely you need to notice a car coming up fast from behind you, and you need to see it quite a distance away. This requires special sensors, such as rear radars, which most cars don’t have. So the solution of having the human check the mirrors works well for now.

More and more stories keep getting excited by “connected car” technology, in particular V2V communications using DSRC. They even write that these technologies are essential for robocars, and it gets scary when people like the transportation secretary say this. I wish the press covering this would take the simple step of asking the top teams who are working on robocars whether they plan to depend on, or even make early use of vehicle to vehicle communications. They will find out the answers will range form “no, not really” to a few vague instances of “yes, someday” from car companies who made corporate support commitments to V2V. The engineers don’t actually think they will find the technology crucial. The fact that the people actually building robocars have only a mild interest, if any, in V2V, while the people who staked their careers on V2V insist it’s essential, should maybe suggest to the press that the truth is not quite what they are told.

This article originally appeared on robocars.com.

S​elf-driving car keeping track of multiple moving objects at once, in this instance two cyclists, depicted as red rectangles. Source: Google.

Google has started publicly disclosing details of accidents involving its self-driving cars.

In the past weeks reports of incidents involving the company’s autonomous vehicles looked to be multiplying. Under Californian law, details of car accidents remain confidential. But, in its May Monthly Report, Google claims that its vehicles were not responsible for any of the 12 accidents they’ve been involved in since testing began in 2009.

Google released the first of what it says will be monthly reports last week. It summarises accident data and highlights from testing, and includes a synopsis of all 12 accidents. Most involved the autonomous vehicles being either rear-ended, sideswiped, or hit after another vehicle failed to follow basic traffic safety laws, such as stopping at a stop sign, according to Google.

The report also gives illustrations and examples of how the driverless cars, which average 10,000 autonomous miles per week on public streets, are able to adapt to traffic situations.

The reports are available to download on a dedicated website: www.google.com/selfdrivingcar/reports

Photo source: Wikipedia [Flckr user jurvetson (Steve Jurvetson) CC BY-SA 2.0]

Earlier this month, the Associated Press reported on several past crashes involving automated vehicles. (Per SAE Standard J3016, I use the term “automated vehicle” instead of “autonomous vehicle” or “self-driving car” or “driverless car.”) A few thoughts.

1) As I wrote in 2012, we would need more information — about the crashes themselves, the conditions under which each company’s automated vehicles are tested, and the situations in which each company’s test drivers intervene — to provide statistical context for these incidents.

2) In some ways, the AP’s inquiry gave us a preview of how public and private actors might respond to future automated vehicle crashes that actually result in injury or death. It may be instructive to view the reactions ofGoogle, Delphi, and the California DMV in this light.

3) Over the last few years, I have advised both developers and regulators of automated systems to put in place specific plans for responding, both publicly and privately, to the first high-profile incidents involving these systems. My sense, however, is that many organizations still have not created these “break-the-glass” or “break-glass” plans.

4) Earlier this semester, my impressive Law of the Newly Possible students did develop two thoughtful break-glass plans: one for the developers of automated driving systems and another for the regulators of these systems. Interestingly, although the private-sector group and the public-sector group each recognized the need to communicate with each other in the event of a crash, each also hesitated in reaching out to the other in the course of planning. In the real world, a broad range of stakeholders should be coordinating these plans sooner rather than later.

5) My book chapter on Regulation and the Risk of Inaction, also released this week, identifies eight public-sector strategies for managing risks related to automated driving. It can be freely downloaded here. A key point is that we must expect more of conventional drivers as well as automated vehicles. To paraphrase myself: I’m concerned about computer drivers, but I’m terrified about human drivers.

6) As always, please visit newlypossible.org for additional materials.

Now, I don’t know about you, but a patent on personality seems like a patent on DNA, as if Google had announced, “All your souls are belong to us.”  It made me a little jumpy because my company writes personalities for voice-control systems, be they robot, IoT appliances or avatars. Maybe, I thought, this will stunt the growth of the robotics industry and we will finally be faced with Evil.

What about all the people developing robots? Maybe they would save us. Maybe, on the other hand, all the heroic robots from 1970s would save us. A robotic personality isn’t a new concept, afterall.

I wasn’t the only one to sit up and raise an eyebrow. To get a sense of perspective I asked Alexander Reben (robotics artist and engineer) what he thought. He replied, via email, that “what Google has laid out in this patent appears to almost define the field of social robotics. It seems to be quite broad claiming to cover aspects which maybe obvious to those in the field.” Kate Darling (researcher of robot ethics and intellectual property) also thought it was quite broad.

Diana Cooper, who specializes in the legalities and and ethics of robotics (she’s Head of UAS and Robotics Practice Group at LaBarge Weinstein), was more optimistic. She wrote, “I can see this technology being very beneficial, particularly when robots are tasked with assisting in providing treatment to vulnerable populations such as children with disabilities. On a broader scale, I am optimistic that the personalization of robots will counteract society’s anxiety about robots and lead to increased consumer adoption.”

Andra Keay, of Silicon Valley Robotics, pointed out, “I think that Google are a little confused about what a robot is. To quote from one section of the patent: ‘something with processing capabilities and sensors‘ … If there weren’t pictures, I’d have assumed the patent was describing a phone or tablet.”

Personality, like the GUI, is a user interface.

However you want to think about it, the patent was issued. So there it is.

Personality is core to user interface. Personality is – like the graphic design of a GUI – the element that not only allows the sorting and categorization of information but also allows a person to emotionally bond with a robot (or avatar or talking IoT appliance or car or whatever). Personality in a video game interface is different from personality in a business SAAS. Personality also squats at the very foundation of affective computing, human-machine interaction, and even human-human interaction.

When it comes to robots, personality might actually BE interface.  If I walk into a hotel room and there’s a robot waiting to greet me, its personality will frame what we do together. If it says “Good afternoon. Would you care to order something from room service?” that’s a very different system than if it says, “Yo! Wanna throw back a few vodkas and play Call of Duty!?” Personality interface, like graphical interface, frames user interaction.

Why This Invention Is Not Patenting Personality

Titled, “Methods and systems for robot personality development” the patent (#US8996429B1 issued by the USPTO on March 31st, 2015) was authored by two employees at Google, one a researcher (Anthony G. Francis, Jr.), the other an R&D UX Creative Lead (Thor Lewis). Knowing their past work is important for understanding this invention – and what place the patent has in our emerging robotics ecosystem.

Google’s Robot Personality Patent

First, according to the work of Francis (a PhD in AI from Georgia Institute of Tech), context influences how we act. Context is something that allows us to remember, reason and then expend energy appropriately. It allows us to make the right decision at the right time. The context of a task guides its action. Francis has also spent a good deal of time looking into affective computing. And it just so happens that he’s a science fiction author.Second, Lewis (artist, 3D modeler, designer, and science fiction buff, himself), has authored other patents based around machine vision, and came to the world of AI vision from the inside out. He went from making visual things with computers to making computers visual things.

So these two get together and we may imagine their conversation in the Google Coffee shop one afternoon:

“Hey, if a robot can see a thing, like a person, and build context, then doesn’t that inform what the robot does?”

“Yeah, and maybe the robot can get a sense of who that person is based on the stored personal information via their online accounts.”

“Sure, so if the person is at a wedding or a funeral and that would change the way the robot would interact with them, right?”

“Right, and it could remember its past personalities and interactions!”

“Totally! And it would live in the Cloud!”

“Awesome!”

“Hey, you got a pen?”

As I read the patent I realized that this isn’t an invention for personality. This is an invention for how to modify a personality in a context. The way that the robot looks, moves, sounds and feels (and touch is mentioned, too), all impact the personality, which can be transferred via the cloud, so personalities can be uploaded, downloaded, shared, and modified. But the patent isn’t as broad as it might first appear. This isn’t inventing personality, per se. It’s inventing how a personality builds context and interacts. The last word of the patent’s title explains that this is not about what personality is, but how personality is developed.

How To Develop A Personality

There are a few pithy lines in the invention. “Personality may be thought of as a personification in the sense of human characteristics or qualities attributed to a non-human thing.” Then the authors set out how those characteristics develop in a context.

Multiple devices are cross-referenced. This allows the robot to collect information on a user and cross-reference data to build context. This means that GoogleBot can now tell that I like to eat cake at a wedding, but not at a funeral, and then modify its personality to align with that context. It will know to offer me cake when people are getting married, not buried.

The patent reviewer, in the “Reasons for Allowance” notes that: “processing to obtain ‘data’ as opposed to information theoretic ‘information’ occurs in response to ‘obtaining’ the information.” This is specifically cited as a part of cloud services as they relate to collecting visual information. Thus the patent differs itself from prior art.

But why would a robot personality development patent be useful to Google?  This is core to Google’s advertising business model. GoogleBot will know whether I prefer to eat cereal in the morning or evening, what kind of cereal, what kind of milk, what kind of bowl, what kind of spoon, table, and the timing it takes me to eat that cereal. And just like the business model of web searches from the late 1990s, Google will then be able to sell that data to the many advertisers that support Google’s highly valuable business. Not just cereal of course, but shoes, shirts, shit, shinola and all the other things we consume, produce, and interact with in our daily lives. This context means that the GoogleBot will become one of the most powerful marketing, advertising, and search mechanisms ever built.

A personality that can consolidate our data is one that truly knows us.

GoogleBot’s adaptive personality can consolidate our data and that consolidation knows us. The GoogleBot, with its emphasis on vision, will be able to tell what other cereals I like, what other bowls I use, what other shirts I wear and when I leave for work. By simply observing it will (like search, email, docs, voice, Google+, and the host of other services that Google offers) know me. What is key to context – and to the value of Google’s USD$50 billion-a-year advertising system – is the consolidation of channels of information.

Channel aggregation is a proven and valuable business model. Apple did something similar. When Apple released the iPad (leashed to iTunes and other services), they consolidated the movie channel, the video rental store, the movie theater, and the television into a single device. Apple allowed us to consume more by aggregating media delivery channels.

Apple aggregates output channels to facilitate media sales and Google aggregates input channels to facilitate advertising sales.

Remember that your data is Google’s product, and they sell it to their customers. Imagine a Google Search engine housed in a little teddy bear that modified what it presented based on your past searches. Now imagine that the little teddy bear understood how you felt about those searches, and why. And it told you so, moved, sounded, and even felt a bit different as a result. Now imagine that the context of your search, via this sensitive, sweet robot that is a cuddly animal with an immense, encyclopedic brain, is now completely understanding you and adapting to your needs. Add the Google AdSense, flavor with Google Analytics, stir in Nest, DoubleClick, Uber, 23andMe, residential fiber, and the other Google goodies, and you’ve got something like the Google Panda Robot that was announced the same day the patent was issued (and no, this wasn’t an April fool’s joke). The Amazon Echo appears to be following a similar model, as I mentioned in a previous article.

Internet penetration growth rate is set to peak in 2016.

Google has to solve some important problems in the coming year, so this robotics approach is also timely. Internet penetration growth rate is set to peak in 2016. Since Google’s P&L is proportional to Internet usage, they need to invent some new models. Their three existing models – consumer devices, ad-supported consumer services, and software-as-a-service – are approaching maximum impact, and we can expect that they will consolidate all three into this robotic personality that will develop so as to offer each aggregated input channel when it is most contextually appropriate. Google can now consolidate the contexts of buyer-seller to complete the holy circuit of advertise-search. And the context of personality is the interface.

To Know Me is To Own Me

If we thought that value was inherent in goods themselves, then we were wrong. As advertisers have long known, the personal information of what cereal I like is far more valuable than the cereal itself. It will be the opposite of what we thought, especially if what is valuable is what the robot collects. The personal information collected to determine if we like cereal or not will be far more valuable than the cereal itself. So like Shroedinger’s cat, the robot will look for our personality in the box, and our personalities will shift as a result. The context will change and this sensitive interface is will adapt because it was invented to do so. Just like search on Google’s webpage, the robot’s personality is more about collection than display.

Google collects our personalities by cross-referencing our context; this new patent is about cross-referencing context to reflect personalities back to us. And this allows for an improved collection method.

So while Google’s recent patent at first appears to be a method for displaying robot personalities, it is actually a means of collecting our personalities, and therefor our purchasing decisions. It’s not that Google will own our souls, but they will own our personalities, and therefor our decisions.

“All your decisions are belong to us” is writ large upon the sky. But for now this is only a patent, and despite the appearance of the cute teddy-bear robots, our decisions are still left up to us.

At least for now.

---

If you liked this article, you may also be interested in:

• Why are JIBO, Pepper, Siri, Google Now and Cortana so important?
• The robots behind your Internet shopping addiction: How your wallet is driving retail automation
• Google commits $1.36 billion for NASA facility, to house their robotics, space and flight technologies
• Google news: Rubin leaves; Pichai promoted to #2; and Google invests in Magic Leap
• Google adds to DeepMind, acquiring 2 UK startups and partnering with Oxford U
•  Robots Podcast: Privacy, Google, and big deals

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

 

Delphi has made waves by undertaking a road trip from San Francisco to New York in their test car, which is equipped with an impressive array of sensors. The trip is now underway, and on their page you can see lots of videos of the vehicle along the trek.

The Delphi vehicle is one of the most sensor-laden vehicles out there, and that’s good. In spite of all those who make the rather odd claim that they want to build robocars with fewer sensors, Moore’s Law and other principles teach us that the right procedure is to throw everything you can at the problem today, because those sensors will be cheap when it comes time to actually ship — especially for those who say they won’t ship for a decade.

At the same time, according to Kristen Kinley at Delphi, their test involves mostly highway driving, with very minimal urban street driving. They are attempting off-map driving, which is possible on highways due to their much simpler environment. Like all testing projects these days, there are safety drivers in the cars ready to intervene at the first sign of a problem.

Delphi is doing a small amount of dedicated short-range communications (DSRC) vehicle-to-infrastructure testing as well, though this is only done in Mountain View where they used some specially installed roadside radio infrastructure equipment.

Delphi is doing the right thing here — getting lots of miles and different roads under their belt. This is Google’s giant advantage today. Based on Google’s announcements, they have more than a million miles of testing in the can, and that makes a big difference.

Hype and reality of Tesla’s autopilot announcement

Telsa has announced they will do an over-the-air upgrade of car software in a few months to add autopilot functionality to existing models that have sufficient sensors. This autopilot is the “supervised” class of self driving that I warned may end up being seen as boring. The press have treated this as something immense, but as far as I can tell, this is similar to products built by Mercedes, BMW, Audi and several other companies, and even sold in the market (at least for traffic jams) for a couple of years now.

The other products have shied away from doing full highway speed in commercial products, though rumours exist of it being available in commercial cars in Europe. What is special about Tesla’s offering is that it will be the first car sold in the US to do this at highway speed, and they may offer supervised lane change as well. It’s also interesting that, since they have been planning this for a while, it will come as a software upgrade to people who bought their technology package earlier.

UK project budget rises to £100 million

What started with a £10 million pound prize in the UK has grown become over 100m in grants in the latest UK budget. While government research labs will not provide us with the final solutions, this money will probably create some very useful tools and results for the private players to exploit.

MobilEye releases their EyeQ4 chip

MobilEye from Jerusalem is probably the leader in automotive machine vision, and their new generation chip has been launched, but won’t show up in cars for a few years. It’s an ASIC packed with hardware and processor cores aimed at doing easy machine vision. My personal judgement is that this is not sufficient for robocar driving, but MobilEye wants to prove me wrong. (The EQ4 chip does have software to do sensor fusion with LIDAR and Radar, so they don’t want to prove me entirely wrong.) Even if they are not good enough on their own, ME chips offer a good alternate path for redundancy.

Chris Urmson gives a TED talk about the Google Car

Talks by Google’s team are rare — the project is unusual in trying to play down its publicity. I was not at TED, but reports from there suggest Chris did not reveal a great deal new, other than repeating his goal of having the cars be in practical service before his son turns 16. Of course, humans will be driving for a long time after robocars start becoming common on the roads, but it is true that we will eventually see teens who would have gotten a licence never get around to getting one. (Teens are already waiting longer to get their licences, so this is not a hard prediction.)

The war between DSRC and more wifi is heating up.

Two years ago, the FCC warned that since auto makers had not figured out how to benefit from the DSRC spectrum at 5.9ghz, it was time to repurpose it for unlicensed use, like more WiFi. There is now a bill to force this proposition.

I think this would be the best thing ever for V2V, and I outlined 2 years ago how doing V2V in phones instead of cars is far more likely to get success. Phones are replaced every two years with a whole new generation, and smartphones are already in 75% of cars in the USA. V2V, even under a mandate, won’t get close to that until after 2030, and by that time it will be 15-year-old technology, long obsolete.

Google Airbags on the outside

A Google patent for placing airbags of a sort on the outside of vehicles to protect others has gotten a lot of press. While I was working at Google, I started the push for this (though the design in the patent is not mine.) I’m glad to see more investigation of it, though. The reality, as everybody knows, is that perfection is not possible, so planning for the hopefully small number of inevitable accidents and mitigating them is the right strategy.

Nvidia

For the past year, Nvidia has been pitching their latest Tegra GPU in the mobile space. They argue that a general purpose chip with very high power, thanks to their giant research budget, will produce better results than a dedicated machine vision chip from a smaller company like MobilEye that takes several years between generations. As a result, they put a big focus on automotive and self-driving at their recent GPU conference. I was in Europe and unable to attend.

More news from China

Reports suggest that Chinese internet companies Baidu and Tencent (WeChat) are starting robocar projects. (Previously Baidu provided some maps for BMW’s projects.) I’ve spoken at a couple of events for Baidu and encouraged this direction. Details are scarce though, and there is some speculation this is more in the “smart” car direction (similar to Tesla) than a full robocar. We should note that the legal situation in China is a lot different, and these large companies might be able to easily get more advanced products to be declared legal well before those in other countries can.

BMW crash

A BMW driver testing night vision collision avoidance systems crashed into a police van in Stuttgart. This was not a robocar, and of course it is amusing that if you have a crash, you might as well do it into a speeding police van! It does raise issues about risks of testing things with safety drivers — in this case, perhaps the test driver was distracted by his testing and didn’t pay enough attention to the environment. Most of the supervised class vehicles being developed today require the driver to keep paying attention, something that Google has decided not to pursue.

This article first appeared on robocars.com.

The head of Google’s moonshot lab Google X has recently announced they have scrapped the initial ‘Project Wing‘ design that they have been testing in Australia. According to X lab Director Astro Teller, the old design was difficult to control and they are now working on a new design. 

Some features from the initial design:

• Project Wing had a wingspan of approximately 1.5m (4,9ft)
• Propulsion was driven from four electrically-driven propellers
• Total weight, including delivery payload, was approximately 10kg (22lb)
• The aircraft weight was 8.5kg (18.7lb)
• Payload 1.5kg (3.3lb)
• The aircraft was capable of VTOL (vertical takeoff and landing)
• The hybrid design used the rotors for VTOL and hovering but used the wing shape for forward flight to maximize range and efficiency
• Payloads would be dispatched from the aircraft while in hover mode using a tether

It is understood that the team have been working on a new secret design since they started testing in Australia. This new design is said to move away from the single-wing approach. Google has been working on drones since 2011 so will surely have a number of different concept designs. However, the ‘Project Wing‘ design we reported on is the only one we’re aware of that made it to field testing. Teller explained that the design didn’t cope well in high winds and its cargo shifted too much when the wing moved up and down.

Former head of ‘Project Wing‘ Nicholas Roy has said that delivery drones “are absolutely viable, but there are a lot of technical hurdles that have to be crossed.” Nicholas Roy is a professor at the Massachusetts of Institute of Technology and says “we are very much in the prototype stage.”

Google are not the only ones looking at turning delivery drones into a reality. Amazon, DHL and Chinese giant Alibaba are all working on delivery drones. Some experts predict that there is another 5 years of development needed before we see large scale deployment.

• Google’s secret drone delivery program
• Is drone delivery really a robot startup?
• Is package delivery using drones feasible?
• The economics of Amazon’s delivery drones

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

]]> https://robohub.org/intuitive-surgical-da-vinci-surgical-system-gets-big-endorsement/ Tue, 24 Mar 2015 01:27:00 +0000 http://robohub.org/intuitive-surgical-da-vinci-surgical-system-gets-big-endorsement/

For over two years, Intuitive Surgical has been sued, tried, analyzed and criticized about the efficacy of their da Vinci surgical robots. But they have continued to thrive, grow and improve their products.

Proof of the success of their efforts came the other day when they won a $430 million contract from the U.S. Department of Defense. It’s a five-year deal to supply da Vinci robots, tools and spares to Army, Navy, Air Force and Marine Corps hospitals.

The DoD contract comes shortly after the company successfully received FDA approval for two of its new surgical systems last year – the da Vinci Xi, a gantry-style upgrade to their basic system that includes a higher definition vision system, better consoles and controls (and in some cases two sets of controls and consoles) and a general upgrade of everything else, and the new da Vinci SP (single port) system. The SP system squeezes a 3D high-def camera and three fully articulating instruments through its 25 mm cannula (single thin tube containing the arms and end-of-arm tools; the tube goes through a single hole in the body).

Intuitive's 2014 revenue was $2.1 billion and the number of procedures performed using da vinci products grew 9% year over year to almost 600,000. Quoting from Intuitive’s Annual Report about 2015:

"In 2015, our focus will be on accomplishing the following objectives: First, we will focus on the expanded use of da Vinci in general surgery, particularly colorectal surgery and hernia repair. Second, we will work to complete our launch and new product introductions of the da Vinci Xi Surgical System in key markets globally. Third, we will continue to develop our organizational capabilities and markets in Europe and Asia. Finally, we plan to build da Vinci SP prototypes that are compatible with our da Vinci Xi system and initiate customer evaluations."

Read more

UPDATED 3/29/15 For over two years, Intuitive Surgical has been sued, tried, analyzed and criticized about the efficacy of their da Vinci surgical robots. But they have continued to thrive, grow and improve their products.

Proof of the success of their efforts came the other day when they won a $430 million contract from the U.S. Department of Defense. It’s a five-year deal to supply da Vinci robots, tools and spares to Army, Navy, Air Force and Marine Corps hospitals.

The DoD contract comes shortly after the company successfully received FDA approval for two of its new surgical systems last year – the da Vinci Xi, a gantry-style upgrade to their basic system that includes a higher definition vision system, better consoles and controls (and in some cases two sets of controls and consoles) and a general upgrade of everything else, and the new da Vinci SP (single port) system. The SP system squeezes a 3D high-def camera and three fully articulating instruments through its 25 mm cannula (single thin tube containing the arms and end-of-arm tools; the tube goes through a single hole in the body).

Intuitive’s 2014 revenue was $2.1 billion and the number of procedures performed using da vinci products grew 9% year over year to almost 600,000. Quoting from Intuitive’s Annual Report about 2015:

“In 2015, our focus will be on accomplishing the following objectives: First, we will focus on the expanded use of da Vinci in general surgery, particularly colorectal surgery and hernia repair. Second, we will work to complete our launch and new product introductions of the da Vinci Xi Surgical System in key markets globally. Third, we will continue to develop our organizational capabilities and markets in Europe and Asia. Finally, we plan to build da Vinci SP prototypes that are compatible with our da Vinci Xi system and initiate customer evaluations.”

 

In spite of this success, or perhaps because of it, Google and Johnson & Johnson announced a collaboration to produce a competing line of products to use robots to assist surgeons. The announcement of this strategic pooling of resources didn’t include a price tag or investment amount.

According to the San Jose Mercury News, “Google stressed that the life sciences team at Google X will only be providing software and sensors for the proposed systems, with J&J’s Ethicon unithandling the rest of work. The Mountain View company’s robotics arm – Google acquired robotics firm Boston Dynamics in late 2013 – will not be involved.”

“We think we can use some of our machine vision and image analysis software to help surgeons see better as they operate (e.g. to highlight blood vessels) or make it easier for them to see information that’s relevant to the surgery at just the right time,” a Google spokeswoman said.

An Intuitive Surgical spokeswoman said that the effort by Google and Ethicon was a validation of Intuitive Surgical’s work.

“This announcement is just the latest confirmation that computer-assisted surgery continues to grow to the point that others can no longer sit on the sidelines. Computer-assisted surgery will continue to flourish around the globe and competitors have been expected.”

 

]]>

Imagine an assistant that knows your schedule, contacts, interests and location, and that can communicate with you to help you stay informed, connected and on top of your game. This is more personalized than your PC, tablet or smartphone and, depending on whether physical or virtual, is in a pleasing, personal and compatible form. If these assistants catch on, they will begin anticipating and handling many of the little decisions that fill our days. The potential for this form of artificial intelligence has tech companies vying to lead the pack.

Apple’s Siri, Google’s Google Now, Microsoft’s Cortana, Facebook, and Baidu are all in various states of pursuing virtual assistants. But JIBO, Pepper, Cubic, EmoSpark and others are just now coming to market and are offering a physical component – a robotic presence – which they hope will enhance the experience.

2015 is the year physical products will be coming to market and available for experimentation and testing. Pepper ships in the summer in Japan, JIBO ships preorders in Q3 as does Cubic in the fall and EmoSpark in the summer.

These types of personal assistants rely on multiple activities including: speech recognition to determine and parse through spoken commands; predictive technology to anticipate what a user will likely do next; and determining and using contextual data (biographic data and situational information, e.g.: location, time and date). Apple, Google and Baidu – all of which attempt to provide this service – want their intelligent interface to fit all front ends, to be the universal contact point. They see a plentiful revenue stream if they are successful. Others want Apple and Google to redirect users to company sites with apps that understand their own content better, e.g.: directing requests for pizza to Dom, the digital front-end for taking Dominos Pizza orders.

Many are waiting to see how the competition plays out. Car companies and other smart product providers, fearing a variety of consequences outside of their control, aren’t enamored of letting Apple or Google (or any of the others) run their voice-powered front-end systems … thus their current intense race for in-house and startup development. But if Google or Apple or ??? gets to market first with a true digital assistant that people embrace and buy – one that can be virtual or physical – car companies may need to make their systems adapt to that system because it’s what everyone wants to use.

The key to the outcome of this race is whether a general purpose AI will be able to steer people through their digital world, or whether users would rather navigate to applications that are specialists (such as American Airlines or Dominos Pizza).

Natural language recognition – and the ability to analyze user desires from that language – is a very competitive science these days. Nuance, the biggest provider of voice and language solutions to the auto industry and a collaborator with IBM up until recently, is now competing with IBM’s Watson team and also Apple, which is rumored to be working on their own language processing system instead of using Nuance. A look at the What’s Next section of Nuance’s website is, to quote them, “The intersection of Science Fiction, super-pi, and technology innovation.” But a more descriptive quote from that page seems more appropriate: “Intelligent self-service delivers a better customer experience.” That’s what this race is all about.

• The second wave of social robots is here
• Amazon’s Echo: A point-of-sale even the kids can relate to
• The next era of conversational technology
• The death of search (or, My dysfunctional relationship with Siri)

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Boston Dynamics just released a video of a new four legged robot named “Spot”.  It is an evolution along the lines of their previous four-legged robots like BigDog and Wildcat, but this one is much smaller and lighter (160lbs / 72.5kg). As usual not many details are known, but Spot is electrically powered (others had an internal combustion engine onboard) and has a prominent rotating LIDAR on top.

Spot can perform the usual Boston Dynamics trick: it can withstand a kick without tipping over in an eerie life-like manner; and it can also move slowly and accurately indoors while being able to run faster if necessary.

You can watch the video below and you can read other Boston Dynamics articles here.

We tackle a listener question about the dream of a one-step solution to strong Artificial Intelligence and if Deep Neural Networks might be it.

Plus, Ryan introduces us to a new way of thinking about questions in machine learning from Yoshua Bengio’s Lab at the University of Montreal, as outlined in their new paper, Identifying and attacking the saddle point problem in high-dimensional non-convex optimization, and Katherine brings up Facebook’s release of open source machine learning tools and we talk about what it might mean.

If you want to explore some open source tools for machine learning we also recommend giving these a try:

• Super big list of ML Open Source Projects! 
• Torch 
• Gaussian Process Machine Learning Toolbox
• PyMC
• Mallet
• Stan
• Weka
• Theano
• Caffe
• Spearmint

Google just unveiled their first ‘real-build’ self-driving prototype vehicle. This is a follow up to the concept presented in May and signifies the progress of this particular Google project.

According to the short Google announcement, the purpose of this new prototype is testing, initially on a closed test track and then later in 2015 on public roads. Even so, very few actual details are known, beyond those we covered in our previous article for the initial mockup.

The differences between the two cars, however, are actually quite interesting. Google emphasizes the presence of headlamps, which are an indication that the vehicle could be road legal, although it’s not clear to what extent. A glimpse of its frame and construction hinted at a properly engineered vehicle that may be possible to fulfill all the necessary requirements for a regulatory approval (crash tests etc). However, a loophole for this highly experimental vehicle is possible; it may, for example, be regarded as an electric golf cart of sorts.

The composite photo above presents the most obvious differences: a better integrated LIDAR, the headlights and other small details. The most important visible difference is the mirrors. On the first mockup, the mirror fairings were small and so could only accommodate the sensors or cameras of the autonomous system. The new one retains the sensors but adds a proper mirror for the driver. It is almost certain that the car will have manual controls (steering wheel, accelerator and brake pedals) so it can be “supervised” on public roads.

2015 will be an interesting year for autonomous driving. Apart from this purpose-built concept, many manufacturers offer ever more advanced systems, and Google is promoting a modular solution of their own.

NASA also said:

Once renovations are complete, Hangar One will again be home to high-tech innovation as Planetary Ventures begins using the historic facility for research, development, assembly and testing in the areas of space exploration, aviation, rover/robotics and other emerging technologies. Hangars Two and Three will be used for similar purposes.

 

This news comes on the heals of Google’s investment in Magic Leap last month, as well the announcement of Andy Rubin’s departure.

Andy Rubin leaves Google; Sundar Pichai promoted to #2 man at Google; and Google invests in $542 million B funding for Magic Leap.

Andy Rubin, the Google executive who was in charge of the company’s robotics group and spearheaded the Android operating system before that, resigned from Google and will start a tech incubator focused on hardware.

According to the NY Times, Mr. Rubin’s departure is part of a series of recent executive moves that seem to give Mr. Page more room to focus on the company’s longer-term bets — like robotics — while handing almost all the responsibility for Google products to Sundar Pichai, a rising star. The robotics group will now be led by James Kuffner, who has been at the company since 2009 and has worked on self-driving cars, a Google spokesman said.

Regarding James Kuffner, who replaces Rubin, The Wall Street Journal said,

Andy Rubin, former leader of Google’s Android mobile business, oversaw a series of robotics acquisitions last year, so his departure comes at a poor time for the company. 
That puts pressure on Kuffner to bring together disparate Google properties like Boston Dynamics and Schaft.

“Robotics is still a difficult technology to pull off well, I think moreso than many or all tech fields,” said Michael Wagner, a senior commercialization specialist with the National Robotics Engineering Center at Carnegie Mellon University. “James has the expertise to navigate traps others with less experience may not expect. So I don’t think this is a setback at all.”

Sundar Pichai becomes Google’s Product Chief. The NY Times said of the promotion:

Sundar Pichai, is to oversee every major Google product except YouTube. The move arguably makes Mr. Pichai the second-most powerful person at the company, and is intended to give Mr. Page more time to focus on more futuristic projects, according to a person with knowledge of the situation.

In a memo on Friday, Mr. Page said that Mr. Pichai’s empire – which already includes the Android and Chrome products, along with Google Apps like Google Docs and Gmail – will now include Research, Search, Maps, the social media network Google Plus, along with Commerce. He will also oversee technical infrastructure, which is essentially the ever-growing racks of servers and switches that keep all those products working.

Much interest within the robotics community has been focused on Google’s acquisitions in the area of robotics because Google hasn’t said anything about what they are doing. So many start-ups …  So much talent … What’s their long-term goal? What’s the common denominator? How, where and when will those purchases and their activities combine into some money-making enterprise(s) and/or product(s)?

Magic Leap

As artificial intelligence software and emerging learning chips fuse into more speedy and capable pursuits, the line between things robotic (which has a specific definition) and AI-driven robotic activities not yet defined (like software bots) is coming into question. Hence the interest in where Google is focusing. Some of that focus has been on acquiring learning system companies (and talent). Additional focus may include virtual reality, 3D imaging, and augmented reality systems. Imagine that you were driving in China and didn’t speak or read Chinese. How would you understand exit signs on the freeway? With augmented reality systems in your car, signs you see could be translated automatically so that what you percieve is the sign in your language without distraction.

The opening scene on the Magic Leap website is of a virtual elephant inside the palm of your hand. No Google Glasses. No Oculus Rift headset and goggles. Scrolling down a bit one sees a classroom scene with colorful sea horses floating inbetween the kids. Back in July, NY Times technology writer John Markoff described Magic Leap’s grand goals.

Thus it was no surprise that Google participated in the recent $542 million B funding round for the start-up Magic Leap, a Florida start-up that claims to merge the physical and digital worlds with hardware and software that surpasses existing virtual and augmented reality solutions. Other participants in the B funding included Qualcomm, KPCB, Andreessen Horowitz, Obvious Ventures, and Legendary Entertainment.

In January, 2014, Google acquired London-based DeepMind Technologies for $643 million. Now it is adding to that purchase with two more companies, ten new hires and a substantial contribution to Oxford University.

Google DeepMind was a British AI start-up with some high-profile investors and a whiz kid founder (Demis Hassabis), and their forte was smart recommendations for online commerce. Now they are acquiring companies, talent and university research resources.

The two acquisitions are Dark Blue Labs (which is creating systems to understand natural language that would allow computers to comprehend the meaning of sentences and how people react to them) and Vision Factory (which is developing systems capable of the visual recognition of objects in the real world; this means, for example, giving robots three-dimensional awareness that can allow them to understand how a cup sits on a table). Both are UK-based, deep learning startups, and both were spun off from Oxford University. According to the Financial Times, the two acquisitions are estimated to have cost Google DeepMind $50 million.

The Google-Oxford partnership will cost more. A substantial contribution to Oxford University is forthcoming to expand Google’s AI and deep learning capabilities.

“Google DeepMind has hired all seven founders of these startups with the three professors holding joint appointments at Oxford University where they will continue to spend part of their time,” Hassabis notes in his blog post. “These exciting partnerships underline how committed Google DeepMind is to supporting the development of UK academia and the growth of strong scientific research labs.”

f you liked this article, you may also be interested in:

• Artificial General Intelligence that plays Atari video games: How did DeepMind do it?
• Google’s robot and artificial intelligence acquisitions are anything but scary
• Google’s DeepMind acquisition in reinforcement learning
• Big Deals Series: What does it mean to have the giants investing in robotics?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Last December, an article named “Playing Atari with Deep Reinforcement Learning” was uploaded to arXiv by employees of a small AI company called DeepMind. Two months later Google bought DeepMind for 500 million euros, and this article is almost the only thing we know about the company . Currently our team is trying to replicate their artificial mind, and in this post we describe its inner workings.

The paper describes a single system that learns to play several old video games by selecting actions based on only the screen images and the reward signal (whether the score in the game is increased or not).

At first (before training), the program does not know anything about the game: it doesn’t know the objective of the game (if it should just stay alive, kill something or solve a puzzle), it has no understanding about its influence on the game (the consequences of its actions) and one could even say that it cannot see the objects in the game. By trial and error the system gradually learns how it should behave to receive a reward. Also note that the system uses the same architecture for all different games without any game-specific hints from the programmers other than how many possible actions there are in the game, e.g. left, right, up, down, space (fire) etc.

The reported results show that the system was able to master several games and play some of them better than a human player. This result can be seen as a step towards artificial general intelligence (AGI), which is really fascinating. Let us see in more detail how this was achieved.

The Task
The system receives a picture of the game screen (this example image is from the simplest game, Breakout) and after analyzing it, it has to choose an action to take. The action is executed and the system is told whether the score increased, decreased or did not change. Based on this information and playing a large number of games, the system needs to learn to improve its performance in the game.

Take a pause and think about it: try to outline in your head how you would approach this task if someone had asked you to program it. Keep in mind that you do not know in advance which game your system will need to play.

Machine Learning and Artificial Neural Networks
Before we dive into the implementation DeepMind came up with, let’s introduce the concepts of machine learning and artificial neural networks.

A very common task in machine learning is: given the information about an object decide which category it belongs to. In the case of DeepMind we want to decide which action to take given the current state of the game. Machine learning algorithms learn from examples: given many example objects and their categories, the algorithm figures out which features of the object are useful for identifying the category. The algorithm produces a model that minimizes misclassification rate in the training set. This model can later be used to predict the category of an unknown object.

Artificial Neural Networks (ANN) is one of the machine learning algorithms. It is inspired by the architecture of our brain. The network consists of nodes (as our brain consists of neurons) that are connected to each other (as neurons in our brain are connected using axons, dendrites and synapses). For each neuron we must specify the condition under which the neuron should transmit the signal forward (emit a spike in real neurons). By changing the strengths of the connections one can make the network perform computation. Modern architectures of neural networks usually consist of:

• Input layer of neurons (they receive the description of an object)
• Hidden layers (the main part, where the learning happens)
• Output layer (it has one neuron for each category, the category with highest activated node is the predicted class)

After the learning is done we can feed new objects to the network and see scores for each category in the output layer.

DeepMind

In this system an ANN is used to assign an expected reward to each of the possible actions given the current game state. The network can answer a question, e.g. “How good would it be to take this action now?”. The input to the network at any time point consists of the last four game screens, so that system is given not only the last position, but also a bit of information about how the situation in the game has evolved recently. This input is then passed through three successive hidden layers of neurons and finally to the output layer of the ANN.

The output layer has one node for each possible action and the activation of those nodes indicates the expected reward from each of them – here, the action with the highest expected reward (the “best” action) is selected for execution. But how does the network know how to calculate the expected scores in a good and correct way?

The Learning Process

After you have defined the structure of your network there is only one thing you can still change: the strength of the connections. The learning process is, therefore, just changing the weights in a way that for all of the training examples the network is capable to propose actions that lead to positive rewards.

It is an optimisation problem where the goal is to maximize the reward over time. The problem is solved by combining gradient descent with a simple idea related to reinforcement learning: the network attempts to maximize not only the immediate reward, but also the cumulative reward of future actions. This can be achieved by estimating the expected value of the next screen image using its current neural network. To put it another way, traditionally you would use (received reward – estimated reward) as the error signal for gradient descent, but in this case we also take in consideration the reward estimate from the next image.

Summary

To summarize all the pieces we have seen, let us have one final look:

1. Build a network and initialize connections to random values

2. Feed lots and lots of game situations into the network

3. Network proposes the action to take in each of them

4. If the action was good, we reward the system, if it was bad we penalize it

5. System updates its knowledge by updating the weights

6. After several hundreds of games the system outperforms a human player (in the case of Breakout)

This result can be seen as a small step forward from the traditional machine learning in the direction of artificial general intelligence. It might be a very small step (one might say that the system does not “know” or “understand” what it is doing), but the learning capabilities of DeepMind’s system are far larger than its predecessors’  – and the range of problems is can solve without specific hints from the programmers is much wider.

Replicate and Make It Open Source

At the moment we are trying to replicate this successful project and make it available to the whole scientific community for further research. The project is open source and you can jump on board to help on Github!

References

The original paper:
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning.

Our implementation at GitHub:
Reproducing the results of “Playing Atari with Deep Reinforcement Learning” by DeepMind.

The authors of this article are from the Computational Neuroscience Group at University of Tartu’s Institute of Computer Science.

If you liked this article, you may also be interested in:

• Google’s robot and artificial intelligence acquisitions are anything but scary
• Google’s DeepMind acquisition in reinforcement learning
• On internal models, consequence engines and Popperian creatures
• Why robots will not be smarter than humans by 2029

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The Atlantic revealed the inside story of Google’s two-year secret X-labs drone-delivery project in a detailed story by Alexis Madrigal. Project Wing, as the drone-delivery venture is called, has already conducted test flights delivering small packages to rural areas of Australia. Watch the Google promo video:

The Atlantic article describes how the present form of drone came to be a tail sitter, i.e., a hybrid of plane and helicopter, which takes off vertically but flies horizontally.

After observing some users attempting to grab the package before it was actually delivered, Google decided for safety reasons to winch the package down to the ground instead of flying down and placing the package there when making a delivery. Thus the drone hovers in a vertical position and winches down the package on a tether line, which also contains an “egg” of electronics. The egg detects when (or if) the package has reached the ground, electronically detaches, and instructs the vehicle to haul up the tether.

Dave Vos, an experienced unmanned aerial and marine systems engineer, is heading up Project Wing for Google. According to the Atlantic story, there are already dozens of people working on various aspects of the project: delivery mechanism, user experience, the app for ordering up the drones, etc. The article didn’t mention where the product that was being delivered came from or how the recipient ordered and paid for that product. That’s another secret waiting to be discovered.

Perhaps the most interesting statement from the article was this:

Of course Google wants the world to believe in delivery by drone as part of the natural progression of a technological society to deliver things faster and faster. This is how the world works.

Take that, Amazon!

Google’s recent acquisition of Emu Messenger is just one of many items in recent news about improvements in perception and artificial intelligence (AI).Emu, not to be confused with the Australian ostrich-like bird nor the European Monetary Union, is a small Palo Alto start-up comprised of some serious software talent with experience in machine learning, natural language processing and mashing up different data, databases and systems at Siri, Apple, AOL and Google. Perception in this case is the feeling that your words and intentions are understood and acted upon. No financial details were disclosed about the acquisition however the Emu app will be shut down next week.

Another form of perception, computer eyesight (AKA machine vision), took a giant step forward when the winners were announced for this year’s Large Scale Visual Recognition Challenge. The NY Times reported the winners: the National University of Singapore, the Oxford University, Adobe Systems, the Center for Intelligent Perception and Computing at the Chinese Academy of Sciences, as well as Google in two separate categories.

Fei-Fei Li, director of the Stanford Artificial Intelligence Laboratory said, “What really excites us is that performance has taken a huge leap.”

Machine vision has been a challenge in automation and robotics but in recent years has become integral in countless applications including computer gaming, medical diagnosis and factory robotics. Carmakers too have added the ability to recognize pedestrians and bicyclists and trigger safety actions. Factory robots need much improved perception systems in order to monitor the progress of their tasks and the tasks of those around them. [A quick Google Images search produces a collage of various uses, from milking to adaptive cruise control.]

Enhanced algorithms, data libraries and faster and cheaper computing are all contributing to the increased accuracy and speed of the systems recognizing objects and identifying them by type and in 3D space, nevertheless, at their best they are still no match for human vision and perception.

Google completed a major step in its long and extensive self-driving cars project by presenting its first purpose-built autonomous car, which is designed from scratch for its role and is not a modified conventional Toyota.

The as yet unnamed car is very small (looks smaller than a Smart) and can accommodate two people and some luggage. It’s probably electric and its maximum speed is limited to 25mph (~40km/h). Its most striking characteristic is that it doesn’t have any controls — no steering wheel, accelerator or brake pedals — and you can ride it strictly as a passenger, which is probably a strange feeling, but according to Google’s video not entirely unpleasant.

This is a purely experimental project in every aspect. Google will make about a hundred prototypes and it looks like it’s a very serious effort. It is equipped with multiple sensors (a big LIDAR on its roof is probably doing most of the work), but there are also sensors in the front, in the back and where the side view mirrors would have been in a regular car. Thanks to Google’s previous experience with self-driving cars, one can expect very good performance in real environments; other Google self-driving cars have completed hundreds of thousands miles with no major incidents.

The car itself may look like a toy but it is cleverly and purposefully designed to be as cute as possible in order to inspire trust and reduce fear of its autonomous status. Small cars in general may be more vulnerable in a crash with a heavier vehicle, but it’s already proven that with clever engineering you can have almost no compromises in their passive safety, and most small cars from established manufacturers achieve very high scores in a crash test. The main chassis is made from robust box sections of (probably) aluminum tubing and there’s also a tubular roll cage above. Even the suspension wishbones look highly over-engineered for a 25mph city car. A substantial crash box is placed in the front, and above it the whole front panel is made of foam, so the car is not only safe for its passengers but for pedestrians as well. Of course at this stage there are no independent crash tests to verify its performance.

As mentioned above, the most striking aspect of the car is its lack of any kind of manual control. This is a very clever move from Google that aspires to overcome the legal and ethical problems of who should be able to drive or control a self-driving car.

If you completely eliminate any kind of input from the occupants (apart from the destination selection), then anyone on board is defined strictly as a passenger and a question like “should we allow a child to ‘drive’ an autonomous car” is transformed into “should we allow a child to be transported by an autonomous car” — a question that is still probably not very easy to answer, but which is certainly easier than the former one.

Google will launch a small pilot program in California in the next couple of years. The prototypes released on public roads will have manual controls for obvious legal reasons but the goal of this project is to learn and develop the technology and know-how. Read the full post in Google’s official blog here.

It’s worth mentioning that similar concepts and ideas (but not a functional prototype) have been presented over the years. The concept closest to this is ‘UC’ by the pioneering design firm Rinspeed, presented four years ago at the Geneva Motor Show. It was also a two-seater (loosely based on a Fiat 500) but its main feature was the interior, where similarly to Google’s car, no manual controls were present.

A discussion with Ryan Calo on Google, your personal data, and the consumer robotics market

Can someone tell me: What’s Google doing, making robots? Or at least: What is it doing, making such acquisitions?

As you probably heard, in the first two weeks of December last year, they bought some pretty heavy companies (house specialties in parentheses):

• Schaft (Navigation)
• Industrial Perception (Vision)
• Redwood Robotics (Manipulation)
• Meka Robotics (People-safe designs)
• Holomini (Navigation)
• Bot & Dolly (Camera tracking)
• Boston Dynamics (Navigation)
• DeepMind (General AI & abstracted reasoning)

This list covers AI, robotic vision, and a famous four-legged ‘bot that can bust into a sprint that could beat Usain Bolt. So it’s safe to say that the focus is on navigation and vision.  This is nice for the roboticists who are waiting for market validation in the consumer space. Google is providing that. They’re also providing us with a fleet of robots that can navigate, see, and use the network.

What’s Google’s motivation?

Presumably, as a publicly-trade company, this is intended to make Google more money.

They did a pretty good job last year. Sitting between Apple and Coca-Cola, they’re the second most valuable brand in the world. Paid clicks were up approximately 31 percent over 2012, and as of year end, Google employed 47,756 full-time employees. They also, in 2013 had advertising revenues of approximately $50 billion.

Google exchanges personal information for online services. Expanding their horizons from online tools, a few years ago, Google gave Android away to handset makers for the same reason Google gave free web search and email away to you and I: to collect personal information and display other information back. This is Google’s job and their profits show that they are very good at that.

It makes sense, then, that Andy Rubin, who headed up the Android effort, is now heading up Google’s robotics effort. After all, robots – like smartphones – are not just a platform for providing products and services; they are also a platform for collecting data.

Personal information as currency

Nothing is free, so if you aren’t paying cash, you’re paying with your personal information. And that information, from 2013, was worth $50B … so it stands to reason that Google’s motive for getting into robotics is that it offers yet another means of harvesting more information about you. Google will be using robots to collect and display personal information, just like Android did, and before that, email and web searches did. If Rubin has moved on from phones to robots, it’s probably because he thinks robots offer the greater potential in terms of opportunity.

There are three reasons why this makes sense:

1. Robots will give Google access to the physical world for a reduced cost. Like Apple, they are seeking to produce advanced hardware with companies that have proven track records for such manufacturing, and Google is evidently partnering with FoxConn to make this happen.
2. Robots will allow Google access to home and business information with unprecedented fidelity of information. By providing us with robots that are cheap (if not free; think of how smartphones as subsidized), Google can eventually build data conduits into our homes that will allow the company to know what we read, what dog food we buy, what we wear, and when we turn on the Nest thermostat (and, yes, the business model is the same for Nest as it is for robotics: more consumer data).
3. As with an open source OS for smartphones, by producing an open source OS for robots, Google will have direct access to all of the information that Android offered the company.

A couple of weeks ago, while attending RoboMadness at my old stomping grounds of SRI, the CTO and EVP of iRobot, Dr. Paolo Pirjanian, made a curious statement on stage. He outlined a ‘holy trinity’ for robotics as: vision, navigation, and the cloud, and pointed out that, “navigation is key” because it will build environment maps that will allow problem solving. So for example, a 3D environment map will allow the indexing of objects, as well as conversational interfaces for voice and gesture control. If a book is out of place and you want your robot to put it back, then the idea of “back” needs to be indexed somewhere. That somewhere is the cloud, so this information will, then, attach to the cloud.

All that personal data …  all those clouds …

Here we have an example of mapping, navigation and cloud robotics, which, if I try to look at it objectively, sounds a lot like what Google appears to be doing.

Is surveillance the new business model for consumer robotics?

In my desperation to get to the bottom of this I called up Ryan Calo, for he once wrote that “robotics combines, for the first time, the promiscuity of data with the capacity to do physical harm; robotic systems accomplish tasks in ways that cannot be anticipated in advance; and robots increasingly blur the line between person and instrument.”

Professor Calo researches the intersection of law, robotics and the Internet. His work on drones, driverless cars, privacy, and other topics has appeared in many famous publications and he serves on a number of advisory boards, including the Electronic Privacy Information Center (EPIC), the Electronic Frontier Foundation (EFF), and others.

I threw these ideas at him and he took difference with the angle. Here was Calo’s response:

The frame is right, though I would quibble with characterizations of profit – there is a potential for win-win and it’s not necessarily a transgression of privacy. I would love that.  In fact I kind of wish that Google and Facebook would consider creating a robot platform for the home. The robotics industry really needs a personal robot in much the same way as a personal computer. We need something that is very versatile – something we could run all kinds of different code on – a platform or PC that is open to 3rd party innovation.

Then what are these companies that collect our data, like Google or Facebook, up to?

They want to bring in the Internet to more places – to where it is hard to bring it.  Amazon wants robots that can shave seconds, or fractions of a second, in the time it takes to get a package from the warehouse. This is how they’re thinking of using robots.

So in other words, with regards to these companies, so far they are leaving a lot on the table and it may be that, cynically, they end up building this platform that personal robotics needs and their motivation is to get a look inside the home, but that doesn’t seem to be the case. I understand there are things like Nest that give some perspective on the home, so it is a natural fit for Google’s other enterprises. But it is a closed platform that does one thing.

Isn’t that why Google has traditionally offered more services? To get more user information?

No, it’s to get more users. Google doesn’t just rely on users, but also on inventory – it needs a place to display those ads. They want more screens. They created email to bring more people into the fold. But I don’t think Google does everything they do just to get more information. They want to go to advertisers to say, “We have X users” and that increases the value.

You could explain a lot by saying, “They think that smart objects of various kinds are the future and they want to use all these new screens.” Are they building driverless cars to collect personal information? Well, they’ve been doing that with people. Maybe they want drones to fly around for the same reason. But Google appreciates that contemporary life is shifting. Once you have driverless cars. the screen can be as big as you want so that Google can own that screen. If everyone is being driven by cars, maybe Google will have a credible map so that people have to build on that infrastructure. They want to be the ecosystem.

Mr Calo and I have more discussions ahead, but for now, like all good underlings, we’ll just have to keep an eye on the surveillance system to see when it wakes up.

If you liked this article, you may also be interested in:

• DARPA’s Gill Pratt on Google’s robotics investments
• Robots Podcast: Privacy, Google, and big deals
• Google’s robot acquisitions likely cost less than $100 million
• AFK (or, Robotic free speech, and what you can do to help)
• Robot love: Spike Jonze’s new sci-fi film ‘Her’ may be closer to reality than you think
• The death of search (or, My dysfunctional relationship with Siri)

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

When Google bought Boston Dynamics last December, the news made headlines, but it was not the first time the Internet giant has invested in DARPA-funded robotics. As part of Robohub’s Big Deals series, we asked Gill Pratt, Program Manager of DARPA’s Defense Sciences Office, to shed some light on what DARPA thinks about Google’s robotics acquisitions , and what it might mean to the robotics and open source communities.

Congratulations, Gill, on the DARPA Robotics Trials – what was your personal assessment of the event?

I was very pleased with how well the teams performed, and in particular that the hardware for the majority of the teams worked so reliably. In general, the teams did slightly better than we had expected and this was probably the result of a lot of very careful preparation. The infrastructure that we set up also worked reliably in the trials. Still, the DRC Trials were really just a beginning and showed us the potential of what can be done in disaster scenarios with robots.

Cloud computing was an important facet of the DARPA Robotics Challenge and especially the Virtual Robotics Challenge … How do you see cloud computing helping the development of robots?

The importance of cloud computing for Robotics falls into two different categories. The first has to do with simulation environments. We used cloud computing for the first time with the Virtual Robotics Challenge, which we had back in June as a precursor to the DRC Trials, to do real-time simulation of tasks in unstructured environments. We rented a whole lot of cloud computing space, we ran a very good simulator developed for DARPA by the Open Source Robotics Foundation, and we adjusted the latency – the time delay – so that it would be the same for all the teams, even though they were located in different parts of the world. It was a tremendous enabler to run a real-time competition based on simulation that also involves human-robot collaboration. Doing so required that the simulation run in real-time. That was a big first and we felt it was very successful.

Another important aspect of cloud computing is something that we have not used to date, but I think has a lot of potential for the future: the ability to have a robot’s computing and data hosted remotely. There’s a lot of potential for this, in particular for doing perception. Forty percent of the human brain is used for perception, and perception is one of the most difficult tasks for an autonomous robot. It’s very difficult to fit a computer with the size, weight, and power that you need to achieve really good perception onto a robot: the computer just gets too large, it consumes too much power, and it weighs too much. However, if you have access to cloud resources with lots of data and lots of computing cycles, they can help move that burden off of the physical robot.

Perhaps most exciting for the future of cloud computing in robotics is that when one robot learns how to perceive something, or learns how to do a particular task, that learning can be instantly shared with other robots. This sharing could have a catalytic effect on the capabilities of robots, particularly in structured environments.

This is really an up and coming field. How likely are we to see commercial interest come out of cloud computing for robotics?

If you look at the research being done we see a lot of possibilities with cloud computing for robots. In the commercial world, I think we are going to see it applied first to structured environments where there are lots of human artifacts — things like doors, stairs, furniture, tools — and where a large database of stored examples of these artifacts can help the recognition problem.

The Atlas robot, created by Bostron Dynamics and DARPA, was used by several teams in the DRC Trials. Source: DARPA

This is an interview for our Big Deals series, so we’re wondering, what does DARPA think about Google buying up some of the technologies that DARPA has spent years building? We’re thinking of the cars, the humanoids …

We are thrilled to see commercial interest. It’s one of the signs of success for the investments that we have made in future technology.

We think that it takes commercial investment to drive down the cost of technology. A very well known example is the cellphone. Cellphones have microprocessors in them that many years ago began with investments by DARPA’s Microsystems Technology Office and others. Cell phones have inertial measurement units to figure out the tilt of the cellphone. They have displays; they have GPS receivers; they have radio devices that work at low power. I can point to any number of DARPA investments that helped start those technologies, and of course cellphones now talk to the Internet, which is probably the best known output of all of DARPA’s investments. But if the Department of Defense wanted to produce a cellphone without the commercial world having picked it up and turned it into a product that billions of people use, it would cost many orders of magnitude more than it does now. It’s really because of the commercial world that we’ve seen the price of cellphones go down.

But DARPA is not only interested in seeding technologies with these early investments; we are also committed to solving real problems for national security. In fact, we have a variety of programs that use cellphones. One of these is called TransApps, where soldiers use smart phones to plan and carry out missions and layer and share data between teams. This is an example where investment by the commercial world made possible a national security capability that otherwise might not be feasible because of excessive cost.

By analogy, what’s going on in robotics is that we’ve seen some interest in the commercial world in taking this to the next step. And they have resources far beyond what DARPA has to further develop a technology and, most importantly, to drive down cost.

So these big deals are a good thing for DARPA?

Yes, they are a wonderful thing for DARPA and, more broadly, for our nation.

Do you think it’s a good thing for the field of robotics in general?

Absolutely, but I understand why there’s some anxiety.  It’s true that, for a very short time, some talented people in the field will be working for a commercial firm rather than for DARPA. However, that effect is transient and very small compared to the much more important thing, which is that these corporate investments both accelerate the field and show that it is going to be real.

This anxiety is a lot like the kind you get when your kids leave home. It’s sad, and for a while you’re going to miss them, but wait a while and then the grandkids come. The important thing is for Robotics to become a field that is not only about prototypes or lab demonstrations, but a field where significant resources are invested to make products that really improve people’s lives. We saw it with the Roomba vacuum cleaner, but we want to go beyond that. These kinds of commercial investments will make the field far more attractive to startups and to students who are planning their careers. The net effect is strongly positive.

How do you think this corporate interest in robotics will impact the open source community?

Open source is very important. It allows us to develop pre-competitive catalysts to help the whole field move forward.

In the integrated circuits world it was recognized that you couldn’t build experimental integrated circuits as easily as you could wire up a circuit with discrete components like capacitors and resistors and transistors that you solder by hand. As a result, people working in the area at that time needed to develop a simulator to allow for the experimentation of how a chip would work without actually having to make the chip. The idea was to do the design and the debug cycle on the computer in simulation instead of having to do it on the bench: when you were done you could press a button and have confidence that the chip would actually work. The open-source simulator SPICE filled that niche.

The very same kind of thing is true in the robotics field. The key is to recognize that simulation is really a pre-competitive technology now.  We don’t need to have proprietary closed simulation systems and keep reinventing the simulator over and over again. We’ve done it enough times now to know that it would be useful to apply open source. In the same way, I think that an operating system is precompetitive – it has been done enough times that a new company should not try to distinguish itself based on a proprietary version of an OS.

Rather, let’s just move on to the next thing where the real innovation is going to occur. DARPA funded the Open Source Robotics Foundation to develop an open source simulator for the DARPA Robotics Challenge because we think that it will catalyze the whole field to everyone’s advantage.

With the purchase of Boston Dynamics, some headlines were saying that Google bought military robotics. Is this an accurate assessment?

I think it’s very inaccurate. It’s based on the confusion – and the conflation – of two aspects of robotics: remote bodies and remote brains. A robot allows you to have a mechanical body take the place of a human being; for example, a robot can work in a dangerous environment while a human supervisor is in a safe place. And a robot also allows you to perform some of the function of the human brain autonomously in a remote location. When you conflate bodies and brains, and say that there is only one idea here, it’s like saying, “The robot looks like person, so it must be as intelligent as a person, and therefore I should be scared of all robots, particularly those funded by the Department of Defense.”

All of the robots that I’m aware of, except for the ones that actually have weapons or specialized sensors on them, are generic. They are neither military nor commercial. They are just robots that have mobility and perhaps some manipulation, but there are not classed to one type or the other. And they are all almost completely empty-headed: their bodies may look like ours, but their heads are almost totally empty.

In terms of autonomy, the technology has such a long way to go that the fears that have been generated are way out of proportion to the state of the art. At the DRC Trials, for example, we had people supervising the robots in pretty tight loops, telling them what to do. While autonomy may be able to handle very simple environments, close human supervision is going to be needed in unstructured environments for some time.

What do you say to people who are uncomfortable with the use of military robots in a commercial context?

I think there are legitimate concerns, but from a rational point of view it’s important to distinguish between a weapon, which none of the robots we are talking about are, and a machine that makes us feel scared because it looks like us but it’s hard to understand exactly how it works. Of course, science fiction has stoked that fear with movies like Terminator and so forth, but that reaction actually goes back a long, long way to stories like Frankenstein – anything that sort of looks like a person but is hard to understand will generate fear. It’s an emotional reaction. However, from a rational perspective, if you consider how little autonomy these real machines actually have, you realize that the worry is way out of proportion to reality.

What are some of the benefits that could come out of combining Google’s expertise in data with hardware from Boston Dynamics?

I don’t know the specific applications that Google is thinking of – it’s their prerogative to discuss them or not, and they haven’t told DARPA.

Generally speaking, when you think about data and robotics, there are a lot of exciting things that could happen. For example, you could do perception better by using a lot of data and sophisticated search. This is not necessarily the kind of search we think of when we type words into our browsers, but the kind that involves images, where a robot could say, “Okay, I see something and because of many prior examples I know what to do,” and share that data in a useful way.

Would voice commands play into this dynamic, in addition to visual search? Would it make it easier for humans to interface with robots?

I’m not an expert on voice, so I don’t want to speculate, but I know that you can train better when you have a lot of data. Voice and natural language recognition have the potential to help with planning and perception, and I think they could also facilitate more intuitive interfaces for sharing information between robots and humans.

Team SCHAFT raises the arms of its S-One robot in victory after successfully completing the Climb Industrial Ladder task at the DRC Trials. SCHAFT won that task and three others, and scored the most points of any team at the event. Source: DARPA.

What new robotic technologies will DARPA be pushing that might appeal to big companies that are not traditionally robotics focused?

We’re really interested in cooperation directly between robots and between human beings and machines. We are also interested in dealing with dynamic environments. What I would like to do for the future of the DRC, and what I think that DARPA is going to push even beyond that, is to answer questions like “What if the communication is intermittent?” and “What if the environment is beyond a set of familiar things that I’ve seen before?” … What do we do if we’re actually outdoors? How do we handle this piece of vegetation that’s in front of us? Do we go through it or around it? If we go through it, do we move the branches out of the way?

We also still have a lot of research to do to figure out how to apply cloud computing when the environment is unstructured, like it would be in disasters or outdoor scenarios.  And if the communications are intermittent, we have to figure out how to cache enough of the intelligence on the robot itself so that it can continue to do what it needs to even when it’s disconnected from the cloud. Of course, disconnected operation may be at a reduced level of effectiveness, but we don’t want the robot to stop altogether when communications go down for a little while. We want the robot to have at least some ability to complete the tasks it was given to do.

How do you actually put behavior in the cloud? How do you take a data driven approach to behavior? I think that’s very much an unknown thing right now. Data driven approaches to perception are easier to understand, but we are going to try to take the next step and look at behavior as well.

If you liked this article, you may also be interested in:

• Snowden BEAMs into TED: How robotic telepresence disrupts borders
• Google’s robot and artificial intelligence acquisitions are anything but scary
• Will companies like Google and Amazon usher in a new era for robotics?
• The robots behind your Internet shopping addiction: How your wallet is driving retail automation
• What do Google, Amazon, IBM, Apple, Microsoft and all the airlines and car companies want?
• Robots Podcast: Privacy, Google, and big deals
• Robots, soldiers and cyborgs: The future of warfare

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Henrik A. Schunk, Managing Partner of SCHUNK GmbH & Co. KG, and Chairman of EUnited Robotics – European Robotics Association

The continuing interest of Google in robotics over the last years and its newest activities in this regard confirms what all players active in robotics will confirm: service robotics is on the threshold of entering a new maturity level. Service robotics conquers new, commercial fields of application and is just becoming an independent industrial sector. Where in the past the focus has often been on research, now new companies and applications are emerging. Research projects have been taken out of the labs and turned into real companies and solutions, new companies and players have entered the game with Google being among them.

Considering that robotics is still a relatively young industry (in 1962, General Motors installed the world’s first robot in a production line!), we have certainly come a long way. According to the latest available IFR information, about 168,000 industrial robots were sold in 2013. In the service robotics segments, the sales figures are significantly higher, at least in units: three million service robotics for personal and domestic use and 16,100 service robots for professional use were sold worldwide. Growth rates of 10% per annum on average since 2006 added to the turnover of professional service robotics, which reached €2.5 billion in 2012.

And much more is yet to come. The International Federation of Robotics (IFR) assumes that between 2013 and 2016 at least 95,000 new professional service robots will be sold worldwide, worth €12.3 billion. Service robotics is definitely a market of the future. So, it seems that what Bill Gates predicted in 2007 is becoming true: the next “hot field” will be robotics. Though it is not yet clear when we will have, in Gates’ words, “A Robot in Every Home” or even a robot in every factory (including in SMEs!), it is definitely apparent that robotics has gained momentum over the years.

To Google and all the other new players and investors in the robotics field, I would like to say, “Welcome to an exciting world and a growing market!” But to all stakeholders in robotics – be it old or new players, investors or enthusiasts alike – I would add: be patient and don’t expect too much too soon. It is not yet clear when robots will become part of our every day lives – but we are on our way!

Greetings from Europe!

Link to audio file (33:53)

What does it mean to have giants like Google, Apple and Amazon investing in robotics? Since last December, Google alone has acquired a handful of companies in robotics, home automation and artificial intelligence. This can be pretty exciting for robotics. But what exactly is the internet giant planning to do with this technology? Is there something we should be worried about? If there is, what can we do about it?

Experts have been actively talking about this in the media, including through Robohub’s recent focus series on Big Deals in Robotics.

In today’s episode, AJung Moon talks with Avner Levin, a privacy and cybercrime expert, about Google’s recent acquisition of companies. Levin sheds light on why we should be concerned about the recent series of acquisitions by the big companies from privacy and cybercrime perspective. He also discusses whether the existing privacy policies are ready to handle what may lie shortly ahead of us in the future — the future of the Internet of Things, or perhaps Google-branded robots.

Avner Levin
Avner Levin is the Chair of the Law & Business Department, as well as the Director of the Privacy and Cyber Crime Institute at Ryerson University in Toronto, Canada. His research interest is in regulation and protection of privacy with respect to technology within Canada and worldwide. In the interview, he advocates for more discussion of privacy issues to take place, not just within the companies that (do/will) hold our data, but by governing bodies.

Links:

• Download mp3 (15.5MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Avner Levin’s Website

 

If you liked this article, you may also be interested in:

• Google’s robot and artificial intelligence acquisitions are anything but scary
• Will companies like Google and Amazon usher in a new era for robotics?
• The robots behind your Internet shopping addiction: How your wallet is driving retail automation
• What do Google, Amazon, IBM, Apple, Microsoft and all the airlines and car companies want?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

 

What is one to make of this focus series … Big Deals: What It Means to Have the Giants Investing in Robotics … with the giants being Google, Amazon and Apple (as given in the preamble)? The assumption, of course, is that robotics investments by these companies represent a turning point in the evolution of robotics — perhaps something wholly different, outside the normal ebb and flow of robotics investments, acquisitions, and mergers. It is also assumed that the investments represent some type of trend, or at least have something in common.

The Google, Apple and Amazon robotics investments do share much.  First, each of these investing companies are considered large, particularly when measured by their market value. The market cap of Apple, Google and Amazon is approximately $468B, $409B and $164B, respectively. All three are Fortune 500 companies (For 2013, Apple 6, Amazon 49 and Google 55), and are listed on the Forbes Global 2000 (For 2013, Apple 15, Google 68 and Amazon 525).

Each of these large companies is also considered innovative and forward thinking, capabilities usually associated with smaller, entrepreneurial firms. But this can also be said of many other large enterprises, although “innovation” per se can be difficult to gauge (unlike market cap, sales revenue and profitability) and the ranking process is usually biased in some way. A comparison of the Forbes list of the world’s most innovative companies with that of Fast Company, and again with Thomson Reuters’ Top 100 Global Innovators (Amazon not even listed), demonstrates the difficulty of quantifying innovation, and provides a great example of expectation bias at work, especially on part of Fast Company.

There is no disagreement that Google, Apple and Amazon are some of the most recognized brands in the world (see here and here). It is on the strength of their respective brands … in a way a measure of their cultural standing … that the announcements of the companies’ robotics initiatives and acquisitions made such an impact. That each company has made a substantial investment in robotics technologies fired the imagination of the public and invigorated the robotics community (business and technology media outlets went positively orgasmic!). That’s a good thing — a very good thing — in ways too numerous to list here. At the very least (and this might be the most important aspect of the various deals) the Google, Amazon and Apple robotics initiatives have legitimized and substantiated robotics for businesses, entrepreneurs and investors in a way that Mars rovers and military drones cannot.

I would argue that the Google, Apple and Amazon investments do not herald a new robotics era, or even a new overarching trend. Instead, they are the logical consequence of robotics technologies becoming ever more powerful (even as they become less expensive), and being exploited by companies to further business aims. The investments also follow the typical usage progression for advanced technologies beginning with research, then moving on to the government/military, then on to business, and ending with consumers. Google, Apple and Amazon are not consumers in the general sense, but they are consumer-facing (mostly), which is another point of commonality.

Google, Apple and Amazon have very different business models, and their robotics acquisitions and initiatives reflect this and should be viewed in this light. Apple, as a designer and manufacturer of computers, tablets, and mobile phones, is investing in robotics to increase productivity during manufacturing, reduce costs and improve quality in the process. According to Apple’s annual 10-K report, the company anticipates using approximately $10.5B in capital expenditures (CapEx) for investments such as new tooling and manufacturing equipment, including assembly robots and high-precision milling machines. The technology will be installed in Chinese manufacturing companies such as Foxconn and Pegatron, as well as in facilities located in the US and elsewhere. In this way, Apple is using its vast cash holdings as a competitive weapon as it battles competitors (for example, Microsoft and Sony for tablets, and Samsung, LG, ZTE and Huawei for smartphones).

For ecommerce retailer Amazon, its purchase of Kiva Technologies for $775M in cash in April 2012 (almost two years ago, further eroding the trend argument), was also made to increase productivity and reduce costs, but in this case it was for fulfillment operations and not manufacturing. At the time of the acquisition, Amazon was cash rich, but fulfillment costs as a percentage of revenue were steadily increasing. The Kiva technology has proven to reduce the cost of pick-pack-ship operations and increase throughput at distribution centers, boosting profits in the process. Applied across Amazon’s many distribution and fulfillment centers, Kiva MPS would have a significant positive effect on the company’s profitability (something Amazon sorely lacks even as revenues increase).

Amazon was also in the news following an announcement on CBSs’ 60 Minutes program that the company was evaluating the use of drone to deliver packages to Amazon customers. The gadget press went wild, but the announcement should be seen for what it was… a swarmy bit of self-promotion and boosterism, which coincidently came just before the announcement of Google’s robotics acquisitions (see below).

Google generates the majority of its revenues selling Internet advertising, and is rapidly acquiring new companies and developing new technologies to broaden its offerings and reduce its dependence on a single revenue source. While not indicative of a general trend, Google’s announcement that it had acquired eight robotics and artificial intelligence technology start-ups as part of a major new robotics initiative was a watershed moment. Google purchased a total of eight privately held robotics companies for an undisclosed amount (estimated at only $80-$100M). These include Industrial Perception (IPI), Meka Robotics, Redwood Robotics, Bot & Dolly, Autofuss, Holomni, Schaft, and lastly, Boston Dynamics.

The initial robotics announcements were followed in quick succession by Google’s purchase of home automation maker Nest Labs and Deepmind, an artificial intelligence start-up. The Nest acquisition will immediately drive revenue, but near term contributions by the other acquisitions are not clear. In media outlets and the blogosphere speculation was rampant, and ranged from the well-founded, to the speculative, and on to the absurd (although great reads all).

What is known is that of the eight purchased companies (really only five), none were public or substantial in size (employees, revenues etc.). Also, none had developed commercial-class products, or delivered robotic solutions targeted to mainstream, sizable markets (research robots, prototypes only). What Google did purchase were pieces that can serve as the generative component of a larger robotics solutions stack. Google’s acquisitions laid the groundwork for the company to become a robotics platform provider and industry accelerant. Software, including Robot Operating System (ROS) and Android, is key. The capabilities required to create new applications and products, whether by Google or others, is more important than the recent robotics acquisitions themselves.

In summation, Google, Apple and Amazon’s robotics acquisitions and initiatives should not be seen as a turning point, but as part of a building wave of robotic technology proliferation into the workplace, public places and the home. This does not diminish the impact of the purchases and investments, and in fact increases their importance to the larger robotics community by continuing to normalize robotics technology.

As their brand strength indicates, Google, Apple and Amazon have a cultural impact that exceeds that of other large, innovative companies with substantial robotics investments (for example, Rio Tinto for mining, GE for the industrial Internet and Boeing for unmanned aerial vehicles etc.). Through their acquisition and initiatives, Google, Apple and Amazon have “blessed” robotics, and in the process dramatically increased the profile and legitimacy of robots and robotic technology in the eyes of the business and investment community, especially the legitimacy of robotics as an investment opportunity. The acquisitions themselves demonstrated a clean, quick exit strategy and software-like ROI to a financial community that is often hesitant to invest in far-horizon, hardware-centric robotics start-ups.

If you liked this article, you may also be interested in:

• Facebook plans to offer Internet service using aerial robots
• Google’s robot and artificial intelligence acquisitions are anything but scary
• Individual investors: Take a cue from corporate capital … the time is right for robotics
• Will companies like Google and Amazon usher in a new era for robotics?
• Big deals: What it means to have the giants investing in robotics

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

[AK] With investments in two accelerators now (Bolt in US and Navigator Campus in Russia), as in well as a range of robotics companies, Grishin Robotics has a bird’s eye view of robot startups. What do you think about the recent high profile announcements about robotics investments … how will this impact the start-up scene? 

[VK] It’s definitely great news. Amazon, for example, announced in a very significant way their interest in robotics a few years ago when they bought Kiva for $775 million. So this isn’t the first time that big corporations not traditionally involved in robotics are starting to pay attention to the field. It really  begun at least in 2012, when Grishin started to invest into robotics startups as well.

However, this time it’s even more about perception. These high profile media announcements are getting picked up by the general public, investors, and entrepreneurs – they attract attention to the industry. No doubts, it is important from an entrepreneurial perspective – there should be more success stories for them, more ‘benchmarks’.

These days many talented mechanical and electrical engineering students and professionals (particularly in the Bay Area) are attracted by the large salaries being offered by big internet companies like Facebook and therefore decide to switch into software and postpone their startup, hardware and robotics dreams indefinitely.

But when they hear these big announcements about new products or M&As, they realize the market is actually happening and they begin to believe that they can start their own company … they don’t have to go work for a big corporation. They can continue to work in a profession that they like and have expertise in. So from that perspective, these announcements are very important.

From the investors’ perspective, it’s largely the same. They need inspiration, they need proof, they need exits, and they need public companies. Of course there are already public companies like iRobot , but still we don’t hear about robotics acquisitions or IPOs every day – not yet. The announcements of the Amazon drones and the Google acquisitions of Nest and other companies really attracted the attention of the venture and finance communities and reminded them that something big is happening. It forced them to pay attention … it forced them to think that maybe another one or two years is too long to wait to make their bet on this space. And that’s really great news.

Google have invested in quite an assortment of robot startups, particularly if you include their investments and resources in AI and NLP. What do you think their plans are?

Interesting question to speculate on. Google itself said that their primary interest at this point lies in the area of manufacturing. But the acquisition of Nest is definitely a consumer play. Frankly, at this point, I think, it may be not so much about any coherent behind-the-scenes grand strategy as much as that they are starting to feel like something big is going to happen to follow the revolutions that came with the personal computer, the internet and the smart phone. Many people are seeing signs that the next big revolution may be robotics.

For Google, Amazon etc. this may be more about announcing some kind of interest in this space and to jump into the game as early as possible. I think there’s a lot of R&D going on internally to figure out where and how exactly to go and approach it.

So while it’s definitely exciting times and more and more startups are launching with interesting ideas, there are still some unknown variables. It’s important for big corporations like Google to figure out the unknowns in this multi-billion dollar market as early as possible, and to do this before the competition does.

Do you see any down side to large companies getting involved in robotics?

Despite all these announcements the market is still at an early stage of development, so the really fierce competition is ahead. It’s not that the market is divided between big companies, after all these news – far from that. Just as a comparison … it’s difficult to launch another smart phone in the market today because the market is fairly mature and is divided between the big companies. But in the robotics market, and in the consumer robotics market specifically, it’s the opposite.

The barrier of entry for robotics startups has been getting lower and lower the last few years. And all these announcements lead to increased investor interest, which means more money for robotics and hardware startups … more investors ready to consider these kinds of companies in addition to the traditional Internet and mobile companies. I just can’t think of any significant downside – at least not at this point and not for the next two or three years.

Possible problems could include losing top talent, or losing access to components or research, or support for community and open source projects, which all have impact on research and startups …

All good points and let’s consider them one by one.

Regarding top talent, all these companies that were bought are fairly small. I deeply respect all these people and I know many of them, but I hope these acquisitions will inspire more robot students or engineers to go out there and start their robotics companies, or it will inspire younger students to select robotics when they pick their majors at universities. I think that the cumulative positive impact will be so much greater than the loss of these people to big corporations.

I also agree that it’s startups that are primarily innovating in new markets rather than the big companies … but it’s not like these people won’t continue to do research. And Google has a lot of money and resources and these people will continue their work with other talents they meet there. It’s very possible that these talented people will be able to accomplish more given the vast resources they’ll have access to inside these big companies. So I urge you consider not just the negative side of this, but the  positive sides as well – word ‘loss’ has too much of a negative connotation, in my opinion.

The second concern you mention is lack of access to components …

Yes, to components, ROS, research findings…

Not necessarily. When big company is buying component technology, of course, it often ceases to be open-source. Discussion of open-source ideals are wholly different topic, but my point is that it is not necessarily means decreased access – may be quite the opposite. Often there is an interest of a big company to make certain technology a dominant platform, therefore, great investments are made to attract more developers to it, improve it, explain benefits etc. And if it will turn out not to be the best platform – competitors or same open-source community will be quick to offer alternatives, especially, again, at this stage of market development, when barriers to entry are quite low.

If you could ask Andy Rubin any question what would it be?

One of the main challenges for consumer robotics market – whether we are talking about connected devices and the Internet of Things or more traditional consumer robotics like Roomba – it pretty much comes down to applications and business models. We can imagine some of the interesting applications or verticals where robotics could make a significant impact, but we can’t always imagine all the possible  areas and ways where robotics can be applied to make our lives better – there are just so many of them, and many are far from being obvious. So I would be interested to know how he thinks robotics can make our lives better in a significant fashion, beyond the usual suspects we already know about, like agriculture, drones etc.

And the same question about business models can be asked from a different perspective: at the moment hardware companies are selling hardware, but it’s getting commoditized … and you have to think about software and data and the cloud connectivity of the hardware components you are making. So, how can business models that are familiar to Google (like advertising, subscriptions, internet) be applied to robotics and consumer hardware companies? And of course, what kinds of new business models will emerge as a result?

We’ve talked about Google and Amazon. Who are the other big new players in the robotics space that are known for things other than robotics but are starting to have an impact in the field …

There may be many more than we are aware of. Like Tesla – of course it’s an automobile company but more and more its top management is starting to say that they are essentially an IT company, considering how connected the car is and how advanced the driving functions and so on. And the same goes with the other automobile companies that are working on self-driving cars.

And the problems they are solving, whether they are BMW or someone else, are very related to the problems the robotics industry in general is facing right now. The advancement of computer vision is basically what the whole robotics industry is reliant on in terms of future growth and potential. Also autonomous capabilities … We’re not talking about the holy grail of full autonomy any time in the foreseeable future, but the algorithms and the decision-making processes needed for these new (semi) autonomous cars may spill over into the startup world tomorrow. Eventually it will impact the whole market in a very significant and positive way.

Many of the big corporations today may essentially be involved in robotics already. For example Foxconn and other manufacturing companies are starting to use robots. And more and more companies in agriculture and mining are beginning to use unmanned vehicles. They are starting to understand that adopting these technologies earlier than their competitors is a necessity if they are going to survive. Robotic capabilities are becoming the basis of their long-term competitive advantage.

To make a comparison, almost everyone involved in physical commerce today is finding that their internet capabilities are getting closer and closer to the heart of their business. I think we will find the same thing happening with robotics. It’s not so much about the specific companies’ names we can come up with today … because in the future it will be almost everybody, just like how almost everybody these days is involved in the internet or is using software or computers.

What do you think are the most interesting recent robotics acquisitions?

It depends on how you define robotics. Is Nest as a robotics company? I for one define it as a consumer robotics company but I can imagine that many people will not agree with me. I believe everybody will eventually agree with this definition but not maybe today. I think the Deep Mind announcement is also very interesting. AI and related software is a very important enablers of robotics.

What do you think about Dyson’s announcement of the $8million (5 million pound) UK research facility specifically into home robots?

It’s an awesome announcement. Dyson himself is an iconic figure, as well as his company. They have been making products that we are used to seeing both in the home and the office, and they’ve been able to make them differently, both in terms of design and in terms of capability. And it’s great when such a company pays significant attention to robotics.

The thing is – it’s not enough to make a good-looking coffee maker or washing machine anymore because people are getting more and more used to smart devices in their homes and cars. Increasingly they are expecting smart capabilities for the same amount of money. And it is possible now to deliver a much smarter device – with Internet connectivity – for almost the same price as a so-called ‘dumb device’.

All this attention is just another confirmation that very soon the fact that you are making smart devices with robotic capabilities will stop to be the competitive advantage and will start becoming the threshold capability. I think Dyson is foreseeing this (probably somewhat ahead of their competitors) – but  eventually everyone will follow.

Grishin Robotics is investing in consumer robotics… Do the recent announcements change any of your plans? 

No. There are a lot of very interesting application areas and verticals in the robotics market, whether it’s health care, agriculture, or manufacturing, but Grishin focuses on consumer robotics simply because we see huge potential, both from the market perspective and from the perspective of having a positive impact on people’s lives. This potential is much greater when we’re talking about robotic products that the average person can buy for $100-$200, and when there are so many prospective customers out there. It’s really all about impact. Big corporations and governments have been benefiting from robotics for so long, but average people have yet to benefit. We think it’s time to change that.

You could say, then, that the recent announcements have validated Grishin Robotics’ interests …

Exactly. In June 2012, when Dmitry Grishin launched the firm, a fair number of people told us that we were absolutely crazy. When so many smart people are telling you that you’re crazy, you have to think that you may not necessarily be right, but at least you’re probably doing something interesting. But when in the end it turns out that you are even may be right … it does feel good.

 [Edited by Hallie Siegel]

If you liked this post, you may also be interested in:

• Robot Launch 2014: The first global startup competition for robotics is here!
• Doggone it, there are no exciting consumer robots
• Dmitry Grishin on evaluating robotic investment opportunities
• Accelerators and opportunities for robotics startups
• Robots Podcast: Grishin Robotics

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

When you’re shopping for the best online deals you’re probably not thinking much about the massive distribution network required to bring that pair of shoes to your doorstep. Is your quest for the best possible deal helping to usher in the next wave of automation?

Internet giants like Amazon and Google have an obligation to stay competitive and are going head-to-head over the online shopping space. Customers choose to purchase the lowest priced product, without loyalty to any online e-tailer, and expect fast shipping and handling. Amazon and Google appear to have two strategies for staying competitive: improving their search experience and decreasing inefficiencies in their supply chain. For the latter strategy, both companies have publicized their interest in robotics – and have invested heavily.

The manufacturing sector has relied on robotics and automation for several decades. In many sectors, productivity has increased while employment levels diverged, leading many people to assume “robots kill jobs.” Technology has been displacing workers for the past century, but I have not heard of a customer who has voluntarily purchased a more expensive product in order to reward a company for their inefficient operations. Robotics and automation are the link between data and the physical world, and automation technology is required for the production of complex products, but the hypocrisy between “having our cake, and eating it too” has put robot companies in an awkward position: automation has advanced the human race with wonderful products, but many people say they would rather throw that productivity away in order to keep their jobs.

Humans live in a physical world: we touch, hold, and eat physical things – so society needs to accept that as long as we demand inexpensive goods and services, companies will choose to automate a process the moment a technology becomes available. This strive for efficiency is constant, ongoing, and cannot be stopped  – even if jobs are displaced in the process.

New robotics companies are finding it difficult to attract investors – especially if their robot can’t be put under a Christmas tree. The online storefronts that Silicon Valley investors have fallen for are subject to market commoditization, and the only way to compete is to make the physical world, production and distribution, more efficient. Due to rising wages and physical limitations on the human body, this cannot be achieved with pure manual labor. In my experience, investors have noticed Amazon’s acquisition of Kiva Systems and are beginning to realize that the backbone of internet shopping lies with automating factories and warehouses.

According to a recent New York Times article: “Google’s robotics effort is not something aimed at consumers. Instead, the company’s expected targets are in manufacturing — like electronics assembly, which is now largely manual — and competing with companies like Amazon in retailing, according to several people with specific knowledge of the project.” In fact, Google just announced a partnership with Foxconn for electronics assembly.

Fuel and labor costs are having a direct effect on Amazon Prime’s pricing, and Amazon announced a rise in prices that provoked a negative reaction by shareholders. I stretch to suggest a large percentage of customers who enjoy Amazon Prime’s free or inexpensive shipping also complain about robots taking jobs – a counter-productive opinion for a society that wants lower priced products but cannot accept the employment ramifications.

Amazon and Google are in competition and both will use robots for a more efficient supply chain. All companies will automate processes when the appropriate technology becomes available. Technical improvements are constantly sought after, manual labor is not, so companies will always demand robots as an alternative to having a less efficient workforce. Folks who choose to educate themselves and gain the necessary skills will always find a job. My wish for all people is they get to spend their lives instigating the projects they find most meaningful and do what people (not robots) are best at: imagining a better world, designing the future, and making amazing things happen.

Perhaps when Google and Amazon has elimiated shipping and handling costs, we can find some room in our wallet to donate $1 of every internet purchase back to our education system.

If you liked this post, you may also be interested in:

• Big deals: What it means to have the giants investing in robotics
• 7 consumer robots to look out for from CES 2014
• The economic impact of the robotic revolution
• The death of search (or, My dysfunctional relationship with Siri)
• Foxconn and Google cooperating to make robots

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Recent eye-popping acquisitions of robotics technology companies by giants like Google and Amazon might suggest that the automated age is upon us. Is it? Why now? Are these the companies that will usher in the commercial robotics and drone era? And will the little guy continue to find a place in the new epoch? While the excitement surrounding all of this suggests that a huge movement is afoot, those in the community should be relieved that we are finally seeing any activity.

The investment situation in robotics has been abysmal for years, and the affirmation of momentum is real and inspiring. The natural perception is that the movement is real, but the market has much to prove. Of course robotics has a place in the future, but modern history has seen the bulk of activity within academia and professors chasing the next grant.  However, big companies positioning long term for the inevitable opportunity is far from unprecedented. For instance, as far back as 1998, Honda shocked the world by demonstrating ASIMO, a robot capable of advanced gait, climbing stairs, communicating, and more … with limited revenue positioning whatsoever. Seeking examples of large companies that have invested in robotics infrastructure is easy – look no further than KUKA, Yaskawa, FUNUC, Rethink, Sony, and Intuitive. The massive payoff for many of these investments has yet to arrive.

Perhaps that gives insight in the state of affairs at Google and Amazon with multiple robotics and drone acquisitions in the recent years. Understanding the emerging market opportunity is sci-fi easy. Recognizing when the market will hit is not, especially when so many companies have yet to capitalize despite decades of huge R&D costs. Recent acquisitions are coming from capital rich companies, and these companies are primed to build the infrastructure required and wait this lucrative market out. Google’s investment in Boston Dynamics has this quality. Amazon’s investment in Kiva had a dramatic infrastructure feel.

While I am hesitant to say that the robotics and drone commercial markets will affect the average person’s day-to-day activity tomorrow despite the renewed activity, the feel now is that the climate has noticeably improved for adoption. For instance, in the commercial drone sector, Amazon provided a great community service with its 60 Minutes showcase highlighting its proposed Prime Air delivery service. A year straight of negative privacy press was diluted when Amazon suggested a very tangible and real use of drones that the average person could digest – one that comes with a huge economic upside. While Amazon suggested they could have the system up in 3 – 4 years, I think most of us in the community expect a much more modest timeline. However, the good that came from the announcement has the community buzzing again after a relatively stagnant period of waiting for the air space for commercial drones to open. The last year of robotics activity seems to be in lock step with this changing drone perception.

Even if the tipping point for the commercial market for robotics and drones may yet take some time to arrive, the advancement to this moment is noticeably accelerated. For ubiquitous application, robotics and drones require significant infrastructure when considering use alongside people and existing systems.

Little has done more for the community then the evolution of layered robotics operating systems, SDKs, and applications that have contributed to the powerful offering available to the community so dependent on infrastructure and smart solutions. Sensors are also at the price point where robotics and drone systems can be competitively priced. Seemingly, the prerequisite ingredients for adoption have arrived, and companies recognize this. Perhaps this is another major reason that we are starting up the proverbial curve.

Rather than thinking that the big players are going to squeeze out the little ones, I expect the opposite. With so much infrastructure and so many markets soon to seek robotic and drone technology, the corporate acquisitions are probably just starting. Prospects are great for the small business providing a part of the overall solution. Positioning an offering in this community makes sense. Much change and adoption is ahead, and this will take time. But the time is probably right to get in if you are going to and have your offering be relevant when it counts.

In essence, the window is not completely wide open for commercial robotics and drones markets, but it is opening. And like anything, you need to get there early. I think the 5 AM alarm is going off.

If you liked this article, you may also be interested in:

• New ETF (NASDAQ:ROBO) focuses on robotics industry
• Robo-Stox: A dream come true
• Forget the octocopter, let’s talk about Kiva

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In a Wall Street Journal story by Lorraine Luk, Foxconn Chairman Terry Gou met with Google Robotics Division Andy Rubin in Taipei recently and they discussed new robotic technologies. Gou was excited by new automation technologies demonstrated by Rubin. Rubin asked Gou to help integrate the robotic companies that Google acquired to target the areas of manufacturing and electronics assembly that Google plans for their future.

Foxconn and Google are calling this collaboration (which has been going on since last year) a “cooperation.” The Wall Street Journal said that the cooperation was focused on improving robotic automation to compete with companies like Amazon in retailing.

“Foxconn needs Google’s help to step up automation at its factories as the company has the lowest sales per employee among the contract makers, given its large workforce,” said Wanli Wang, an analyst at CIMB Securities. “Using robots to replace human workers would be the next big thing in the technology industry. Not just Google, other major technology companies such as Microsoft and Amazon also have been developing robotics technology to capture future growth opportunities.”

Foxconn has sent some engineers to Massachusetts Institute of Technology to learn the latest manufacturing and automation technology. The company has been branching out from just being a contract manufacturer by selling various mobile accessories it has manufactured. Last August, Google purchased a number of display-related patents from Foxconn including head-mounted technology enabling superimposing of images a la Google Glass. Although the patent purchase was likely a safeguarding step to beef up Google’s patent portfolio as they near coming to market with Google Glass, it was also the beginning of a relationship between the two companies. And early 2013, Foxconn launced their new robotics research facility in Taiwan.

Source: Factory picture by Techspot.com; facetious after image by Tommytoy.typepad.com

If the world outside the robotics community didn’t know about Boston Dynamics, Kiva and Nest, they do now. Recent robotics acquisitions and investments by major-league players like Google, Apple and Amazon have generated a blitz of headlines in the robotics world and beyond. Are we witnessing a power play in the making? What does it mean for the future of robotics? And is all the hype beneficial or harmful to the robotics community?

This month’s Robotics by Invitation will serve as a launch for Robohub’s newest focus series on how big time corporate attention effects the culture of robotics. In the coming weeks we will be bringing you insight from the likes of Steve Cousins, Dan Kara, Valery Komissarova, Avner Levin, Chad Partridge, Gill Pratt, Erin Rapacki, Frank Tobe, and Rob Wilson.

Today, RBI panelists Mark Stephen Meadows, AJung Moon and Alan Winfield weigh in …

Alan Winfield on “What does it mean to have giants like Google, Apple and Amazon investing in robotics?”

Judging by the levels of media coverage and frenzied speculation that has followed each acquisition, the short answer to what does it mean is: endless press exposure. I almost wrote ‘priceless exposure’ but then these are companies with very deep pockets; nevertheless the advertising value equivalent must be very high indeed. The coverage really illustrates the fact that these companies have achieved celebrity status …

Read more →

Mark Stephen Meadows on “What does it mean to have giants like Google, Apple and Amazon investing in robotics?”

Google, is the wild card for me.  With more acquisitions than Apple, Amazon, Facebook, and Microsoft combined, the GOOG looks to be rigging up a kit that would offer excellent image recognition + navigation + mobility.   A robot that can roam around your home or maybe your city, and offer a helpful service. My bet is that these Googlebots (perhaps they will be cars) will have sharp eyes, simple yet helpful demeanors, and we’ll trade that robotic service for our personal info …

Read more →

AJung Moon on “What does it mean to have giants like Google, Apple and Amazon investing in robotics?”

It’s exciting for the robotics community that the giants (Google, Apple, and Amazon) are actively investing in robotics. Large scale investments by giant corporations, regardless of what their actual plans are, can have a positive impact on the job market for roboticists, increase competition between the giants and push robotics forward. But I began to feel uneasy when Google acquired Boston Dynamics …

Read more →

If you liked this post, you may also be interested in:

• Robohub focus: Dealing with the ‘jobs’ question
• Robohub focus on frontier robotics: High risks, high rewards
• Robotics by Invitation: Is open source a good business model for robotics?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Judging by the levels of media coverage and frenzied speculation that has followed each acquisition, the short answer to what does it mean is: endless press exposure. I almost wrote ‘priceless exposure’ but then these are companies with very deep pockets; nevertheless the advertising value equivalent must be very high indeed. The coverage really illustrates the fact that these companies have achieved celebrity status. They are the Justin Beibers of the corporate world. Whatever they do, whether it is truly significant or not, is met with punditry and analysis about what it means. A good example is Google’s recent acquisition of British company DeepMind. In other words: large AI Company buys small AI Company. Large companies buy small companies all the time but mostly they don’t make prime time news. It’s the Beiberisation of the corporate world.

But the question is about robotics, and to address it in more detail I think we need to think about the giants separately.

Take Amazon. We think of Amazon as an Internet company, but the web is just its shop window. Behind that shop window is a huge logistics operation with giant warehouses – Amazon’s distribution centres, so no one should be at all surprised by their acquisition of brilliant warehouse automation company Kiva Systems. Amazon’s recent stunt with the ‘delivery drone’ was I think just that – a stunt. Great press. But I wouldn’t be at all surprised to see more acquisitions toward further automation of Amazon’s distribution and delivery chain.

Apple is equally unsurprising. They are a manufacturing company with a justifiable reputation for super high quality products. As an electronics engineer who started his career by taking wirelesses and gramophones apart as a boy, I’m fascinated by the tear-downs that invariably follow each new product release. It’s obvious that each new generation of Apple devices is harder to manufacture than the last. Precision products need precision manufacture and it is no surprise that Apple is investing heavily in the machines needed to make its products.

Google is perhaps the least obvious candidate to invest in robotics. You could of course take the view that a company with more money than God can make whatever acquisitions it likes without needing a reason – that these are vanity acquisitions. But I don’t think that’s the case. I think Google has its eyes on the long term. It is an Internet company and the undisputed ruler of the Internet of Information. But computers are no longer the only things connected to the Internet. Real world devices are increasingly networked – the so-called Internet of Things. I think Google doesn’t want to be usurped by a new super company that emerges as the Google of real-world stuff. It’s not quite sure how the transition to the future Internet of Everything will pan out, but figures that mobile robots – as well as smart environments – will feature heavily in that future. I think Google is right. I think it’s buying into robotics because it wants to be a leader and shape the future of the Internet of Everything.

Read more answers →

Google, is the wild card for me.  With more acquisitions (DeepMind, Boston Dynamics, Redwood Robotics, Industrial Perception, Meka, Schaft, and others) than Apple, Amazon, Facebook, and Microsoft combined, the GOOG looks to be rigging up a kit that would offer excellent image recognition + navigation + mobility.  A robot that can roam around your home or maybe your city, and offer a helpful service.  My bet is that these Googlebots (perhaps they will be cars) will have sharp eyes, simple yet helpful demeanors, and we’ll trade that robotic service for our personal info.  Then Google will sell that info to advertisers.  So the same Google formula the company has used with email, docs, voice, maps, and the others.

It’s exciting for the robotics community that the giants (Google, Apple, and Amazon) are actively investing in robotics.

Indeed, my initial response to hearing about Google’s first seven of a series of acquisitions of robotics-related companies was a very positive one (these companies include Shaft, Industrial Perception, Meka Robotics, Redwood Robotics, Bot & Dolly, Autofuss, and Holomni, see Frank Tobe’s article for more details on these companies). Technologies from these companies are ones that can make robots mobile, perceptive, responsive, and interactive. The fact that the companies are now housed under one giant roof of Google made me imagine a future of next generation robots where Google-branded humanoids will be the next Baxter, working shoulder-to-shoulder with people in dynamic manufacturing environments, or assisting caregivers and serving seniors at care facilities. How could it not be exciting?

Even if Google’s actual direction is to put bipedal robots in self-driving cars so that people can enjoy fast, fully-autonomous door-to-door delivery of flowers on Valentine’s Day, my view was that such investments can only be good for the robotics community. Large scale investments by giant corporations, regardless of what their actual plans are, can have a positive impact on the job market for roboticists, increase competition between the giants and push robotics forward.

Then I began to feel a little bit uneasy when Google acquired Boston Dynamics. That’s because Boston Dynamics is not only a company with one of the most impressive control systems I’ve ever seen (I’m sure many of you will agree with this from the video of Boston Dynamic’s Big Dog, which has over 15 million views on YouTube), but it’s also a company that is known to have developed such impressive systems for the Pentagon, of all clients.

What does this all mean? What is Google planning to do with robots that can keep itself upright after being kicked, walk through rough terrains, run as fast as 29 miles per hour? The combination of Google-acquired companies became even more mysterious when the London-based AI company, DeepMind, was added to the mix. It’s safe to say that the uncertainty over Google’s plans with these companies has stirred up quite a bit of concern to the general public: some media coverage on the topic have been hyping the idea of Google building Skynet or being in the business of building robot soldiers.

Unless Google is indeed headed in those directions, I don’t believe that the public concern about the possibility of human extinction by Google-made robots in the near future is a good thing, either for the company or for the field of robotics in general. I think that public concerns over Google’s investment directions exist today because we, as a society, lack the presence of governing bodies whose role is to ensure that robotics technologies are developed and used for purposes desirable for the society, in manners aligned with society’s values.

However, I do applaud the fact that Google is setting up an ethics board to presumably oversee what they will and will not do with their technologies. If Google takes ethics seriously, I think Google could be a game changer in robotics (although it already is), not just by producing and selling cool robots in the future, but more importantly, by leading the way of tightly coupling ethics in robotics and making robots that “don’t do evil”.

Read more answers →

They want to communicate more accurately, resolve issues faster and with higher levels of satisfaction than at present. They also want to enable more complex communication. And they want to do it amiably. As we move toward the Internet of Things, more players will enter the fray – all having the same goal.

Google appears to be acquiring people and companies to build a better human interaction interface. This interface can be the front-end of search, IoT devices, robotics and voice systems. Even if Google keeps their algorithms private, the first to deploy complex communication modules that can understand and resolve queries is going to have a big winner on their hands.

Certainly Google is going all out in this endeavor:

• Ray Kurzweil has been at work at Google for over a year now to develop natural language understanding. If successful, Google’s computers will be able to pick up semantic meaning enabling people to ask more complex questions.
• Just last week Google acquired DeepMind, a British AI start-up with some high-profile investors and a whiz kid founder (Demis Hassabis). DeepMind’s forte appears to be smart recommendations for online commerce.
 
• Facebook attempted to buy the deep learning group within DeepMind but Google outbid them.
• Deep learning is AI that trains AI to recognize patterns within streams of data instead of programming immense decision trees. For example, DeepMind trained software to play an Atari video game without teaching it the rules. The software had to learn through trial, errors and low scores.
• Google has hired and bought up leading researchers in AI for years including Sebastian Thrun, Peter Norvig and Geoffrey Hinton.

In a recent New Yorker article by Gary Marcus, Hyping Artificial Intelligence, Yet Again, it was clear that AI has received a lot of enthusiastic press attention over the years but hasn’t delivered upon those promises. Perhaps Google is on the right track toward converting all that desire into reality.

If you liked this article, you may also be interested in:

• Google acquires Nest for $3.2 billion
• Amazon isn’t the only delivery service testing flying robots
• Apple, Amazon and now Google: It’s a pretty exciting day for robotics!
• What were the top stories in robotics from 2013?
• Forget the octocopter, let’s talk about Kiva

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In another surprise release to the press, Google announced the acquisition of Nest Labs, the inventor and manufacturer of the stylish learning thermostat device, for $3.2 billion.

According to Aaron Tilley of Forbes, Google has attempted several times to gain access to “connected home” type systems – including its own energy monitoring service.

The smart thermostat device is just one of a series of home products that are robotic in nature and function but not technically robotic. All of Nest’s line of products involve sensing, automation and control to effect changes (change temperatures, ring alarms, send messges, etc.) in the physical world.

Not only was the “internet of things” the main theme at CES (Consumer Electronics Show) last week in Las Vegas, with this acquisition Google gains a hardware and software consumer products channel, some great talent, and a resource for unlimited home data. It acquired it’s own entre to the “internet of things” and all that data.

Note that Yoky Matsuoka, Nest’s VP of Technology, is a roboticist first and foremost.

Is Google getting a robot company? Or just another source of roboticists with the Nest acquisition that was announced today? Alongside Google’s recent acquisition of eight robotics companies, there has been a slow and steady flow of robotics talent into the Silicon Valley based behemoth. The small sampling of roboticists I’ve spoken to who are employed at Google have shed little light on future plans, but it seems that with so many roboticists, spread out across so many different areas, that a unified robot research park is the least likely outcome. Perhaps Google is collecting libraries and IP instead, in the same way that Wolfram is talking about owning the database of the internet of things.

Google is to buy Nest Labs, Inc. for $3.2 billion in cash. Nest launched in 2011 with a smart thermostat and has recently launched a smoke alarm. Both products are doing very well in sales. Nest will continue to operate under the leadership of Tony Fadell and maintain a distinct brand identity but as a part of the Google stable. For more information from Nest as to the transition.

Larry Page, CEO of Google, said: “Nest’s founders, Tony Fadell and Matt Rogers, have built a tremendous team that we are excited to welcome into the Google family. They’re already delivering amazing products you can buy right now–thermostats that save energy and smoke/CO alarms that can help keep your family safe. We are excited to bring great experiences to more homes in more countries and fulfill their dreams!”

Tony Fadell, CEO of Nest, said: “We’re thrilled to join Google. With their support, Nest will be even better placed to build simple, thoughtful devices that make life easier at home, and that have a positive impact on the world.”

Colin Angle, CEO of iRobot, described Google’s recent acquisitions as a logistics play around self driving vehicles and that when Google solved the first mile/last mile problem, iRobot was planning on meeting them at the house door.

There are plenty more robotics companies left for Google to acquire, but if I were looking it would be interesting to consider Unbounded and Otherlab.

* Extra information was added to this story by the author on 1/14/14

There is serious momentum in robotics these days, evidenced by recent news from Apple, Amazon and Google:

• Apple announced that they were investing $10.5 billion in supply chain robots and automation equipment and recently confirmed their acquisition of PrimeSense for $350 million (PrimeSense is the developer of the Kinnect 3D system used by MS Xbox)..
• Amazon, in a CBS 60 Minutes piece aired last Sunday, displayed a new concept delivery system using an octocopter. Remember that in 2012 Amazon spent $750 million to acquire Kiva Systems, the robot technology enabling robotically-delivered goods to a picker/packer.
• And now Google has set up a robotics division headed by the man behind the Android operating system, Andy Rubin. In Rubin’s first six months he has acquired seven robotic companies to jump start his new operation.
• UPDATE 12-14-13: Google confirms its eighth acquisition: Boston Dynamics. The new acquisition is an engineering company that specializes in building dynamic robots and software for human simulation. The acquisition adds 80+ technicians, engineers and scientists to the new Google Robots Division talent pool – plus a new location.

What does it all mean?

First, some details.

• From a NY Times article by John Markoff interviewing Google’s Andy Rubin:
  • Google acquired 7 tech companies in the last 6 months. Schaft, a Japanese start-up developing a humanoid robot; Industrial Perception, a Silicon Valley start-up that developed a computer vision system for loading and unloading trucks; Meka Robotics, a robot developer for academia; Redwood Robotics, a start-up intended to compete with the Baxter robot (and others) entering the small and medium-sized shop and factory marketplace; Bot & Dolly, a maker of robotic camera systems used for special effects such as in the movie “Gravity;” Autofuss, a design and marketing firm and a partner in Bot & Dolly; and Holomni, a maker of powered caster modules for omnidirectional vehicles.
  • Although Google won’t disclose their plans, the article suggests that the company’s initial market will be in manufacturing, e.g., electronics assembly, which is mostly done by hand. “Manufacturing and logistics markets not being served by today’s robotic technologies are clear opportunity markets,” said Rubin.
  • The article suggests that the new Google robots will be able to automate any or all of the processes from the supply chain to the distribution channels to the consumer’s front door thereby creating a massive opportunity.
  • Google is already experimenting with urban deliveries including making home deliveries for companies like Target, Walgreens and others.
  • According to Markoff, “Mr. Rubin said he had pondered the possibility of a commercial effort in robotics for more than a decade. He has only recently come to think that a range of technologies have matured to the point where new kinds of automated systems can be commercialized.”
• From The SFGate Tech Chronicles by James Temple:
  • Google is transforming itself in many ways including its new robotics division. Its constantly transforming its search engine into a sophisticated learning machine using artificial intelligence tools. Some of that AI talent is moving over to the new Robotics Division.
  • It’s been hiring super brains such as Ray Kurzweil and Peter Norvig and inhouse star Andy Rubin to head groups and divisions moving toward product development and even hardware manufacturing (remember that Google owns Motorola – a ready-made client for assembly and materials handling robots).
  • “Google’s move into robotics is likely to draw renewed attention and money into the space,” said Brian Gerkey in the article. “It’s a pretty exciting day for robotics when someone like Google makes an investment like that in robots, others are likely to follow suit. It can only spur investment and innovation.”
• From Bloomberg News by Adam Satariano:
  • Apple is investing $10.5 billion in new technologies and robotics to polish the new iPhone 5C’s colored plastic cover, to laser and CNC machines to carve the MacBook’s aluminum body, and for testing and inspection gear for iPhone and iPad lenses.
  • Apple invested $6.5 billion on similar robotics and factory automation equipment in their previous fiscal year.
  • Samsung has indicated it plans to spend $22 billion in capital expenditures this year but didn’t disclose any further details.
• For a review of the CBS 60 Minutes interview of Amazon CEO Jeff Bezos and Charlie Rose, see my previous post, “Jeff Bezos Reaches for Tip of UAS Iceberg.”
• UPDATE 12-14-13: From a NY Times article by John Markoff about the Boston Dynamics acquisition:
• Markoff says: “The deal is also the clearest indication yet that Google is intent on building a new class of autonomous systems that might do anything from warehouse work to package delivery and even elder care.”
• Boston Dynamics, a 1992 spin-off from MIT, has been a great resource for youTube videos of wild robots. Their Big Dog video has been watched by more than 15 million viewers; their ATLAS robot video, the base robot given to the DARPA Robotics Challenge teams – a challenge to speed development of robotics technology for disaster response – has already passed the 2.5 million mark.
• This is not an insignificant acquisiton. An ongoing business employing 80+ highly paid engineers and scientists has to have cost Google a very high amount, perhaps in the low 9 figures.
• Boston Dynamic’s CEO and founder Marc Raibert was quoted in the article: “I am excited by Andy and Google’s ability to think very, very big, with the resources to make it happen.”

What does all this mean?

I think the yellow highlighted quote from Brian Gerkey sums up all these investments nicely: it’s a pretty exciting day for robotics when Google, Apple and Amazon ALL invest in robots and robotics. Others are likely to follow spurring further investment and innovation. Up until now, the big four of industrial robotics (KUKA, Fanuc, ABB and Yaskawa Motoman) were all foreign firms while two smaller American firms (iRobot and Intuitive Surgical) led the emerging service new-tech robotics sector. The remainder of this decade will be filled with amazing new robotics products from a variety of new providers — including Apple, Amazon and Google.

]]> https://robohub.org/google-buys-up-robotics-companies-from-drc/ Wed, 04 Dec 2013 15:44:37 +0000 http://robohub.org/?p=23650

If you’ve recently wondered where all the roboticists were going, the answer is to an unassuming complex in Palo Alto. Google has backed Android developer Andy Rubin to acquire at least 7 major robotics companies, including Schaft, Industrial Perception Inc, Meka and Redwood Robotics. Other companies include Holomni, makers of high tech wheels, Bot&Dolly and Autofuss, the super cool robotics software companies behind the special effects in the film Gravity. Unsurprisingly, a number of the new acquisitions are also involved in the upcoming DARPA Robotics Challenge, where robots take on human scale problems in the latest Grand Challenge.

According to John Markoff’s interview with Andy Rubin in the New York Times, the new research company has plans for the manufacturing and logistics industry, with particular focus on the consumer electronics industry, where Rubin sees ‘clear opportunities’ to combat the increasing complexity of the massively growing field..

“I feel with robotics it’s a green field,” he said. “We’re building hardware, we’re building software. We’re building systems, so one team will be able to understand the whole stack.”

Google hasn’t yet announced whether or not this new robotics research group is a subsidiary or separate. But Rubin talks about a 10 year runway, and a vision. The vision is clearly to capitalize on the DARPA Grand Challenges, as Google has done so thoroughly with the automobile/self driving challenge. DARPA create challenges that plausibly have a 10 year solution, with sufficient early incentive and the self-driving cars have met most of the technical hurdles in less than a decade and now primarily face regulatory issues.

Google has purchased a number of teams and people involved in the upcoming DARPA Robotics Challenge or DRC, with an aim to commercializing humanoid robot research for manufacturing and logistics. And pure pleasure. With a 10 year runway, this is still a bit of a playground. Like Willow only double.

]]> https://robohub.org/roadmap-for-creating-singooglarity-2/ Wed, 23 Jan 2013 22:41:00 +0000 http://robohub.org/?guid=39208408ea0386e9fca26364aa542202 Credit: Ray Kurzweil/Viking Press Ray Kurzweil, one of the world's leading inventors, thinkers, and futurists, announced in December 2012 that he will be joining Google as Director of Engineering to work on new projects involving machine learning and language processing.
Creating Singooglarity 
Visorary and transhumanist Kurzweil has published several bestseller books such as The Singularity Is Near and Transcend: Nine Steps to Living Well Forever, that may serve as roadmaps for creating new super-intelligent Google services.
In his latest 352-page book, How to Create a Mind, he explores how artificial intelligence can enrich and expand human capabilities. Kurzweil discusses how the brain functions, how the mind emerges from the brain, and the implications of vastly increasing the powers of our intelligence in addressing the world’s problems. He thoughtfully examines emotional and moral intelligence and the origins of consciousness and envisions the radical possibilities of our merging with the intelligent technology we are creating. Drawing on the most recent neuroscience research, his own research and inventions in artificial intelligence, and compelling thought experiments, he describes his new theory of how the neocortex (the thinking part of the brain) works: as a self-organizing hierarchical system of pattern recognizers. Kurzweil describes how these insights will enable us to greatly extend the powers of our own mind and provides a roadmap for the creation of superintelligence—humankind's most exciting next venture. In Kurzweil´s vision we are now at the dawn of an era of radical possibilities in which merging with our technology will enable us to effectively address the world’s grand challenges.

Kurzweil's ideas have been controversial before and generated criticism within the scientific community and in the media, including Microsoft co-founder Paul Allen, cognitive scientist Douglas Hofstadter and biologist P. Z. Myers. Also his new book has drawn criticism i.e. from Gary Marcus, a professor of psychology at N.Y.U.


======================================= G L O B A L R O B O T I C S B R A I N ======================================= http://robotland.blogspot.com

]]> Ray Kurzweil, one of the world’s leading inventors, thinkers, and futurists, announced in December 2012 that he will be joining Google as Director of Engineering to work on new projects involving machine learning and language processing.

Credit: Ray Kurzweil/Viking Press

Creating Singooglarity 
Visionary and transhumanist Kurzweil has published several bestseller books such as The Singularity Is Near and Transcend: Nine Steps to Living Well Forever, that may serve as roadmaps for creating new super-intelligent Google services.
In his latest 352-page book, How to Create a Mind, he explores how artificial intelligence can enrich and expand human capabilities.

From the book:

Kurzweil discusses how the brain functions, how the mind emerges from the brain, and the implications of vastly increasing the powers of our intelligence in addressing the world’s problems. He thoughtfully examines emotional and moral intelligence and the origins of consciousness and envisions the radical possibilities of our merging with the intelligent technology we are creating. Drawing on the most recent neuroscience research, his own research and inventions in artificial intelligence, and compelling thought experiments, he describes his new theory of how the neocortex (the thinking part of the brain) works: as a self-organizing hierarchical system of pattern recognizers. Kurzweil describes how these insights will enable us to greatly extend the powers of our own mind and provides a roadmap for the creation of superintelligence—humankind’s most exciting next venture. In Kurzweil´s vision we are now at the dawn of an era of radical possibilities in which merging with our technology will enable us to effectively address the world’s grand challenges.

Kurzweil’s ideas have been controversial before and generated criticism within the scientific community and in the media, including Microsoft co-founder Paul Allen, cognitive scientist Douglas Hofstadter and biologist P. Z. Myers. Also his new book has drawn criticism i.e. from Gary Marcus, a professor of psychology at N.Y.U.

]]> https://robohub.org/stanford-hosts-panel-on-making-autonomous-cars-legal/ Sun, 18 Nov 2012 22:35:56 +0000 http://robohub.org/?p=5749 The Center for Internet and Society at Stanford released a video of their panel discussion on “How an (Autonomous Driving) Bill Becomes Law” that took place earlier this month. The discussion builds on recent legislation in several US states (Nevada, California, Florida) to authorize autonomous vehicles. Focus is given to Nevada, which was the first in the world to issue a test plate for Google’s self-driving cars.

The panel includes:
Marilyn Dondero Loop – Chair, Assembly Transportation Committee
Bruce Breslow – Director, Nevada Department of Motor Vehicles
Troy Dillard – Deputy Director, Nevada Department of Motor Vehicles
David Estrada – Legal Director, Google
Bryant Walker Smith (Moderator)- Resident Fellow, Center for Internet and Society

You can read more on the Stanford CIS blog.

]]> https://robohub.org/google-hangout-extension-arduino-example/ Sat, 27 Oct 2012 19:47:29 +0000 http://robohub.org/?p=4837 New tutorial- Get your Arduino working with Google+ Hangouts!

Watch video on YouTube

Learn more here

]]> https://robohub.org/robot-party-google-hangout/ https://robohub.org/robot-party-google-hangout/#respond Thu, 13 Sep 2012 16:30:06 +0000 http://alpha.robohub.org/?p=891

Join us this week for another exciting edition of the Robot Party! Grab your robots and tell your friends, hope to see cool robots there!

Read on for more details!

The Robot Party is a weekly Google+ hangout that brings together robot builders from all over to share their robots, knowledge, and ask questions! It is for all age ranges, young or old, just bring your robot!

The Robot Party is every Thursday at 8PM ET (UTC-4)! If you have a robot, join the hangout on Google+ HERE! Be sure to leave a comment on one of the posts so we can add you to our circles for the party. If you don’t have a robot/don’t want to be in the hangout, you can watch the mirrored stream on this page!

Hope to see ya there, Robot Party ROCKSTAR! ;)

]]> https://robohub.org/robot-party-google-hangout/feed/ 0 https://robohub.org/googles-response-to-apples-siri-buy-cleversense/ Mon, 19 Dec 2011 01:28:00 +0000 http://www.therobotreport.com/index.php/blog/post/googles_response_to_apples_siri_buy_cleversense/ … Cleversense, a learning AI with a natural language processing front-end – and the developer of the popular Alfred personal assistant app – has been sold to Google for an undisclosed amount.
… Fear that Apple’s Siri “could provide a new paradigm for searching the Web that circumvents Google’s search engine and ads” may be the reason.
… The first language processing front-end made available to developers by either Apple or Google will be a windfall for human-robot communication systems development.
]]>