PARAMOUR on Broadway – A Cirque du Soleil Musical. Credit: Richard Termine

Last year, Intel partnered with Lady Gaga on the Super Bowl Halftime Show to showcase its latest aerial technology called “Shooting Star.” Intel did a reprise performance of its Shooting Star technology for Singapore’s 52nd birthday this past week. Instead of fireworks, the tech-savvy country celebrated its National Day Parade with a swarm of 300 LED drones animating the night sky with shapes, logos, and even a map of the country.

Intel’s global drone chief, Anil Nanduri, explained, “There’s considerably more operational complexity in handling a 300 drone fleet, compared with 100 drones in a show. It’s like juggling balls in your hand. You may be able to juggle three, but if you juggle nine, you may have to throw them higher and faster to get more time.” Earlier this year, Intel first showcased its 300 drone show at Coachella music festival on the heels of claiming the Guinness World Record of a 500 drone performance.

Choreographed drones are winning the hearts of Cirque du Soleil theatergoers with a fleet of flying acrobatic vehicles dancing around its human performers. These drones are the brain child of Professor Raffaello D’Andrea of the ETH Zurich, Switzerland and his new startup Verity Studios. D’Andrea is probably best known as one of the three founders of Kiva Systems and now he is taking the same machine intelligence that sorts and delivers goods within Amazon’s warehouses to safely wow audiences worldwide.  The flying lampshades (shown in the video below) are actually autonomous drones that magically synchronize with the dancers, without safety nets or human operators.

Verity’s customer, Cirque du Soleil’s Chief Creative Officer Jean-Francois Bouchard, said D’Andrea’s “flying machines are unquestionably one of the most important statements of the PARAMOUR show.” The key to the flying machines’ success over 7,000 autonomous flights on Broadway is its proprietary technologies that enable multiple self-piloted drones to be synchronized within flight. Verity’s drone is part of a larger performance system called “Stage Flyers.”

The Stage Flyer platform has proven itself in the field, flying next to thousands of people each evening by having built-in redundancy to any single failure. According to Verity’s website, the system is “capable of continuing operation in spite of a failed battery, a failed motor, a failed connector, a failed propeller, a failed sensor, or a failure of any other component. This is achieved through the duplication of critical components and the use of proprietary algorithms, which enable safe emergency responses to component failures.” This means that the drones can operate safely around audiences and performers alike, carrying payloads of cameras, mirrors, and special lighting effects. As shown in the above diagram, the drone system includes a fleet of self-piloted drones that utilize one positioning system and control unit. The company boasts that its system only takes a few hours to install, calibrate and learn how to operate.

Drone swarms are not just for entertainment, as today there are a number of upstarts and established players utilizing these mechanics for e-commerce fulfillment centers. Last June, Amazon was issued a patent for a “Multi-Level Fulfillment Center for Unmanned Aerial Vehicles.” The beehive-looking distribution center (above) is designed to facilitate traffic between inbound and outbound delivery drones. The patent illustration details “multiple levels with multiple landing and take-off locations, depending on local zoning regulations.”

This is all part of Amazon’s larger plan to grow its robotic workforce over the next two to three years. Instead of human truck drivers, the patent displays delivery bays that open and close automatically based on the direction of the drones and interior platforms that cycle around the hive. CB Insights reports that the patent describes “impact dampeners,” such as nets, for receiving inbound drones and “launch assist mechanisms,” such as fans, for launching outbound drones. It appears that Amazon will be looking again to technology like D’Andrea’s research to revolutionize its global network of warehouses with synchronized swarms of drones that safely soar above human workers.

Amazon’s competitor Walmart announced last summer its plans to utilize swarms of indoor drones for inventory management, replacing the need for people climbing dangerous ladders to manually scan labels. The New York Times first reported last July that the retailer applied for a FAA exemption to begin testing drones inside its massive distribution centers. Shekar Natarajan, the vice president of last mile and emerging science for Walmart, demonstrated for the newspaper how swarms of drones could easily move up and down aisles, from floor to ceiling, to scan 30 barcode images a second (an efficiency that would be impossible for even the most agile humans). Walmart has publicly boasted that it will spend close to $3 billion on new technology and other cost-saving investments to bolster its e-commerce business which is growing, but at a slower pace than its nemesis Amazon.

The race to dominate the warehouse has led to increased investments in the logistics sector and even an accelerator dedicated solely to technology around distribution centers. Chattanooga, Tennessee-based Dynamo Accelerator showcased last May its second cohort of startups. One of the most successful showings was Chicago-based Corvus Robotics, a software company that uses indoor aerial drones to scan inventory (similar to the Walmart example above). According to Dynamo’s managing directors, Corvus is building enabling tools that allow operators to fly drones autonomously, scan & sync barcodes, and enter the SKU data into the existing warehouse management system.

Santosh Sankar, the director at Dynamo, explained his accelerator’s mission succinctly in a recent blog post: “We believe our focus and hands-on approach is one of our value-adds. As such, we’re leaning into being seed investors and upholding our commitment to transforming our industry by focusing on our founders and our corporate partners. We’ve opted to not hold a quota for our programs and hone in on companies we can truly help because that ultimately makes for good seed investments.” Sankar added that several of the program participants “are already well on their way to generate ($1 million or more) in annual revenue and/or have raised their initial round of capital.”

Corvus may be the latest indoor drone startup to enter an already crowded warehouse market, which includes established players like the Hardis Group, Smartx, and DJI. Drones continue to amuse, amaze and evolve as the growing need for more unmanned systems in our lives appears to be almost insatiable. Next month, we plan to dig deeper into the drone market with our RobotLabNYC event series on September 19th at 6pm in WeWork Grand Central. Joining us in the discussions include thought-leaders from NASA, AUVSI, and Genius NY, RSVP today, as space is limited.

Source: Intel

Intel announced plans to acquire Israel-based Mobileye, a developer of vision technology used in autonomous driving applications, for $15.3 billion. Mobileye share prices jumped from $47 to $61 (the tender offering price is $63.54) on the news, a 30% premium. The purchase marks the largest acquisition of an Israeli hi-tech company ever.

Source: Frost & Sullivan;VDS Automotive SYS Konferenz 2014/

This transaction jumpstarts Intel’s efforts to enter the emerging autonomous driving marketplace, an arena much different than Intel’s present business model. The process to design and bring a chip to market involves multiple levels of safety checks and approvals as well as incorporation into car company design plans – a process that often takes 4 to 5 years – which is why it makes sense to acquire a company already versed in those activities. As can be seen in the Frost & Sullivan chart on the right, we are presently producing cars with Level 2 and Level 3 automated systems. Intel wants to be a strategic partner going forward to fully automated and driverless Level 4 and Level 5 cars.

Mobileye is a pioneer in the development of vision systems for on-board Driving Assistance Systems; providing data for decision making applications such as Mobileye’s Adaptive Cruise Control, Lane Departure Warning, Forward Collision Warning, Headway Monitoring, High Beam Assist and more. Mobileye technology is already included in BMW 5-Series, 6-Series, 7-Series, Volvo S80, XC70 and V70 models, and Buick Lucerne, Cadillac DTS and STS.

Last year, Intel reorganized and created a new Autonomous Driving Division which included strategic partnerships with, and investments in, Delphi, Mobileye and a bunch of smaller companies involved in the chipmaking and sensor process. Thus, with this acquisition, Intel gains the ability to offer automakers a larger package of all of the components they will need as vehicles become autonomous and perhaps gaining, as well, on their competitors in the field: NXP Semiconductors, Freescale Semiconductor, Cypress Semiconductor, and STMicroelectronics, the company that makes Mobileye’s chips.

Mobileye’s newest chip, the EyeQ4, designed for computer vision processing in ADAS applications, is a low-power supercomputer on a chip. The design features are described in this article by Imagination Technology.

Bottom line:

“They’re paying a huge premium in order to catch up, to get into the front of the line, rather than attempt to build from scratch,” said Mike Ramsey, an analyst with technology researcher Gartner in a BloombergTechnology article.

BMW says it will deploy a fleet of 40 cars in late 2017

Bumping up the timetables, BMW has declared it will have a fleet of 40 self-driving series 7 cars, using BMW’s technology combined with MobilEye and Intel. Intel has recently been making a push to catch up to Nvidia as a chipmaker supplier to automakers for self-driving. It hasn’t quite said what the cars will do, but they will be trying lots of different roads. So far BMW has mostly been developing its own tech. More interesting has been their announcement of plans to sell rides via their DriveNow service. This was spoken of a year ago but not much more has been said.

https://www.youtube.com/watch?v=sqRkYemvjr8

Intel also bought 15% of “HERE” the company formerly known as Navteq and Nokia. Last year, the German automakers banded together to buy HERE from Nokia and the focus has been on “HD” self-driving maps.

Hyundai, Delphi show off cars

There are demo cars out there from Delphi and a Hyundai Ioniq. Delphi’s car has been working for a while (it’s an Audi SUV) but recently they have also added a bunch of MobilEye sensors to it. Reports about the car are good, and they hope to have it ready by 2019, showing up in 2020 or 2021 cars on dealer lots.

Toyota sticks to concepts

Toyota’s main announcement is the Concept-i meant to show off some UI design ideas. It’s cute but still very much a car, though with all the silly hallmarks of a concept — hidden wheels, strangely opening doors and more.

Quanergy announces manufacturing plans for $250 solid state LIDAR

Quanergy (Note: I am on their advisory board) announced it will begin manufacturing this year of automotive grade $250 solid state LIDARs. Perhaps this will stop all the constant articles about how LIDAR is super-expensive and means that robocars must be super-expensive too. The first model is only a taste of what’s to come in the next couple of years as well.

New Ford has a sleeker design

Ford has become the US carmaker to watch (in addition to Tesla) with their announcement last year that they don’t plan to sell their robocars, only use them to offer ride service in fleets. They are the first and only carmaker to say this is their exclusive plan. Just prior to CES, Ford showed off a new test model featuring smaller Velodyne pucks and a more deliberate design.

I have personally never understood the desire to design robocars to “look like regular cars.” I strongly believe that, just like the Prius, riders in the early robocars will want them to look distinctive, so they can show off how they are in a car of the future. Ford’s carm based on the Fusion hybrid, is a nice compromise — clearly a robocar with its sensors, but also one of sleek and deliberate design.

Nvidia keeps its push

Nvidia has a new test car they have called BB8. (Do they have to licence that name?) It looks fairly basic, and they show a demo of it taking somebody for a ride with voice control, handling a lot of environments. It’s notable that at the end, the driver has to take over to get to the destination, so it doesn’t have everything, nor would we expect it. NVIDIA is pushing their multi-GPU board as the answer to how to get a lot of computing power to run neural networks in the car.

What’s coming

Announcements are due tomorrow from Nissan and probably others. I’ll report Friday from the show floor. See you there!

If you liked this post on robocars, you’ll also enjoy these articles:

• What if the city ran Waze and you had to obey it? Could this cure congestion?
• What are the right disability rules for robotaxis?
• Google car is now Waymo
• Comma.ai cancels comma-one add-on box after threats from NHTSA
• Robocar news: Comma One goes open source, creating simulations for robocars in New Zealand earthquakes

See all the latest robotics news on Robohub, or sign up for our weekly newsletter

Source: Intel

Intel is establishing an autonomous driving division; hacker George Hotz is open-sourcing his self-driving software in a bid to become a network company; LiDAR and distancing devices are changing. What does it all mean?

Intel

Semiconductor giant Intel, in a strategic move to become THE supplier of chips to the fast-emerging self-driving industry, is re-organizing and creating a new division. Fortune magazine said:

“The decision is a bet not just on a future of connected devices; it’s also a wager that vehicles will be the centerpiece or at least a core component in the world of IoT. Intel CEO Brian Krzanich has predicted that by 2020, the Internet of Things will include 50 billion devices and each user of those gadgets will generate 1.5 gigabytes of data every day. But, he has said, the average autonomous car will create about 40 gigabytes of data each minute. That’s a lot of data to process and who better to handle the load? Intel.”

Intel in recent months has made key acquisitions and strategic partnerships in pursuit of their autonomous driving division’s plans:

• A partnership with German automaker BMW and Israeli Mobileye to produce self-driving cars for city streets. The companies plan to develop the technology as an open platform that can be used by other automakers or ride-sharing companies.
• Delphi and Mobileye announced they will use Intel’s “system on chip” for autonomous vehicle systems.
• Intel acquired three companies with related products and plans for self-driving cars:
  1. Yogitech is a semiconductor design firm that specializes in adding safety functions to chips used in self-driving cars and other autonomous devices.
  2. Arynga, which was acquired by Intel’s Wind River Unit, develops software that enables vehicles to received over-the-air updates—a capability used frequently by automaker Tesla to enhance performance and fix security bugs in its cars.
  3. Itseez, a company that specializes in machine vision technology that lets computers see and understand their surroundings – a necessity for autonomous vehicles which need to be able to perceive the world around it.

Comma.ai and George Hotz

In a move toward becoming a network provider, and in response to a recent NHTSA letter which caused Hotz to cancel his self-driving aftermarket kit, Hotz announced his plans to open his software code and hardware designs to the world. On November 30th all of the code and plans were posted on Github.

Hotz is encouraging people to build their own self-driving kits. His software enables adaptive cruise control and lane-keeping assist for certain Honda and Acura models.

Fortune magazine quoted Hotz as saying:

“The future is not hardware. If you look at a lot of companies’ business models now it has a lot more to do with ownership of networks than hardware sales. We want to make this as easy as possible for as many people as possible,” Hotz told Fortune, noting that the company even redesigned parts of the Comma Neo to make it easier for hobbyists to build.

nuTonomy

NuTonomy, an MIT spin-off and autonomous vehicle systems startup that’s been testing its technology in Singapore, announced it will begin testing its self-driving Renault Zoe electric vehicles in an industrial park in South Boston in the next few weeks. Although the area is a public area, NuTonomy will, because of the nature of the industrial park, still be somewhat removed from normal vehicle and pedestrian traffic.

Credit: nutonomy

Oryx Vision

Oryx Vision, an Israeli startup which recently got $17 million in a Series A funding round, shows off its new coherent optical radar which uses a laser to illuminate the road ahead, but radar to treat reflected signals as a wave rather than a particle. In an article in IEEE Spectrum, it was revealed that:

“The laser in question is a long-wave infrared laser, also called a terahertz laser because of the frequency at which it operates. Because human eyes cannot focus light at this frequency, Oryx can use higher power levels than can lidar designers. Long-wave infrared light is also poorly absorbed by water and represents only a tiny fraction of the solar radiation that reaches Earth. This means the system should not be blinded by fog or direct sunlight, as lidar systems and cameras can be.”

One of the potential cost savings of Oryx’s technology comes from the fact that its laser does not need to be steered with mechanical mirrors or a phased array in order to capture a scene. Simple optics spread the laser beam to illuminate a wide swathe in front of the vehicle. (Oryx would not say what the system’s field of view will be, but if it is not 360 degrees like rooftop lidars, a car will need multiple Oryx units facing in different directions.)

The clever bit—and what has prevented anyone from building a terahertz lidar before now—is what happens when the light bounces back to the sensor. A second set of optics direct the incoming light onto a large number of microscopic rectifying nanoantennas. These are what Oryx’s co-founder, David Ben-Bassat, has spent the past six years developing.

“Today, radars can see to 150- or 200 meters, but they don’t have enough resolution. Lidar provides great resolution but is limited in range to about 60 meters, and to as little as 30 meters in direct sunlight,” says Wellingstein. He expects Oryx’s coherent optical radar to accurately locate debris in the road at 60 meters, pedestrians at 100 meters, and motorcycles at 150 meters—significantly better than the performance of today’s sensor systems.

Regarding other industry improvements and inventions, the IEEE Spectrum article also said:

“But don’t write off lidar just yet. Following a $150 million investment by Ford and Chinese search giant Baidu this summer, Velodyne expects an “exponential increase in lidar sensor deployments.” The cost of the technology is still dropping, with Velodyne estimating that one of its newer small-form lidars will cost just $500 when made at scale [at present, the price on the website is $7,999 for the new PUCK shown on the right].

Solid-state lidars are also on the horizon, with manufacturers including Quanergy, Innoluce, and another Israeli start-up, Innoviz, hinting at sub-$100 devices. Osram recently said that it could have a solid-state automotive lidar on the market as soon as 2018, with an eventual target price of under $50. Its credit card–size lidar may be able to detect pedestrians at distances up to 70 meters.

That’s not quite as good as Oryx promises but is probably fine for all but the very fastest highway driving. The window for disrupting lidar’s grip on autonomous vehicles is closing fast.”

What’s it all mean:

Fast-emerging may be a catch-all phrase but it seems appropriate to the various parts of the self-driving vehicle marketplace. There are multiple stakeholders about to be disrupted, as I wrote about in April:

“The stakeholders are numerous and the effects can be devastating. Think secretaries, airline reservationists, and all the others disrupted during the emergence of the digital era. The transition to self-driving vehicles and car-sharing systems is likely to cause similar worldwide disruptions.

Consider the insurance industry. If accident rates go down by 90%, as many are predicting, premiums will need to go down too because the reserves for payouts, which are built into the rates, will go down as well. Also, there will likely be fewer insured drivers and car owners thereby lowering the pool of insured people.

Consider hotels and motels and their real estate values and employees – particularly those along the major highways. Why stop at a motel when you can snooze comfortably in a long-haul high-speed driverless car? This will likely affect short-haul airlines and airline employees as well.

Think about professional drivers – truckers, taxies, limos, buses, shuttles. It is expected that many will be replaced by on-demand point-to-point self-driving devices.”

The list of stakeholders also goes on in the auto manufacturing industry and all the Tier 1, 2, 3 and 4 component providers and wannabees. As self-driving vehicles show their needs – for example, the need for low-cost distancing vision systems – new products will be developed and new vendors could replace old ones, hence the posturing by Velodyne, LeddarTech, Quanergy, Bosch, Valeo and all the other LiDAR providers.

As the technologies emerge, it becomes clear that vehicles will be the core component in a larger connected world, streaming massive amounts of data to be analyzed in ways we haven’t even fathomed yet, hence Intel’s new division and the new chips from Nvidia, Qualcomm and others. It’s a fast-emerging marketplace on multiple fronts.

Times and our definition of cars are changing day by day – and fast emerging – a true moving target!

Intel CEO Brian Krzanich during his keynote at CES 2016. Source: Consumer Technology Association

Intel’s Brian Krzanich described the use of embedded chips and cloud services to enable experience-based transactions. While many of these services are about driving sales, others are about enhancing user experience. Krzanich provided many examples, but four stood out as he made his case:

1. In a new Guinness World Record for the most unmanned aerial vehicles airborne simultaneously, 100 flying drones flew and shone their colored lights in sync with an orchestra playing Beethoven’s Fifth Symphony. Because it was so colorful, and timed to the music, Krzanich characterized the event as a potentially safer and reusable replacement for fireworks shows.https://youtu.be/mOBQXuu_5Zw
2. Using augmented reality, the ModiFace mirror enables consumers to try out different makeup looks (blush, eye shadow, lip stick color, foundation, glosses, etc.) and then buy their favorite color and product combinations. Typically done in public in department stores, sampling makeup can be an embarrassing process for some. An augmented reality product such as this one disrupts the process and provides a better customer experience.
   
3. Krzanich also demonstrated a Yuneec Typhoon H with an onboard RealSense 3D camera and chip, enabling a follow-me collision-avoiding drone that will be available later this year as a consumer product.
4. A pair of Oakley sunglasses provided the sensors and communication to track, interact, coach and provide data extrapolated from sensors tracking the wearer’s progress, comparing that progress to set goals, and providing spoken coaching and responsive reports along the way – the result being an experience instead of a pair of sunglasses.

(As an aside, Intel recently announced its acquisition of Ascending Technologies, the German developer of the collision avoidant autopilot system used in the video above. Last year Intel also made a big investment in Yuneec (whose drones now include onboard Intel RealSense cameras and chips) and Airware (a competing autopilot developer). And this week Intel invested in robotics startup Savioke, maker of the Relay robot, which autonomously navigates around hotels delivering toothpaste, towels, Starbucks coffee and other items.)

IBM CEO Ginni Rometty onstage at CES 2016. Source: Consumer Technology Association

IBM CEO Ginni Rometty told a packed CES audience that Watson and IBM were changing the nature of data processing from transactional to cognitive. Last year Robert High (CTO of IBM’s Watson Group) said that the emerging era of embodied cognitive computing is leading to “cognition as a service” — for example, a welcome 3rd hand for a lab technician, or a concierge or office assistant such as Jibo or Echo, or an assistant in field settings like search and rescue. The cognition process involves machines interacting with humans in a variety of ways, including via text, verbally, with tactile and visual cues, and with gestures. Cognitive algorithms and cloud computing are the keys to Watson’s feats, and they also happen to be where Rometty has pushed IBM since her tenure began.

Rometty described Watson’s progress thus far and demonstrated on stage many areas where applying Watson to an application is changing the nature of the experience. It’s not enough to just be able to hear what a user says. Watson must be able to understand what they want, be able to make that happen, and then interact back with the user in as conversational a tone as possible. SoftBank’s Pepper robot is an example of how this works. In this excerpt from Rometty’s presentation, she and SoftBank Robotics’ Kenichi Yoshida announced that IBM will provide global distribution and support for SoftBank’s Watson-powered Pepper robot as they scale up to begin selling into China and the US. Pepper has already sold 7,000 units in Japan and is in 300 bank branches, and 100 stores.

Bottom line

These two CES keynote presentations illustrate how artificial intelligence and data synthesis will provide the backbone to enable meaningful and productive interaction between humans and machines — not only on screens, but with gestures, visual cues and spoken understandable communication to and from smart devices and robots of all types. And it’s not just a near-term future they foretell — they give examples of where it is happening already.

Intel Corp. has acquired German drone and autopilot developer Ascending Technologies. Ascending’s LED light painting technology was used to fly and draw the Intel logo while the moves were shot with long exposure photography.

According to Ascending Technologies’ spokesman Matthias Beldzik, Intel and Ascending Technologies will focus on UAV obstacle avoidance systems for drones.

In an Intel blog, the chip company said the acquisition lets it “integrate the computing, communications, sensor and cloud technology required to make drones smarter and more connected. The UAV business is an important computing platform of the future.”

No financial details were provided about the transaction. Ascending Technologies has 75 employees and is located in Krailling, near Munich, Germany.

Ascending’s auto-avoidance system is designed to prevent drones from crashing into trees, people and airplanes. Some of the company’s other products already use Intel’s RealSense 3D camera for 3D mapmaking and other surveillance and visualization needs. In addition to it’s autopilot system, Ascending also manufacturers the AscTec Neo, a programmable, totally redundant UAV. The drone is resistant against single-point-of-failure in electronics and hardware, including motors.

Collision avoidance autopilot systems are a big focus within the drone industry. Emlid, Airware, Teledyne, Boeing, CMU, MIT and many others are all developing products that can be incorporated by drone makers for this purpose, hence Intel’s interest.

Intel also invested $60 million in Yuneec Electric Aviation, a Chinese drone maker, a few months ago. The two companies said that Intel and Yuneec will work on developing future products. Intel has also invested in other drone companies such as Airware and PrecisionHawk. These drone relationships fit with Intel’s strategy to make investments in companies developing products with the potential to expand the market for semiconductors, as the company searches for new devices for chips. Qualcomm is doing the same kind of strategic investments for their chip development.

Guest talk in the ShanghAI Lectures, 2009-10-29

How can science fiction help prototype emerging science theory and experimentation? Expanding on the framework of consumer experience architecture, this talk explores how a fictional story, based specifically on current works of scientific research, can lead to the expansion and further experimentation of a dramatic new approach to artificial intelligence and domestic robots.

The talk will explore a novel and innovative methodology (science fiction prototyping) which facilitates the creation of new research opportunities and commercial products, based around the concept of integrating science fiction and fact within a regenerative prototyping cycle.

The ShanghAI Lectures are a videoconference-based lecture series on Embodied Intelligence run by Rolf Pfeifer and organized by me and partners around the world.

https://www.youtube.com/watch?v=ul3IVWAWTyk

The future is Brian David Johnson’s business. As a Consumer Experience Architect he develops future products for the Intel Corporation, a global microprocessor manufacturer. He speaks and writes extensively about future technologies in articles and scientific papers as well as in science fiction novels (Fake Plastic Love, the forthcoming This is Planet Earth) and short stories. He has directed two feature films, and an illustrator and commissioned painter.

The ShanghAI lectures have brought us a treasure trove of guest lectures by experts in robotics. You can find the whole series from 2012 here. Now, we’re bringing you the guest lectures you haven’t yet seen from previous years, starting with the first lectures from 2009 and releasing a new guest lecture every Thursday until all the series are complete. Enjoy!