Image: Festo

Ask a child to design a robot, and they’ll produce a drawing that looks a little like you or I—the parts may be gray and boxy, but it will have two arms, two legs, and a head (probably with an antenna coming out of the top). Starting from the beginning of robotics, the human form has seemed like an excellent starting point. One of the best places to draw inspiration for robotic design, however, is the kingdom of insects, arachnids, snails, and slugs.

Challenges inherent in the human form

Here’s a simple question: how do you walk? It’s not by putting one foot in front of another. You walk by falling, and then catching yourself. This is a really complex movement, which has been best described by mathematical models that describe the human body as an inverted pendulum. Therefore, it’s safe to say that walking upright is not an easy challenge for robots to master. A recent DARPA robotics challenge—showcasing the most advanced humanoid robots on Earth—still saw nearly half of the contestants fall flat on their faces.

Designing robots to imitate humans—or any mammal—is a huge challenge. Our gait, our musculature, even the ways we see the world are all vastly more complicated than any machine we’ve ever made. Insects are simpler creatures by contrast, and therefore are much easier to replicate.

The insectoid advantage

Here are a few reasons why a budding robot designer should study insects. First of all, insects get a lot of mobility out of small bodies with simple joints. Better still, they’re robust—a ladybug can fly into a window, bounce off, and keep going. Small, mobile, and durable—all of these qualities represent admirable characteristics in a practical robot.

Small robots can be produced easily en masse, which can also lend itself to useful applications. By mimicking the swarming behavior of a cockroach, the makers of a robot known as Velociroach hope to use their tiny robots as rescue workers. Small robots can easily crawl through the cracks and crevices of a collapsed building in order to locate survivors and notify rescue workers.

Similarly, small insect bodies equate to small insect brains. While this isn’t likely to be much help to an insect in an IQ test, it’s a great way for engineers to study the secrets of robot autonomy. Despite their small brains, insects can avoid predators and display complex behavior. Even a creature as simple as a maggot—with just 10,000 neurons—can still learn relatively complex tasks, and can actually be trained to a certain extent.

The metamorphosis from insects to robots

Insects are simple enough that a myriad of insect-inspired robots have already made it from the drawing board into real life. Researchers from Seoul University have made a robot, for example, that takes its cues from the water strider—it can literally jump on water.  Sensors based on light-sensing organs found in insects have been used to stabilize drones.  Other robots have demonstrated the ability to swarm like ants.

When designing a robot, it makes sense to copy simpler organisms first, and then move on to more complex organisms. Robotics engineers have already learned a great deal from insects, and this knowledge lends itself to advancements in both mechanics and artificial intelligence.

If you enjoyed this bio-inspired article, you might also be interested in:

• QuadBot: The 3D printed crab-bot scuttling beyond its Kickstarter goal
• Robohub roundtable: Robotic bee swarms from Black Mirror – what’s hype, what’s real?
• New insect-inspired vision strategy could hasten development of mini-drones
• A drone with insect-inspired folding wings
• mROBerTO: The modular millirobot for swarm behavior studies

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Curiosity Heading for Mount Sharp, during the 329th Martian day, or sol, of the rover’s work on Mars (July 9, 2013). The turret of tools at the end of Curiosity’s robotic arm is in the foreground, with the rover’s rock-sampling drill in the lower left corner of the image. Image credit: NASA/JPL-Caltech.

NASA may be known for sending men to the moon, establishing the International Space Station, and planning for a base on Mars—but apart from astronauts, its best-known spokesmen aren’t men at all—they’re robots.

Rovers like Spirit, Opportunity, and Curiosity, and landers like Viking and Philae, make the perfect ambassadors into hostile, freezing, and nearly airless environments. Not only do these explorers bring back valuable scientific data from Earth’s planetary neighbors, they also make perfect showcases for practical robotics.

Pioneer 13—Venus

Venus is no one’s idea of a vacation home. Its atmosphere is a mixture of sulfuric acid and carbon dioxide, and with a surface temperature of 863 degrees Fahrenheit, only the toughest machine could possibly survive on its surface.

This is why it was surprising when a probe launched as part of the Pioneer 13 exploration satellite in 1978 managed to survive and transmit data. This probe wasn’t designed as a lander—it was supposed to fall through the atmosphere, collect chemical data, and then destruct on impact with the surface. Instead, it survived and managed to transmit data automatically for 67 minutes after landing. By comparison, a purpose-built Russian probe called Venera 7 only lasted 20 minutes at Venusian ground level.

Philae—Comet 67P

The Philae lander was not quite a success, and not quite a failure either. Anyway you care to put it, the explorer embodies the ancient philosophy of Ad Astra, Per Aspera—to the stars, through difficulties. Philae’s landing system was always considered to be quite experimental. As the first space vehicle ever to attempt a landing on a comet, Philae needed to contend with low-gravity conditions, as well as the comet’s surface, which was fragile, uneven, and icy.

Gravity on 67P was so low that Philae couldn’t “land” in the traditional sense. Instead, the plan was for Philae to fire harpoons into the crust, and then a landing rocket to essentially glue itself down onto the comet’s surface. This didn’t happen—neither the harpoons nor the retro rocket fired as intended, and thus Philae didn’t so much land as bounce. Philae survived the unplanned bounces, however, and was able to send valuable exploration data in spite of its unconventional landing.

Opportunity Rover—Mars

Let’s face it—we probably could have written this entire entry just on the topic of NASA’s Mars rovers. For the sake of restraint, however, we’ve just picked two.

Opportunity is special. The rover landed on Mars in 2004, and was designed for a mission lasting just 92 days.  This proved to be a pessimistic assessment, as the vehicle didn’t just outlast its original mission plan—it’s still operating today, over 4,500 days after it was expected to fail. NASA chalks this viability up to two factors: sturdy construction, and good operational choices. Importantly, both the Spirit and Opportunity rovers would stay parked for months at a time, allowing them to conserve power during months of weak sunlight.

The Opportunity rover also possesses limited autonomy. Depending on the position of the planets, radio signals can lag between 4 and 24 minutes between Mars and Earth. This makes it impractical to drive a rover like an RC car. Unless the rovers are in the midst of particularly difficult terrain, NASA controllers will simply pick a point a short distance away and tell the rovers to go there. Using 3D terrain maps generated by a stereo camera, Opportunity picks the best available path, drives down it, re-calculates, and drives again.

Curiosity—Mars

In many ways, Curiosity is an iteration of the Spirit and Opportunity rovers. Everything about it is bigger, its systems are more robust, and it possesses even more autonomy. This level of engineering and processing power allowed it to perform incredible feats starting on Day One of its mission. You may remember Curiosity’s landing—gently spooled to the ground on a crane held aloft by rockets. Not only was this process controlled entirely by the mission’s onboard computers, those computers were less powerful than a 2012 iPhone.

While NASA has sent most of its wheeled rovers to Mars, there are plans in works for the space exploration agency to send its robotic explorers to several other locations. NASA is currently formulating plans for a potentially aquatic rover to drill and explore beneath the icy oceans of Europa, and the agency recently launched a sample return mission—Osiris Rex—which will recover a portion of a near-Earth asteroid.

NASA’s designs are everything we want to see out of robotics. Maximum autonomy using minimal computer power. Robust designs that last years beyond their intended lifespan. These results aren’t just perfect for space exploration—they also result in innovations that drive the adoption of low-cost practical robotics here on our home planet of Earth.

As far as party games go, the Imitation Game is a pretty clever and fairly entertaining one. A man and a women answer questions as if they were the other, trying to fool party guests into guessing the wrong gender when identifying the answers. The idea of impersonation is too intriguing to pass up in a fun context.

This game, with its basic premise and mystery, is the basis of the infamous Turing Test. British mathematician Alan Turing used the game to describe and develop a test that measures a machine’s ability to imitate a human. The question long asked by early robotics pioneers was “Can machines think?” But Turing reframed the question to instead ask, “Can machines show they can think like a human?”

His test revolutionized how machine intelligence was viewed in relation to human thinking and functionality. The philosophy behind his question and test has guided the progress made in advancing modern artificial intelligence (AI).

While the Turing Test has historical significance and contributed a great deal to the study and growth of technology, it shouldn’t be the sole standard or measure of artificial intelligence. As the goals of artificial intelligence have made gains, the Turing Test is becoming a relic of the past and an irrelevant measure of the effectiveness of AI with a few caveats.

Complete human behavior

The Turing Test was developed to measure if a machine could act like a human, meaning it would imitate both intelligent and unintelligent behavior. Turing’s test was not a way to perfect a machine to solve a complicated problem, but to emulate the human brain—imperfections and all. In the context of the test, a machine would imitate human typing errors to try and prove it was human and not machine.

It’s important to remember that the Imitation Game—both the original party favorite and the technical version—is a game of deception not accuracy. The machine is trying to be identified as the human and the woman is trying to be identified as the man. With honest answers and the element of imitation removed, it becomes a question and answer game with very little point.

To be the ultimate player of the Turing game, machines are not programmed to produce the best answers, but the most human ones. When just answering Turing’s reframed question, “Can machines behave like humans?” the answer can most certainly be yes.

The Loebner competition, founded in 1991 as a glorified show of the Turing Test, proved that. Declared “artificial stupidity,” the Loebner competition is reviled by AI experts because instead of focusing on improving AI to do what it is intended, people are competing to prove a bot can replicate human fallibility.

Updated standards

Now, as scientists and engineers try to craft artificial intelligence for practical purposes and daily use, the importance of the Turing Test is shrinking, but not disappearing. The goal of AI now is to, in fact, solve complicated problems and equations that humans would normally struggle with. Machine learning and the advancement of how technology understands and fits into the world is becoming the standard. Machines are here to help, not simply create more human-like behavior.

On the other hand, including human-like elements in robotics and AI keeps the technology grounded in human reality. As Professor Sethu Vijayakumar FRSE, who holds a Chair in Robotics within the School of Informatics at the University of Edinburgh told Sputnik International, human forms and human behavior in robots, like the one Vijayakumar is working on to man a mission to Mars, help the people using it understand its functions in a relatable context.

His point is that we should use artificial intelligence as a way to enhance the human experience. Using technology to accomplish tasks risky to human life and using the technology to refine discoveries is an ideal application of AI and robots.

Developing AI should still involve the Turing Test to ensure there is a human element included in its design. It shouldn’t be the end all be all of AI’s effectiveness or ability to succeed in the world, but it should be a small part of determining how well it will fit into a society’s reality.

Robot “pig” in Germany, helps to aerate and dry sewage plant sludge. Source: Thermo-Systems

Climate change is one of the greatest issues facing the world today. With temperatures rising to their highest point in 4,000 years due to human activity and monthly average carbon dioxide levels exceeding 400 parts per million, the climate is seeing significant and devastating effects. The number of natural disasters has risen 50% in 20 years, which is greatly impacting the quality of life of many communities. Even these most basic statistics are proof the environment is suffering from human actions and the need to mitigate the issues is growing exponentially every day.

As attention to climate change, natural disasters and the human effect on the earth comes into focus and becomes a great global concern, leading minds will search for ways to solve the continuing crisis. One of the most innovative and sustainable solutions emerging from climate change technology are bots. A variety of robots and tools are being created to solve one specific piece of the environmental crisis. Together, the engineers who develop them and the bots doing the work will be able to make a difference and lessen the impact of human activity destroying the planet.

These are some of the most creative and interesting bots with a mission for sustainability and a mind toward a better future.

Fighting Wildfires

On average, there are over 100,000 wildfires in the U.S. every year. These ravaging infernos clear 4 to 5 million acres and take all the nature, homes and even people in its path. Often fueled by drought and dry conditions, wildfires can quickly get out of control and damage more than is good for the environment, not to mention the destruction it can cause on communities and homeowners.

To avoid the peril of sending firefighters in to risk their lives, many companies have developed robots of all types to extinguish flames more creatively. The original idea was a German bug equipped with water and extinguishing agents that would be autonomous and guided by GPS. The reality includes large, remotely controlled bots tethered to hoses, massive tank-style apparatuses that can follow a course or a human, and evolving humanoid firefighting robots.

https://www.youtube.com/watch?v=Gf6-VZU-cVg

Exploring and Restoring the Ocean’s Health

The ocean is a critical key in understanding the environment and discovering ways to remedy the damage that’s been done. With such a heavy influence on climate, caring for the ocean and learning how it continues to shape the world can greatly contribute to improving the climate crisis.

After the BP oil spill in 2010, it was clear the world needed a better solution for clearing the ocean of dangerous pollutants. One designer imagined a small robot drone that could degrade oil with certain oil-hungry bacteria. The devices would also be powered by ocean waves and have a feature that keeps wildlife out of the way of the cleanup zone. This bio-cleaning robot isn’t a reality, but ideas like this are leading the way for how robotics will help the ocean.

One truly revolutionary robot that has come to life is OceanOne, who explores the ocean floor and sends sensory signals back to his controls above the water. On its maiden voyage, OceanOne got stuck in a shipwreck. The robot’s creator, Oussama Khatib, could feel it and maneuver him out of the rusty cannons. The intention of OceanOne is to research the threats to the ocean’s biodiversity. By being able to go deeper and stay under water longer than a human, researchers have far more opportunity to learn about the mysteries of the deep blue sea.

Preventing Toxic Environments  

Not all robots are going to the depths as OceanOne, but are still performing important environmental work. Robotic “pigs” are being used in Germany to aerate and dry sewage plants. When sludge is extracted from sewage treatment, it creates a hazardous mud that needs to be dried immediately to avoid releasing dangerous pathogens and hormones into the environment. The pigs use solar power to do the job without creating any adverse side effects common of other methods.

Each of these robots was engineered with one goal in mind. When approached with singular clarity and focus, robotics has the potential to solve many environmental issues. No one person or machine is going to solve the climate crisis with one magic solution, but if scientists and innovators continue to solve one problem at a time, we might have a chance to reverse the damage and work toward a truly sustainable future.

https://youtu.be/xKFaYuBF9HU

Jibo may be a robot, but the last thing the team wants is for Jibo to sound like a robot. In these two video interviews, Jibo’s design team talks about how they selected Jibo’s voice, how that then manifests itself as the voice you’ll hear when you interact with Jibo, as well as the engineering challenges of Text-To-Speech (TTS) technology and how the team solved for them.

Blade Kotelly, VP of Design 

https://youtu.be/3xYlzFdLrAs

Sriddhar Raghavan and Roberto Pieraccini

https://youtu.be/OddZWgc0na8

Through the Jibo SDK, developers have high-level access to Jibo’s audio processing, visual processing, persona and interaction, and movement capabilities. The SDK gives developers the tools to build a wide range of Jibo Skills for personal enjoyment or as a business opportunity. We sincerely believe developers have a huge role to play in extending Jibo’s personality and capabilities, making him the social robot people are looking forward to being part of their lives.

To realize this, Jibo’s SDK development team has been working on creating an SDK that makes developing on Jibo accessible to a wide range of developers. Our skills development team has been using the alpha version internally for a number of months now. In this video, Jonathan Ross and Francois Laberge talk about the challenges of building Jibo’s SDK and some of the innovative approaches they’ve used including the use of Electron and the Atom Editor from Github.

https://youtu.be/94n6A_1GSaE

https://youtu.be/_f8VjQLO8Xw

If you liked this article, you may also be interested in:

• The second wave of social robots is here
• Why are JIBO, Pepper, Siri, Google Now and Cortana so important?
• Jibo raises $1M in first week of crowdfunding campaign

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Our goal is to give the Jibo developer community the tools and support they need to make expressive content that takes advantage of all of these output modalities.

To help our developers create compelling skills, we’ve given Jibo a sophisticated animated system that builds on more than a decade of research from Jibo’s founder, Dr. Cynthia Breazeal’s personal robotics group at the MIT Media Lab.

Catch a sneak peek of Jibo’s development and watch our newest video:

https://www.youtube.com/watch?v=mx-WYHSzEGY

http://youtu.be/OhtmjlePduY

If you liked this article, you may also be interested in:

• Jibo raises $1M in first week of crowdfunding campaign
• JIBO: A game changing social robot
• The second wave of social robots is here
• Designing Jibo: Subtracting the obvious and adding the meaningful

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The idea of robots being a part of our daily lives is a controversial topic to say the least. It seems everyone you talk to has a strong opinion about the issue as well as an equally vivid image in their head of what a robot looks like. Cliché robot visuals are everywhere in our culture and much of it heavy-handed and unflattering. Popular images conjure a complicated machine with poorly simulated human-like parts mixed with exposed joints, sensors and lights.

Robots have looked the way they do for a good reason. Most of these machines are a collection of motors/sensors/chips/joints that strive to mimic the movements/appearance of lifelike beings. The overtly mechanical look is driven by physical necessity, allowing these machines to do real work in a very task-oriented manner.

A social robot is a very different machine however. The vision that Cynthia and the Jibo team laid out to us almost a year ago painted the picture of a socially connected, intelligent home companion. Its primary skill would be its ability to emotional connect with its user and become a helpful extension of the family. Jibo would have no need to zip around the house on 2 wheels or reach over and pour you a glass of water. Instead, its movements, expressions and communication all needed to be focused directly on the user in a subtle and pleasing way. This robot’s primary mechanical task was to make you smile J.

Every design project has unique challenges and this one was certainly no exception. How do we erase the visual constructs of what a robot is supposed to look like and focus this new experience? What qualities will make this little machine deliver on the promise in sophisticated, visually bold way? John Maeda, renowned graphic designer and author of The Laws of Simplicity, famously said that the key to creating harmony between object and user arises when you “subtract the obvious and add the meaningful”. This quote summarizes our fundamental design approach with Jibo from early concepts to final product.

Subtract the obvious:

Given the primary task of connecting emotionally and re-defining social intelligence, decisions over eliminating or hiding traditional robot visual elements proved crucial to defining the look of Jibo. Some of these decisions were easy. The Jibo team envisioned a desktop companion freed from the need to complete mechanical tasks, so there was no need for arms and legs. Looking deeper at the elements of a robot, the remaining decisions of what could be eliminated were much smaller, subtle but equally important.

Jibo needed a head, but what level of detail was necessary to convey and connect with users? Working closely with the Jibo team we explored a range of head proportions and played with the level of detail needed. We all agreed we didn’t want to create anything that resembled a toy so any character-driven detail needed to be treated with abstract simplicity. Eliminating as much unnecessary detail in the face and head without taking away the visual ‘character’ necessary to connect with users was a crucial and delicate balance. Through this elimination process, we discovered that actually very little overt facial detailing is needed.

The simplification exercise then moved from the aesthetic features we could safely subtract to the more complicated issue of functional elements we wanted to hide. We certainly didn’t want a complicated mess of sensors, visible cameras and body segments to detract from the bold simplicity of Jibo. This was a heavily collaboration working closely with the extended Jibo team of software, electrical and mechanical engineers. For this effort our previous design experience working on laptops, tiny wearables and countless high tech gadgets was key to understanding where the opportunities to reduce complexity laid.

Add the meaningful:

Understanding the core elements that will make the physical Jibo resonate in a ‘socially connected’ way was equally important to the simplification of form and details. The Jibo team had established a few basic elements that would be key to this emotional connection in the hardware: a friendly silhouette activated by an expressive movement that is directly linked to an intelligent UI ‘face’ and likeable voice.

The design exploration of form and movement were closely linked and needed to be explored together to define the silhouette. Fardad Faridi, Jibo’s talented in-house animator, introduced us to the basics of character expression in animation and shared examples of basic forms that can deliver the necessary ‘line of action’ of thoughtful poses. These movements are crucial to reinforcing the emotions and interaction conveyed through Jibo’s face and voice.

A variety of form + movement architectures were explored but the design that emerged was based on earlier work from the Jibo team in Boston. The architecture sub-divided a very simple cylindrical vertical body into a series of angled/rotating sections. By controlling the movement of each section independently Jibo was able to quickly snap into a variety of expressive poses and appear to magically dance in a pleasing way. The most impressive aspect of this architecture was its inherent simplicity. The traditional pivoting joints seen on most other robotic mechanisms would have been overkill and far more visually complicated. More importantly, the cylindrical nature of the mechanism meant that the eventual form could take on a very elemental geometric quality: an abstracted character composed of a simple cylindrical torso and pure spherical head.

The design of Jibo’s face was probably the most difficult part to get right. Since the user experience would rely heavily on software driven apps and programs we all agreed his face would have to have a display somewhere on it. As discussed above, the subtraction of elements played an important role here but eliminating the display was not an option.  The screen UI (working in concert with the voice and movement) would be integral in creating the facial character of Jibo while also providing necessary supporting visual communication. Luckily, the desire for impactful graphics (that can be easily seen from across the room) aligned with the desire of a large-headed, friendly character silhouette. It was crucial that users not focus on the technology inside Jibo. Hiding the display component behind the face was a seemingly small detail that has a huge psychological impact on how users react to their little companion.

Completing the vision:

In the end, the details make the design. Eliminating as many unnecessary details as possible and celebrating those few remaining with a subtle sophistication was key to the refinement of the physical Jibo. The boldly simple form silhouette made up of pure geometric shapes is accented with subtle precision engineering details and sophisticated surface transitions to express the qualities of a truly premium and intelligent device. Playful elements (such as the friendly proportions, expressive movement, high-touch surfaces of the body and the borderless face) work together to build the approachable human-like quality that other smart devices lack. The combination of these contrasted elements creates a bit of mystery around the intelligence hidden inside and creates an own-able visual icon for the first social robot. We are excited to be able to introduce Jibo to the public.

Building a robot company is difficult, very difficult. It requires unwavering belief in yourself, in your idea, in your team and ultimately in your product. But it is also one of the most thrilling experiences anyone could imagine.

Why? Because people love robots. Especially social robots.

Robots are the reflection of us, in our belief in technology, in our ambition to shape the future. Robots are the exploration between people and machines, the science of artificial intelligence, AI. They test our theory of life, how we think, how we behave and how we want technology to interact with us.

“People want a relationship with robots, they just don’t know it yet.”

This was our driving belief in building Ugobe pronounced “you-go-be”. Ugobe created Pleo, a life-like robot using our “LifeOS” technology that animates robots with life-like movements similar to living creatures. After countless experiences of watching people transform into proud pet owners through interactions with Pleo, I am absolutely convinced that people want to connect with robots in a meaningful way that touches our hearts. When robots truly become social the world will change; we will look at technology as living relationships that enrich our lives, not detract from it. Pleo was the beginning of a new age in robotics.

The journey of Pleo and how we created an amazing product experience is the real story. To build a social robot that transforms people into proud pet owners is not a simple task. You need to build a company that mixes creative “wonderment” with savvy know-how and incredible engineering skills. You need to blend art with science and challenge traditional thinking and experiences. As Gordon Radley, former President of Lucas Films would say to me “you need creative tension in your product development process to build great products.”

We learned how to build creative tension into many of our decisions and company culture. We never allowed ourselves to get “comfortable” with how other companies built products. We always challenged ourselves in a collaborative, creative way. These experiences became pearls of wisdom for Ugobe. Of course none of these experiences would have occurred if we were building an industrial robot. No, we were building a social robot. Something new and different. We needed to push the boundaries of what people thought about robots and their relationship to technology. We thought different, we were different. That’s why Pleo is different… and magical.

I began my journey into the world of robots in 2003 when I met Caleb Chung, the creator of Furby and a master puppeteer. Caleb discovered a way to manipulate robots through the use of an adapted motion capture suit that mimicked the balance and movement of animals. The result was so surreal and uncanny; a seemingly simple device stitched together by hobby servos, wires, and RadioShack batteries began to move as if it were alive: thinking and behaving like organic life. I felt the urge to touch it, to pet it.

At that moment I knew this invention was more then just an idea, it was a platform for creating life-like robots, for blurring the line between people and machines. This was the genesis of Ugobe, the architect and maker of Pleo.

Caleb and I decided to launch our big idea at CES in 2004 where we met Greg Appelhof of TRG. Greg represents consumer tech companies (such as Harmon Kardon and Griffin Technologies) and gets them into Best Buy, Target and other top tier retailers. Greg encouraged us to launch Pleo into retail after describing WowWeeʼs success with Robosapien — selling over 190,000 units through retail channels. He believed in our vision to change the robot category to our benefit. Retailers viewed robots as the next big thing and were hungry for innovative products and solutions.

After a trail of endless demos and mindless meetings with investors who loved robots but didnʼt invest in them, I eventually raised $26MM for Ugobe. I also forged a strategic partnership with Terry Gou of Foxconn and built a worldwide distribution network with tier 1 partners. We attracted the best and brightest people from Apple, Lucas Arts, EA, Blizzard, HP and Google. Our customers included Nolan Bushnell, Vladimir Putin, Jet Li, Steve Wozniak, Sean Parker and many Silicon Valley luminaries. We learned to stay up late so we could communicate with Asia and wake up early to support our European partners. It took a long time to build Ugobe but it feels like a brief and brilliant moment.

Pleo, and the unfortunate exit of Ugobe, five years ago is now robot lore. But what is not known is why we failed and how we could have succeeded if we knew about the 2-year itch. That’s a topic for my next blog. Stay tuned.

If you liked this article, you may also be interested in:

• The next era of conversational technology
• Embodied communication: Looking for systems that mean what they say
• In-corporating body language into NLP (or, More notes on the design of automated body language)
• The death of search (or, My dysfunctional relationship with Siri)
• Google’s response to Apple’s Siri: Buy Cleversense

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In 1922, the Elmwood Button Co. commercialized a toy that could respond to voice commands. The toy, called Radio Rex, was a dog made of celluloid that would leap out of its house when you called its name: “Rex!” That was two decades before the first digital computer was even conceived, and at least three generations before the Internet would jet our lives from the real world into the virtual realms of e-commerce, social networks, and cloud computing. Radio Rex’s voice recognition technology, so to speak, was quite crude. Practically speaking, the vibration produced by the “r” in its name, when spoken loudly, would cause a thin metal blade to open a circuit that powered the electromagnet that held Rex in its house, causing Rex to jump out. A few years ago, I watched in awe one of the few remaining original Radio Rex toys in action. It was still working.

Some people recognize Radio Rex as the first speech recognition technology applied to a commercial product. Whether that could be called speech recognition or not, we have to agree that the little celluloid dog was probably one of the first examples of a voice controlled consumer robot. While rudimentary, I can imagine the delight and wonder that people felt when Rex responded to their own voice. Magic!

Indeed, speech recognition is something magical. That’s why I have spent more than 30 years of my life building machines that understand speech, and many of my colleagues have done the same. The machines we have built “understand” speech so well that there is a billion dollar industry based on this specific technology. Millions of people talk to machines every day, whether it is to get customer support, find a neighborhood restaurant, make an appointment in their calendar, or search the Web.

But these speech-understanding machines do not have what Radio Rex had, even in its own primitive simplicity: an animated body that would respond to voice. Today, social robots do. And even more: they can talk back to you, see you, recognize who you are and where you are, respond to your touch, display text or graphics, and express emotions that can resonate with your own.

When we consider voice interaction for a social robot, what type of speech and language capabilities it should have? What are the challenges? Yes, of course Siri and Google Now have paved the way to speech-based personal assistants, but despite their perceived popularity, is speech recognition technology ready for a consumer social robot? The answer is yes.

First, let me explain the difference among the speech recognition technologies used for telephone applications (à la “Please say: account, payments, or technical support”), for personal assistants (like Siri, Google Now, and the most recently Cortana) and for a social robot.

In telephone-based applications, the speech recognition protocol is synchronous and typically initiated by the machine. The user is expected to speak only during well-determined intervals of time in response to a system prompt (“Please say …”). If the user does not speak during that interval, or if the user says something that is not encoded in the system, the speech recognition machine times out, and the system speaks a new prompt. This continues until something relevant is said and recognized, or either the machine or user lose their patience and bail out.  We have all experienced this frustration at some point.

In telephone-based systems, any type of information has to be conveyed through the audio channel, including instructions for users on what to say, the information requested, and the possible clarification of any misunderstanding between the human and the machine (including the infamous “I think you said …”). Anything spoken has to be limited in time, in a rigid, turn-taking fashion between the system and the user. That is why telephone-based systems are so “clunky”, to the point where more often than not, they are quite annoyingly mechanical and repetitive. Although I have built some of the most sophisticated and well-crafted telephone-based speech recognition systems for over a decade — and these systems are used by millions of people every week, — I never heard anyone say: “I love my provider’s automated customer care service. It is so much fun to use it that I call it every day!”

Then came the smartphone. The fact that a smartphone includes a telephone is marginal. A smartphone is a computer, a camera, a media appliance, an Internet device, a computer game box, a chatterbox, a way to escape from boring meetings, and yes, also a telephone, for those who insist on making calls. Although it hasn’t happened yet, smartphones have the potential to make clunky, telephone-based speech understating systems obsolete. Why? Because with a smartphone, you are not forced to communicate only through the narrow audio channel. In addition to speech, you can enter text and use touch. A smartphone can talk back to you, and display text, images, and videos. The smartphone speech recognition revolution started with Siri and Google voice search. Today, more and more smartphone services are adopting speech as an alternative interface blended with text and touch modalities.

The distinction between telephone-based, audio-only services versus smartphone speech understanding is not just in the multi-modality of the communication protocol. The advent of fast broadband Internet has freed us from the necessity of having all the software on one device. Speech recognition can run in the cloud with the almost unlimited power of server farms and can truly recognize pretty much everything that you say.

Wait a minute. Just recognize? How about understanding the meaning of what you say?

There is a distinction between recognition, which is the mere transcription of the spoken words, and understanding, which is the extraction and representation of the meaning. The latter is the domain of natural language processing. While progress in speech recognition has been substantial and persistent over the past decades, natural language has moved forward more slowly. Siri and its cloud-based colleagues are capable of understanding more and more, and even some of the nuances of our languages. The time for speech recognition and natural language understanding is finally ripe for natural machine and human communication.

But…is that all? Is talking to our phones and tablets or open rooms — as in the upcoming home automation applications — the ultimate goal of spoken communication with machines?

No. Social robots are the next revolution.

It has been predicted by the visionary work of pioneers like Cynthia Breazeal, and corroborated and validated by the recent announcements of social robot products coming to market. This is made possible by the confluence of the necessary technologies that are available today.

Social robots have everything that personal assistants have—speech, display, touch—but also a body that can move, a vision system that can recognize local environments, and microphones that can locate and focus on where sounds and speech are coming from. How will a social robot interact with speech among all the other modalities?

We went from prompted speech, like in the telephone systems, to personal assistants that can come to life on your smartphone.  For a social robot, there should neither be a “speak now” timeout window, nor the antiquated  “speak after the beep” protocol, nor a button to push to start speaking. Instead, a social robot should be there for you all the time and respond when you talk to it. You should be able to naturally cue your robot that you are addressing it by speaking its name (like Star Trek’s “Hello Computer!”) or simply by directing your gaze toward it while you talk to it. Fortunately, the technology is ripe for this capability, too. Highly accurate hot-word speech recognition—like in Google Now—and phrase spotting (a technology that allows the system to ignore everything that is spoken except for defined key-phrases) are available today.

While speech recognition accuracy is always important, we should always consider that social robots can have multiple communication channels: speech, touch, vision, and possibly others.  In many cases, the different channels can reinforce each other, either simultaneously or sequentially, in case one of them fails. Think about speech in a noisy situation. When speech is the only alternative, the only recourse is to repeat (at nauseam) the endless “I did not understand what you said, please say that again?”  For social robots, however, touch and gesture are powerful alternatives to get your intent across when placed in situations where speech is difficult to process.

In certain situations, two alternative modalities can also complement each other. Consider the problem of identifying a user. This can be done either via voice with a technology, called speaker ID, or through machine vision using face recognition. These two technologies can be active at the same time and cooperate, or used independently — speaker ID when the light is too low, or face ID when the noise is too high.

Looking ahead, speech, touch, and image processing can also cooperate to recognize a person’s emotion, locate the speaker in the room, understand the speaker’s gestures, and more. Imagine the possibilities when social robots can understand natural human behavior, rather than the stilted conventions we need to follow today for machines to understand what we mean.

I have spent my career working at the forefront of human-machine communication to make it ever more natural, robust, textured – and, yes, magical. Now, we are at the dawn of the era of social robots.  I am thrilled to help bring this new experience to the world.  Just wait and see.

If you liked this article, you may also be interested in:

• Embodied communication: Looking for systems that mean what they say
• In-corporating body language into NLP (or, More notes on the design of automated body language)
• The death of search (or, My dysfunctional relationship with Siri)
• Google’s response to Apple’s Siri: Buy Cleversense

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

It is a fantastic time for technology. We live in a more connected world, and compared to even a few decades ago, we have vastly improved access to information and content, and have dramatically expanded our ability to connect with one another in interesting new ways to socialize despite time and distance.

As much as I love what technology does for us, and it certainly does quite a bit, our relationship with technology can still be dramatically improved.

Technology can feel dehumanizing. Technology often demands that we think and act more like machines to use it. It treats us like technology, beeping, buzzing, and pushing data and information at us without concern for politeness. It interrupts our conversations, not caring if we understand the information, or determining if the information is wanted. I let my son play on my smartphone once, and now I’m rewarded with daily updates about zombie harvesting – enough! We need to do better.

We need to humanize technology so that it treats us the way we want to be treated.

But there is an even deeper reason to improve this interaction. My fundamental gripe with technology is that it fails to support a more holistic human experience. It falls short in giving people a personally meaningful, emotionally engaging experience.

This shortfall actually limits the ability of technology to support and empower us to succeed, thrive, and grow.

Really?

Yes. Here’s why.

People think, feel, and act not just because of information, but also because of personally meaningful experience. Things have to matter to us, not just inform us, to effect real change.  There is a rich scientific literature on the role of emotion in intelligent decision-making and behavior change.

People who make careers of helping other people — educating others, helping to heal or care for others, or helping people to take better care of themselves — all know the power of high touch on positive human outcomes. Learning new knowledge and skills opens opportunities for growth. Adopting behaviors that keep you healthier increases your chances of thriving as you age. Forging and maintaining stronger relationships with those you love promotes mental and emotional wellbeing. Your life is better when you can do these things.

We should care about these things at face value. And there is urgency to do so.

We are facing juggernaut societal challenges where we cannot rely solely on our institutions to take care of us, to educate us, to keep us healthy. It is too expensive and the growing demand continues to outpace available institution and human resources. We see the headlines: the education crisis, the healthcare crisis, the eldercare crisis, and more.

Millions of people are grappling with these issues every day, and we can help relieve the crushing burden on our institutions by empowering people in their homes.

My vision is that:

… Even in the most time-crunched families, parents will have a reliable, high-quality partner in education for all children. Including our youngest learners who shall enter school ready to learn, or children with special needs, receptive to the quality education they need.

… Those people who struggle with health or chronic disease issues will have the right kind of tools to change behavior, and can independently manage their health and improve their treatment.

… Elders will be able to age independently in their homes with the help of a technology that feels much more like an attentive companion than yet another digital tool or a “Big Brother” monitoring system, relieving pressure on oversubscribed institutions, and remain emotionally connected to their families and loved ones despite distance.

And so much more.

To realize this vision, we all know that technology must be part of the solution. It’s affordable. It’s scalable. It’s accessible at home. This is a good start. But it needs to be more.

Personal technology needs to go beyond giving us access to information anywhere we are. It needs to go beyond opening a communication channel between people. It needs to go beyond being easy to use. It needs to go beyond being cool and desirable. All of these attributes are still critical pieces. But they do not yet form the full answer.

Technology needs to support what people need to be the most empowered to effect positive change in their life and for their loved ones. High tech must support high touch engagement for the best and most lasting results. Only then will technology maximally empower our human ability to fully engage with it — how we experience, think and behave at the intersection of our cognitive, social, emotional and physical capabilities — to succeed, thrive, and grow with technology like never before.

Technology needs to be humanized.

Computing has gone through several waves, each resulting in billion dollar industries and impacting millions if not billions of people’s lives on global scale. We’ve benefitted from the wave of information, the wave of mobility, the wave of social media. We’re seeing more and more wearable computing and the Internet of Things that combines these three.

Now, we are on the threshold of the next wave of computing: emotion.  And social robotics is a particularly powerful expression of this next wave.

This is what social robotics is about. This is why it is so transformative. This is why it is revolutionary.

This is why it will change your life.

This post was originally published on blog.myjibo.com, where Cynthia Breazeal will be curating articles about social robots.

If you liked this article, you may also be interested in watching Cynthia’s ROBOTS: From imagination to market talk, or in reading the following articles:

• Robots14 brings it all together
• Robots Podcast: A Code of Ethics for HRI Practitioners
• What should a robot do? Designing robots that know right from wrong
• Consumer AI devices entering the marketplace

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.