Source: YouTube/TechCrunch

Viv is an AI personal assistant launched at TechCrunch Disrupt NY. It promises to radically simplify our interface with everything. Viv is everything that Siri cofounders — Dag Kittlaus and Adam Cheyer — wanted to build with Siri, but couldn’t. Viv is a few years more advanced than Siri, but the big difference is that Steve Jobs preferred to keep Siri in a walled garden and not allow open partnerships.

Viv is a global platform that enables developers to plug into and create an intelligent, conversational interface to anything.

Cheyer and Kittlaus are focusing on creating a developer ecosystem so that Viv will be the way every device interacts with you in the future. They believe no one company can bring everything you need. But, one personal assistant can interact with all the companies and services that you need.

Viv does ‘conversational commerce’ with breathtaking speed and accuracy. I think you can forget about deep linking as the next mobile/online paradigm. Kittlaus described Viv as “the next major marketplace and channel for offering content, commerce and services.”

Viv will launch a gradually expanding developer program later in 2016. And as thousands and thousands of developers add to the Viv intelligence, the power of Viv expands. While the demo focused on showing Viv’s strengths on a phone, Viv is clearly intended to be the interface for all of our smart connected devices. And robots.

In fact, I’ll bet that my future home robot will be running on Viv.

Is there a downside?

What are could possibly go wrong in a future world full of personal assistants?

Well, while Dag Kittlaus was launching Viv at TechCrunch Disrupt NY, cofounder Adam Cheyer was in San Jose on the CHI2016 panel: “On the Future of Personal Assistants” with Phil Cohen,VoiceBox Technologies; Eric Horvitz, Microsoft Research; Rana El Kaliouby, Affectiva; and Steve Whittaker, University of California at Santa Cruz.

Firstly, the panel noted that the term ‘personal assistant’ was deceptive as we are transitioning from helpful but passive assistants, such as Siri, to active agents, such as Echo, Viv and perhaps Jibo, or other home robots in the future.

Assistants are just helpful. Agents go out into the world and perform actions.” said Phil Cohen, VoiceBox.

There are also huge potential implications in the transition from personal assistants to family assistants. Amazon’s Echo is an example of a family assistant, where the device is no longer in our hands or pockets, but sits in a room and interacts with a range of people.

How do we deal with multiple devices in various areas of our lives, ie. from car to kitchen to work to bed, and then, how do devices interact across our individual family members? Will we be creating functional or dysfunctional family agents? Navigating these issues of privacy, trust and boundaries should be at the forefront of our development.

Eric Horvitz, Microsoft, described the ‘penumbra – or bubble of data and devices’ that travels with each individual. We expect these agents to collaborate with us, but who else do our agents collaborate with in order to provide us services?

The next step is to consider how our agents can in the future interpret our emotional states. What inferences can be made, and shared, with whom? What control can be effected by these agents.

Rana El Kaliouby, Affectiva said “your fridge will understand your emotional state and help you make better decisions, if you want to change your behavior. That’s a very powerful promise.”

Cheyer was clear he doesn’t want “a plethora of different assistants… I want one assistant who can learn my preferences over time and do a better job.”

Although the intelligence might be in the cloud, the form factor of the device will give very different feel to how we interact, whether it’s a screen, TV, or a robot. And as we start to embody our assistants we change the relationship significantly.

Horvitz told an anecdote about going on holidays and turning back to bring Alexa along. Kind of like a member of the family.

Steve Whittaker, UCSC said that “we can use the metaphor of a house as an approach to designing different categories of personal assistants. A house is a collection of resources for family activities. At some level they are shared facilities for a family, but it is segmented.”

“Cortana developers are already tackling ideas of primary agency,” said Horvitz. “When I walk into a room, can I use someone else’s agent? And then is it my agent or some one else’s? And what level of shared agent? How do we develop the trust?”

“Both technically and ethically, how much access to the underlying personal information can someone else’s agent have?” added El Kaliouby.

Phil Cohen, VoiceBox questioned whether agents needed to track our beliefs, desires and intentions, or underlying emotions.

“What value or service is augmented intelligence providing in people’s lives? What tasks should a system be designed to address and for whom? Especially when a device is ambient (taking action on its own) not simply being addressed.”

 

Cheyer said that a useful assistant has to be transparent and allow you to change your mind about what is being shared. Whittaker said that while that might be the case for personal agents it would be very complex for family agents.

Horvitz pointed out that there are many elegant designers in the CHI community who can meet the challenge, as long as we focus on high value services that are tolerant of some uncertainty or error, so that there is a low cost of failure.

El Kaliouby added that we faced a moral obligation to give permission to some algorithms to take action. For example, some agents are showing ability to spot depression and suicidal states from things like social media posts. But what permission to they have to take action and what actions?

Whittaker pointed out that while spotting suicidal behaviors is a clear social good, generally people aren’t that good at identifying and tracking emotional states, so at what point does mood tracking create useful and actionable information?

Horvitz would like to see us address time management for people, seeing it as a huge problem that we’ve barely begun to scratch the surface of.

Cheyer sees time management or life coaching as just one of the domains that a system like Viv will start to have impact on. Although we’ll need better understanding of how we interact with AI and devices to be able to get to the next level: “Everyone is talking about the strengths of AI, but it’s not really going to solve all our problems. Where Siri was pretty dumb, Viv has light weight planning ability. Viv is able to do complex tasks with several steps which is a lot closer to the way we really ask for things. It’s not going to be doing the life planning things that Eric was asking for. But we’re working on taking actionable insights out of the unstructured data of our lives. And in the end that’s a very exciting development across all domains.”

In a room full of interactionists and ethicists, it’s no surprise that the first question to the panel was: “Is it good for society to be creating a slave class?”

This opened the door on a discussion that ranged from the ethics of ‘nagware’ up to lethal force. At which point the challenge was made for designers shift away from negativity and to create delightful solutions that would enhance and improve our lives. People-centered rather than task-centered design.

Cheyer concluded by bringing the discussion back to Viv. “Every 10 years if you look back we’ve had a new paradigm for interacting with our devices, from desktops, to web, to mobile. And each one had its problems. But I think the time is right for the assistant/agent paradigm and it’s going to be far more impactful than everyone realizes.”

Horvitz added, “The only thing I can promise for sure in the next 5 to 10 years is that we will have impacts and effects that no one has predicted.”

*note: quotes are not verbatim and have been edited for clarity.

The use of Cortana, an AI personal assistant created by Microsoft initially for Windows phones, has been a big hit in Japan and China. The Mandarin language version, named Xiaoice, is being used daily by a TV station to report the morning weather.

Pronounced Shao-ice, meaning “little bing” in Mandarin, the youthful-sounding female voiced Xiaoice has been downloaded by over 20 million people and is being used as a chatbot throughout China on various platforms such as Weibo and 163.com, and now as a weather reporter for a Chinese TV station.

The NY Times said that “Microsoft has been able to give Xiaoice a more compelling personality and sense of intelligence by systematically mining the Chinese Internet for human conversations.” Microsoft said that the average user of Xiaoice talks to her more than 60 times a month!

In doing the weather report, Microsoft says Xiaoice calls upon smart cloud technologies and big data to learn about and interpret weather readings and then deliver reports with her “unique artificial intelligent style of emotional comments.” According to MS, the AI is more human-like than any other system of its kind, scoring 4.32 out of 5 in linguistic naturalness tests. For comparison, humans have an average score of 4.76.

MS's deep learning software enables Xiaoice to structuralize weather data and also interact with live scenes as weather news is visually broadcast (Xiaoice doesn't have a presence; no face or screen avatar; just a voice). Xiaoice also has an an emotional component that enables it to offer instant comments on weather data. The result is that Xiaoice has taken over the position of daily weather presenter on the morning Dragon TV “Morning News.”

“Xiaoice is a critical artificial intelligence product for Microsoft and she has been first launched in China and Japan.” Dr. Yongdong Wang said, “We feel thrilled to introduce Xiaoice into live TV and to jointly open a new chapter in the history of the scientific technology and television. We will pay more attention to practicality by developing artificial intelligence products available to everyone rather than just put forward mere sci-tech concepts. As Xiaoice is gradually penetrating into human life, engaging herself in more jobs and playing more social roles, Microsoft expects her to bring more bliss to human beings.”

It's a misnomer to call Xiaoice a robot although many in the media call it that. In reality, Xioaice is an AI software product.

“A robot requires physicality as a way to affect physical changes on its environment,” says GoogleX roboticist Travis Deyle. “An avatar or digital-only personality may possess artificial intelligence, but without a physical body it is not a robot. To put it another way: Robots often incorporate aspects of character development that are mainstays of digital avatars in much the same way that a robot incorporates machine learning, computer vision, mechanics, electronics, etc.  But we don't call every machine learning algorithm, computer vision system, mechanical apparatus, and printed circuit board a robot… as then the term “robot” loses all meaning.”

Imagine an assistant that knows your schedule, contacts, interests and location, and that can communicate with you to help you stay informed, connected and on top of your game. This is more personalized than your PC, tablet or smartphone and, depending on whether physical or virtual, is in a pleasing, personal and compatible form. If these assistants catch on, they will begin anticipating and handling many of the little decisions that fill our days. The potential for this form of artificial intelligence has tech companies vying to lead the pack.

Apple’s Siri, Google’s Google Now, Microsoft’s Cortana, Facebook, and Baidu are all in various states of pursuing virtual assistants. But JIBO, Pepper, Cubic, EmoSpark and others are just now coming to market and are offering a physical component – a robotic presence – which they hope will enhance the experience.

2015 is the year physical products will be coming to market and available for experimentation and testing. Pepper ships in the summer in Japan, JIBO ships preorders in Q3 as does Cubic in the fall and EmoSpark in the summer.

These types of personal assistants rely on multiple activities including: speech recognition to determine and parse through spoken commands; predictive technology to anticipate what a user will likely do next; and determining and using contextual data (biographic data and situational information, e.g.: location, time and date). Apple, Google and Baidu – all of which attempt to provide this service – want their intelligent interface to fit all front ends, to be the universal contact point. They see a plentiful revenue stream if they are successful. Others want Apple and Google to redirect users to company sites with apps that understand their own content better, e.g.: directing requests for pizza to Dom, the digital front-end for taking Dominos Pizza orders.

Many are waiting to see how the competition plays out. Car companies and other smart product providers, fearing a variety of consequences outside of their control, aren’t enamored of letting Apple or Google (or any of the others) run their voice-powered front-end systems … thus their current intense race for in-house and startup development. But if Google or Apple or ??? gets to market first with a true digital assistant that people embrace and buy – one that can be virtual or physical – car companies may need to make their systems adapt to that system because it’s what everyone wants to use.

The key to the outcome of this race is whether a general purpose AI will be able to steer people through their digital world, or whether users would rather navigate to applications that are specialists (such as American Airlines or Dominos Pizza).

Natural language recognition – and the ability to analyze user desires from that language – is a very competitive science these days. Nuance, the biggest provider of voice and language solutions to the auto industry and a collaborator with IBM up until recently, is now competing with IBM’s Watson team and also Apple, which is rumored to be working on their own language processing system instead of using Nuance. A look at the What’s Next section of Nuance’s website is, to quote them, “The intersection of Science Fiction, super-pi, and technology innovation.” But a more descriptive quote from that page seems more appropriate: “Intelligent self-service delivers a better customer experience.” That’s what this race is all about.

• The second wave of social robots is here
• Amazon’s Echo: A point-of-sale even the kids can relate to
• The next era of conversational technology
• The death of search (or, My dysfunctional relationship with Siri)

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.