Robohub.org
 

Stefanie Tellex: Learning Models of Language, Action and Perception for HRC | CMU RI Seminar


by
10 October 2016



share this:

Link to video of seminar on YouTube

Abstract: “Robots can act as a force multiplier for people, whether a robot assisting an astronaut with a repair on the International Space station, a UAV taking flight over our cities, or an autonomous vehicle driving through our streets. To achieve complex tasks, it is essential for robots to move beyond merely interacting with people and toward collaboration, so that one person can easily and flexibly work with many autonomous robots. The aim of my research program is to create autonomous robots that collaborate with people to meet their needs by learning decision-theoretic models for communication, action, and perception. Communication for collaboration requires models of language that map between sentences and aspects of the external world. My work enables a robot to learn compositional models for word meanings that allow a robot to explicitly reason and communicate about its own uncertainty, increasing the speed and accuracy of human-robot communication. Action for collaboration requires models that match how people think and talk, because people communicate about all aspects of a robot’s behavior, from low-level motion preferences (e.g., “Please fly up a few feet”) to high-level requests (e.g.,”Please inspect the building”). I am creating new methods for learning how to plan in very large, uncertain state-action spaces by using hierarchical abstraction. Perception for collaboration requires the robot to detect, localize, and manipulate the objects in its environment that are most important to its human collaborator. I am creating new methods for autonomously acquiring perceptual models in situ so the robot can perceive the objects most relevant to the human’s goals. My unified decision-theoretic framework supports data-driven training and robust, feedback-driven human-robot collaboration.”




John Payne

            AUAI is supported by:



Subscribe to Robohub newsletter on substack



Related posts :

Watch the IROS 2026 plenary and keynote talks

  16 Nov 2026
Catch up on the invited presentations from the International Conference on Intelligent Robots and Systems.

Robot Talk Episode 170 – Robots in the real world, with Daniela Shchegelskaya

  13 Nov 2026
In the latest episode of the Robot Talk podcast, Claire chatted to Daniela Shchegelskaya from the University of Twente about getting robots to operate reliably in safety-critical environments.

AI100 report released: embodied AI highlighted as grand challenge

  11 Nov 2026
Embodied AI is listed as one of the most inspiring open grand challenges.

AIhub coffee corner: Are humanoids the future of embodied AI?

  09 Nov 2026
The AIhub coffee corner captures the musings of AI experts over a short conversation.

Robot Talk Episode 169 – Robot sensing inspired by nature, with Dexter Shepherd

  06 Nov 2026
In the latest episode of the Robot Talk podcast, Claire chatted to Dexter Shepherd from the University of Sussex about using artificial intelligence to help robots understand and adapt to their environment.

Robots in society, business and culture: October 2026

We showcase a selection of robotics stories from society, business, research and culture.

Muscle coordination can be mimicked by electric motors – it could lead to improved robots

  02 Nov 2026
Researchers built a physical model using ordinary electric motors to try and simulate how muscles may perform tasks.

Robot Talk Episode 168 – Advancing robotics with novel electronics, with Zlatka Stoeva

  30 Oct 2026
In the latest episode of the Robot Talk podcast, Claire chatted to Zlatka Stoeva from DZP Technologies about flexible, stretchable and soft printed electronics for dexterous robots.


↑


AUAI is supported by:







Subscribe to Robohub newsletter on substack




 















©2026.05 - Association for the Understanding of Artificial Intelligence