Research

Our ultimate goal is to create new types of machines that can acquire an interest in learning and knowledge, the ability to learn and discover, and the ability to interact with the world and others. We want to develop Curious Minded Machines that use curiosity to serve the common good by understanding people’s needs, empowering human productivity, and ultimately addressing complex societal issues.

Led by Professor Srinivasa, the University of Washington team is addressing the challenges of enabling robots to work effectively in human environments. By taking a page from child-learning through exploration, the team aims to construct a mathematical model of curiosity.

Read More

When placed in a new and sometimes challenging environment, children exhibit curiosity and explore, unlike today’s robots.

The UW team has a goal to build a mathematical model of curiosity, for the robots to formalize qualitative ideas and assumptions concretely, and with such a computational model implemented on robots to autonomously demonstrate curiosity.

When interacting with humans, the team believes that curiosity can drive a robot to learn what might be most useful in the long term or relevant for their users, and a robot perceived as curious or inquisitive will be more acceptable to its users given its limitations, such as the inability to perform some tasks or mistakes it makes in available tasks.

The team’s approach rests on three intertwined pillars of research:

  • To understand the psychology of human curiosity via the analysis of human subject experiments both in person and via Amazon Mechanical Turk
  • To build a formal mathematical model of curiosity as a cooperative game between the robot, its human agents, and its environment, with partial observability
  • To implement algorithms on real physical systems and test new theories of curiosity via physical experiments and human-robot interaction studies

Led by Professors Kaelbling, Lozano-Perez, Roy and Tenenbaum, the MIT team is addressing a key limitation in robotic action planning by focusing on establishing a causal theory of sensor percepts to predict future percepts and the effect of future actions.

Read More

Any such theory must be a combination of a low-level, continuous representation based on sensor data and motor commands, and a discrete, abstract representation that allows efficient inference, planning and communication. In robotic task and motion planning, for example, a common approach is to represent the world with a fixed, symbolic representation. 

This formalism relies on hand-coded primitives that connect the symbolic representation to the underlying physics representation that is strongly tied to a particular geometric form. Such systems typically have no ability to acquire new concepts, with limited predictive capabilities, and consequently can’t adapt to changes from the existing model or even notice such changes.

In contrast, a truly generally intelligent robotic agent must be able to acquire a theory of the world that captures the underlying physics, including models of the sensors and motors, and allows for discrete abstractions that enable computational efficiency. 

With the postulate that an essential driver for acquiring a complete and consistent theory is curiosity, that is, the purposeful acquisition of information aimed at improving the agent’s ability to predict what it observes in the environment.

The team takes its Curious Minded Machine approach as follows:

  • Building up a predictive theory of the world by learning, from experience (pushing, poking, etc.) and from interaction with human "teachers"
  • Robot-self-awareness in detecting certain phenomenon (in the physical world or in its interaction with humans) that its theory fails to predict well, followed by plan-generation to acquire information that will enable it to improve its predictive theory

Led by Professor Daniilidis, the University of Pennsylvania team is addressing today’s challenges in machine perception by learning from biological systems, applying an embodied, active and efficient approach to acquiring representations of the surrounding world and actions.

Read More

Progress in deep learning has enabled top performance in visual recognition and winning video games, largely facilitated by millions of annotated examples used in supervised techniques or millions of move-trials from game-playing. With non-modular architectures for visual recognition, however, these powerful tools could not easily scale to other problems such as localization and exploration for robotic applications. 

In this project, the team seeks to mimic biological learning by applying an embodied, active, and efficient approach to acquiring representations of the surrounding world and actions. Biological species can learn patterns and actions with a few active trials without external supervision. They learn equivariance and invariance without being exposed to thousands of transformed copies as is done in data augmentation today, and seem to seek stimuli selectively and actively to facilitate their knowledge acquisition.

The team focuses their pursuits particularly on the following:

  • Introducing models for embodied cognition driven by curiosity and triggered by first-person stimuli
  • Investigating curiosity as playing in an augmented reality environment
  • Studying the modularity of curiosity in the brain and efficient coding for curiosity in terms of information maximization and time constraints
  • Developing exploration architectures that explicitly encode innate knowledge about geometry and physics to enable event-triggered active perception