Seminar Series

Abstract:

While developing curiosity in robots has many potential benefits, including improving lifelong learning capabilities, there are also potential risks associated with robots being curious when people do not expect or want robots to be curious, especially when the robot already has work tasks to perform. In an online experiment (N=30), we examined people's perceptions and interpretations of four levels of curious robot behavior. We also examined the impacts of the robot being on-duty vs. off-duty when it engaged in curious behaviors. The results of this study have implications for the design and development of robots that engage in curiosity-driven learning.

Abstract:

Robots that curiously learn about the world will need to ask humans for help. For example, consider a new office aide robot that is learning about the building. It may need to ask humans what particular locations are called or where to find particular people or objects. As a first step towards developing such a curious information-seeking robot, we present a model of humans' helpfulness towards a robot in an office environment, learnt from online user study data. Our model disaggregates between the individual and contextual factors involved in human help-giving behavior, and our associated POMDP planning framework enables the robot to curiously learn how inherently helpful humans are over repeated interactions with them. Our evaluation user study demonstrates that our model reaches 1.75X more rooms than a model without individual factors and receives 3.39X fewer rejections when asking for help than a model without contextual factors (the state of the art).

Abstract:

Can intelligent agents explore their environment and learn general knowledge that prepares them for future downstream tasks? Task-specific reinforcement learning can solve the training tasks, but generalizes poorly. We propose Plan2Explore, a self-supervised agent that explores by planning for novelty and learns a model of its environment. This world model provides general and reusable knowledge, which enables Plan2Explore to adapt to specific downstream tasks in zero-shot or few-shot manner through planning in its own imagination. Plan2Explore obtains state-of-the-art zero-shot and few-shot performance on continuous control benchmarks with high-dimensional input images.

Abstract:

Model-based curiosity combines active learning approaches to optimal sampling with the information gain based incentives for exploration presented in the curiosity literature. Existing model-based curiosity methods look to approximate prediction uncertainty with approaches which struggle to scale to many prediction-planning pipelines used in robotics tasks. We address these scalability issues with an adversarial curiosity method minimizing a score given by a discriminator network. This discriminator is optimized jointly with a prediction model and enables our active learning approach to sample sequences of observations and actions which result in predictions considered the least realistic by the discriminator. We demonstrate progressively increasing advantages as compute is restricted of our adversarial curiosity approach over leading model-based exploration strategies in simulated environments. We further demonstrate the ability of our adversarial curiosity method to scale to a robotic manipulation prediction-planning pipeline where we improve sample efficiency and prediction performance for a domain transfer problem.

Abstract:

There are many tasks a robot may be asked to perform when deployed in a novel environment. Often these tasks may be unknown to the robot apriori. With this in mind, we consider model-based curiosity as a task-agnostic way to learn about the world. Using a domain with compositional properties, we consider three aspects of manipulation problems that a robot may use to drive its curious exploration in the world. First, we show how curiosity can enable a robot to learn about object-specific properties, such as an object’s center of mass. Next, we develop an active learning framework that enables a robot to efficiently learn about the compositional dynamics of its environment. Finally, we present a method for a robot to discover object-specific properties jointly with the environment’s dynamics.

Abstract:

In order to integrate into human environments, robots need to be aware of how their actions are perceived. This is challenging because the process by which observers form behavioral attributions is nebulous. We model behavioral attribution as probabilistic classification of observed trajectories and show how such models can be used to select plans that elicit desired attributions. We show how to construct these models by iteratively collecting human data and demonstrations to improve their performance. To evaluate this approach, we consider a simplified 2D home environment and attributions of curiosity, competence and brokenness to a cleaning robot.

Abstract:

A long-term goal for researchers is to build intelligent robots that can perform known tasks robustly and are capable of handling novel scenarios and adapting to new tasks. This goal envisions that these intelligent machines will continually learn from their curious or purposeful interactions with their environment. These interactions typically involve a robot to perceive, plan, and execute an action and then reason over the change induced by the actions. To seamlessly learn from the interactions, it is essential to equip an intelligent robot with efficient ways to localize and track object entities in motion. In the context of manipulation tasks, estimating and tracking 6D poses of objects from sensor data is an important problem. To address this problem, we present a novel deep neural network for 6D pose matching named DeepIM, which matches hypothesized poses (rendered images) of an object against the observed input image to produce accurate results. Given an initial pose estimation of an object, our network can iteratively refine the pose by matching the rendered image against the observed image. The network is trained to predict a relative pose transformation using a disentangled representation of 3D location and 3D orientation and an iterative training process. Experiments on two commonly used benchmarks for 6D pose estimation demonstrate that DeepIM achieves large improvements over state-of-the-art methods. We furthermore show that DeepIM is able to match previously unseen objects.

Abstract:

We address the problem of efficient exploration for transition model learning in the relational model-based reinforcement learning setting without extrinsic goals or rewards. Inspired by human curiosity, we propose goal-literal babbling (GLIB), a simple and general method for exploration in such problems. GLIB samples relational conjunctive goals that can be understood as specific, targeted effects that the agent would like to achieve in the world, and plans to achieve these goals using the transition model being learned. We provide theoretical guarantees showing that exploration with GLIB will converge almost surely to the ground truth model. Experimentally, we find GLIB to strongly outperform existing methods in both prediction and planning on a range of tasks, encompassing standard PDDL and PPDDL planning benchmarks and a robotic manipulation task implemented in the PyBullet physics simulator.

Abstract:

We will talk about our research projects in this seminar talk. In the first half of this talk, we will discuss a proposed self-supervised framework that enables us to learn various tasks to better understand object affordance in context. These tasks include: 1) finding a set of plausible locations to insert a given object into a context; 2) hallucinating human pose according to the interacting object; 3) hallucinating 6D object pose for a given object in a scene. In the second half of the talk, we will talk about learning contingency awareness in the visual predictive model. The idea of contingency awareness is to learn a model that can separately predict the deterministic forward dynamics and stochastic diversity caused by intrinsic object property and environment.

Abstract:

In order to interact with the world, agents must be able to predict the results of the world's dynamics. A natural approach to learn about these dynamics is through video prediction, as cameras are ubiquitous and powerful sensors. Direct pixel-to-pixel video prediction is difficult, does not take advantage of known priors, and does not provide an easy interface to utilize the learned dynamics. Object-centric video prediction offers a solution to these problems by taking advantage of the simple prior that the world is made of objects and by providing a more natural interface for control. However, existing object-centric video prediction pipelines require dense object annotations in training video sequences. In this work, we present Object-centric Prediction without Annotation (OPA), an object-centric video prediction method that takes advantage of priors from powerful computer vision models. We validate our method on a dataset comprised of video sequences of stacked objects falling, and demonstrate how to adapt a perception model in an environment through end-to-end video prediction training.

Abstract:

Abstract:

Abstract:

Abstract:

In the course of exploring the world, agents must be able to not just understand how they might use objects to accomplish their goals in novel ways, but also learn from their explorations and generalize successful strategies to other situations. In this talk, I will discuss a set of experiments exploring how people plan and learn new strategies in a physics-based puzzle solving task, and propose models that explain these behaviors. First, I will discuss the Virtual Tools Game that was designed to study rapid trial-and-error learning in people, and describe the Sample, Simulate, UPdate (SSUP) model that explains people’s behavior on this game. I will then discuss recent work studying how people learn and generalize across levels of this game. These experiments show that people consistently learn strategies rather than generically better world models, and selectively transfer these strategies when the crucial context for the strategy is met. These strategies are themselves composable with physical knowledge: people can adjust their strategies to account for new object weights despite no direct interaction experience with these objects. Together, this suggests that people quickly learn abstract strategies that go beyond simple model-free policies and are instead object-oriented, adaptable, and can be composed with model-based variables. Finally, I will discuss a model that formalizes these strategies and the learning process as developing logical program policies over a spatial, object-oriented DSL.

Abstract:

On our presentation at the Honda curiosity summit in late 2019, we presented our work on meta-learning curiosity algorithms, showing how our search had discovered two novel algorithms with super-human performance: a very simple one we could understand and a more complex one we couldn't. Since then, we've understood that the machine re-discovered the concept of cycle consistency, proposed in the GAN literature, but never before applied it to curiosity. In the first part of our talk, we will briefly describe how this algorithm was obtained and how it leverages this concept for efficient exploration. In the second part of our talk, we will present our preliminary work on making algorithms like AlphaGo or MuZero plan more efficiently by being curious in their plans.

Abstract:

We consider multiple agents using different strategies to compete for resources with a diverse distribution of rewards. Statistical physics shows that two kinds of equilibria are possible: (1) where some agents “settle” on a fixed resource while others visit diverse sites, and (2) where all agents pursue a similar strategy of visiting diverse sites. The first equilibrium shows a highly skewed reward distribution; in the second equilibrium most agents are similarly successful. We show that a population of agents can learn these equilibrium strategies through reinforcement learning. In conventional Q-learning, the population of agents learns the equilibrium strategy with skewed rewards. If we add curiosity, an intrinsic motivation to explore, Q-learning produces the second equilibrium in which most agents get similar average rewards. Thus, curiosity increases equality.

Abstract:

Granular media is common to many manipulation tasks in human environments. In this work, we implement an end-to-end framework for manipulating granular media. We seek to answer three key questions: (1) How do we represent this granular media in a manner that allows for effective manipulation in the real world? (2) How can we learn the dynamics of different granular media given our representation (i.e. how this media changes when an action is applied to it)? (3) How can we use our granular media representation and dynamics model for planning? In this talk, I will discuss our process of building each of the key components of our framework and how we can leverage curiosity for effective manipulation in the real world.

Abstract:

Although people often perceive robots as having some characteristics associated with animacy, less is known about how people view curiosity in robots. Taking a sociocultural approach, we explored children’s ideas about robots within a social and cultural context. We examined how 3 to 6 year old children’s ideas about robots varied based on their parents’ STEM vs. non-STEM occupation in a play session with a robot. We first explored how children and parents interacted with a robot during a play session, and then linked elements of the parents’ interactions to children’s ideas about the cognitive abilities of the robot. Next, we connected the ways parents and children interacted with the robot to how children judged the animacy of the robot. Finally, we asked parents and children two questions about curiosity: “How would you know if a robot was curious?” and “If robots could be curious, what do you think they would be curious about?” Participants gave a variety of responses to these questions, saying things such as curious robots would ask meaningful questions and understand the answers, explore places and objects on their own, be playful, and be curious in the same ways that people are curious. Alternatively, some people believed that robots cannot or should not be curious, saying that robots may not possess the qualities of a living thing necessary to be curious (e.g., a brain). Interestingly, some of these responses to the questions about curiosity related to other variables including: children’s animacy judgement of the robot, the proportion of animate pronouns the child used for the robot during the play session, the age of the child, and the parent’s STEM occupation. These results demonstrate the importance of broadening our view of how children develop their understanding of robots to include parents’ background and parent-child interactions when playing with robots. Consideration of these social and cultural factors can reveal some of the informal ways that children are developing their understanding of emerging technologies, and how this may relate to curiosity in robots.