Amy Zhang
link
Abstract: The benefit of multi-task learning over single-task learning relies on the ability to use relations across tasks to improve performance on any single task. While sharing representations is an important mechanism to share information across tasks, its success depends on how well the structure underlying the tasks is captured. In some real-world situations, we have access to metadata, or additional information about a task, that may not provide any new insight in the context of a single task setup alone but inform relations across multiple tasks. While this metadata can be useful for improving multi-task learning performance, effectively incorporating it can be an additional challenge. In this talk, we explore various ways to utilize context to improve positive transfer in multi-task reinforcement learning.
Bio: Amy Zhang is an assistant professor at UT Austin in the Chandra Family Department of Electrical and Computer Engineering . My work focuses on improving generalization in reinforcement learning through bridging theory and practice in learning and utilizing structure in real world problems. Previously I was a research scientist at Meta AI and a postdoctoral fellow at UC Berkeley. I obtained my PhD from McGill University and the Mila Institute in 2021, and also previously obtained an M.Eng. in EECS and dual B.Sci. degrees in Mathematics and EECS from MIT.
Pulkit Agrawal
link
Abstract: Robots are getting smarter at converting complex natural language commands describing household tasks into step-wise instructions. Yet, they fail to actually perform such tasks! A prominent explanation for these failures is the fragility and inability of the low-level skills (e.g., locomotion, grasping, pushing, object re-orientation, etc.) to generalize to unseen scenarios. In this talk, I will discuss a framework for learning low-level skills that surpasses limitations of current systems at tackling contact-rich tasks and is real-world-ready generalizes, runs in real-time with onboard computing, and uses commodity sensors. I will describe the framework using the following case studies. (i) a dexterous manipulation system capable of re-orienting novel objects. (ii) a quadruped robot capable of fast locomotion and manipulation on diverse natural terrains. (iii) learning from a few task demonstrations of an object manipulation task to generalize to new object instances in out-of-distribution configurations.
Bio: Pulkit is the Steven and Renee Finn Chair Assistant Professor in the Department of Electrical Engineering and Computer Science at MIT, where he directs the Improbable AI Lab. His research interests span robotics, deep learning, computer vision, and reinforcement learning. His work received the Best Paper Award at Conference on Robot Learning 2021 and Best Student Paper Award at Conference on Computer Supported Collaborative Learning 2011. He is a recipient of the Sony Faculty Research Award, Salesforce Research Award, Amazon Research Award, a Fulbright fellowship, etc. Before joining MIT, he co-founded SafelyYou Inc., received his Ph.D. from UC Berkeley, and Bachelor's degree from IIT Kanpur, where he was awarded the Directors Gold Medal.