APR Seminar: Shie Mannor (Technion, Israel Institute of Technology)

January 19, 2017 | 1:10pm - 2:10pm

APR Seminar: Shie Mannor (Technion, Israel Institute of Technology)

Mudd Hall 303
Title: Risk in Reinforcement Learning: Nothing Ventured, Nothing Gained

Abstract: In many sequential decision problems all that we have is a record of historical trajectories. Building dynamic models from these trajectories and ultimately sequential decision policies may result in much uncertainty and bias. In this talk we consider the question of how to create control policies from existing historical data and how to better sample trajectories so that future control policies would be better. This question has been central in reinforcement learning in the last decade if not more, and involves methods from statistics, optimization, and control theory.

We will focus on two possible remedies to uncertainty in sequential decision problems. The first is to robustify the model parameters and the second is to use risk measures such as the conditional value-at-risk as the objective to be optimized rather than the ubiquitous expected reward. We consider the complexity and efficiency of both approaches focusing on large-scale decision problems. Our main theme is that considering risk or robustifying is essential to obtain resilience to model uncertainty and  model mismatch.

We will then briefly describe two challenging real-world domains that have been studied in our research group in collaboration with experts from industry and academia: diabetes care management in healthcare and asset management in high-voltage transmission grids. For each domain we will describe our efforts to reduce the problem to its bare essentials as a reinforcement learning problem, the algorithms for learning the control policies, and some of the lessons we learned.
 
Bio: Shie Mannor is a professor of Electrical Engineering at the Technion, Israel Institute of Technology. Shie graduated from the Technion with a PhD in Electrical Engineering in 2002. He was a Fulbright postdoctoral scholar at LIDS at MIT from 2002 to 2004. He was at the Department of Electrical and Computer Engineering at McGill University from July 2004 until August 2010, where he held the Canada Research Chair in Machine Learning. Shie has been with the Department of Electrical Engineering at the Technion since 2008 where he is currently a professor. His research interests include machine learning, planning and control, and networks.

Shie has published over 70 journal papers and over 130 conference papers and holds 8 patents. He is an associate editor of Operations Research and of Mathematics of Operations Research and an action editor of the Journal of Machine Learning Research (JMLR). His research awards include several best paper awards, the Henri Taub Prize for Academic Excellence, an ERC Starting Grant, an HP Faculty Award and a Horev Fellowship


500 W. 120th St., Mudd 315, New York, NY 10027    212-854-2942                 
©2014 Columbia University