عنوان انگلیسی مقاله:
From inverse optimal control to inverse reinforcement learning: A historical review
ترجمه فارسی عنوان مقاله:
از کنترل بهینه معکوس تا یادگیری تقویتی معکوس: یک بررسی تاریخی
Sciencedirect - Elsevier - Annual Reviews in Control, Corrected proof. doi:10.1016/j.arcontrol.2020.06.001
Nematollah Ab Azar ∗, Aref Shahmansoorian, Mohsen Davoudi
Inverse optimal control (IOC) is a powerful theory that addresses the inverse problems in control systems, robotics, Machine Learning (ML) and optimization taking into account the optimal manners. This paper reviews the history of the IOC and Inverse Reinforcement Learning (IRL) approaches and describes the connections and differences between them to cover the research gap in the existing literature. The gen- eral formulation of IOC/IRL is described and the related methods are categorized based on a hierarchical approach. For this purpose, IOC methods are categorized under two classes, namely classic and modern approaches. The classic IOC is typically formulated for control systems, while IRL, as a modern approach to IOC, is considered for machine learning problems. Despite the presence of a handful of IOC/IRL meth- ods, a comprehensive categorization of these methods is lacking. In addition to the IOC/IRL problems, this paper elaborates, where necessary, on other relevant concepts such as Learning from Demonstration (LfD), Imitation Learning (IL), and Behavioral Cloning. Some of the challenges encountered in the IOC/IRL problems are further discussed in this work, including ill-posedness, non-convexity, data availability, non- linearity, the curses of complexity and dimensionality, feature selection, and generalizability.
Keywords: Inverse optimal control | Inverse reinforcement learning | Learning from demonstration | Imitation learning