Tantárgy adatlapja
The goal of this course is to highlight the foundations of reinforcement learning (RL) and its general applicability to robotic problems. In addition to reviewing RL-fundamentals, the course includes a brief review of classical approaches to basic robotics computational problems (direct and inverse kinematical problem, etc), in order to help the deeper understanding of how modern RL-approaches work.
Discussed topics include:
- General concepts of deep neural networks
- Modern architectures, training strategies and fine details of practical realizations
- General formalism of the RL-problem
- The main RL-apporaches: value-based and policy-based
- Latest results on modern value-based and policy-based approaches, hybrid methods I.
- Latest results on modern value-based and policy-based approaches, hybrid methods II.
- Environmental simulators in a nutshell (pyBullet, MuJoCo, etc)
- Baseline algorithms in existing simulated environments
- Recap on theoretical aspects of robotic manipulators
- Constructional aspects of a new simulated environment, interfacing with a modern RL-algorithm
- Training strategies of the realized environment-agent combination, debug opportunities
Sim-to-real considerations and the necessary extensions to the created combinaton
List of publications:
I. Goodfellow, Y. Bengio, A. Courville, “Deep learning,” MIT Press, www.deeplearningbook.org , 2016 R. S. Sutton, A. G. Barto, “Reinforcement Learning: An Introduction,” 2nd edition, MIT Press, incompleteideas.net/book/the-book-2nd.html , 2020 Cs. Szepesvári, “Algorithms for Reinforcement Learning,” Morgan&Claypool Publishers, https://sites.ualberta.ca/~szepesva/rlbook.html , 2009 J. Achiam, “Spinning Up Documentation,” https://spinningup.openai.com , 2020 L. Sciavicco, B. Siciliano: “Modelling and Control of Robot Manipulators,” 2nd edition, Springer, 1999 Lectures of UC Berkeley “Deep Reinforcement Learning” by Sergey Levine, https://rail.eecs.berkeley.edu/deeprlcourse/ 2021
