AMS 557: Fundamentals of Reinforcement Learning
Formerly AMS 691: Topics in Applied Mathematics and Statistics. I took this course under its former designation, with the topic Fundamentals of Reinforcement Learning.
This page is part of my personal academic record, not an official course website.
Course description
A deep understanding of reinforcement learning (RL) is essential for machine learning researchers, data scientists, and practicing engineers working in areas such as artificial intelligence, machine learning, data/network science, natural language processing, and computer vision, among others. RL has found applications in our everyday lives, such as large language models (LLMs), AlphaGo, AlphaFold, autonomous driving, healthcare, etc. This course will provide an introduction to the field of RL and emphasize hands-on experiences. Students are expected to become well versed in key ideas and techniques for RL through a combination of lectures, written assignments, and coding assignments. Students will advance their understanding and the field of RL through a course project. The topics that will be covered (time permitting) include, but are not limited to:
- Markov Decision Processes (MDPs);
- Value Functions;
- Policy Iteration and Value Iteration;
- Monte Carlo Methods;
- Temporal-Difference (TD) Learning;
- SARSA and Q-learning;
- \(\mathrm{TD}(\lambda)\);
- (Linear) Function Approximation;
- Policy Gradient Algorithms;
- Other topics (e.g., multiagent RL, RL for LLM Reasoning).
Textbooks and resources
Supplementary textbooks
- Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto, 2nd edition, 2018, Bradford Books
- Artificial Intelligence: A Modern Approach by Stuart J. Russell and Peter Norvig, 4th edition, 2022, Pearson
- Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville, 2016, The MIT Press
- Reinforcement Learning: State-of-the-Art, edited by Marco Wiering and Martijn van Otterlo, 2012, Springer
- Dynamic Programming and Optimal Control Vol. I by Dimitri P. Bertsekas, 4th edition, 2017, Athena Scientific
- Dynamic Programming and Optimal Control Vol. II: Approximate Dynamic Programming by Dimitri P. Bertsekas, 4th edition, 2012, Athena Scientific
- Applied Probability Models with Optimization Applications by Sheldon M. Ross, 1992, Dover Publications
- Optimization and Control by R. Weber, 2016