AD-802 (B) Reinforcement Learning Notes
5 notes for Reinforcement Learning, unit by unit.
Notes for AD-802 (B)
- Unit 1 · Short notes Introduction to Reinforcement Learning Definition of reinforcement learning framework and Markov Decision Process ($MDP$) · Policies · Value functions and Bellman equations · Exploration vs. exploitation strategies · Components and architecture of an RL agent · Common challenges and problems in reinforcement learning
- Unit 2 · Short notes Bandit Algorithms and Dynamic Programming Multi‑armed bandit algorithms: Upper Confidence Bound (UCB), Probably Approximately Correct (PAC), Median Elimination · Policy gradient methods for bandits · Full reinforcement learning and $MDP$ formulation · Bellman optimality principle · Dynamic programming techniques: value iteration, policy iteration, and policy evaluation · Extensions of DP and convergence analysis using contraction mapping
- Unit 3 · Short notes Q‑Learning, Function Approximation and Deep Q‑Networks Q‑learning algorithm and its relation to temporal‑difference methods · Eligibility traces in Q‑learning · Function approximation for value functions: linear, non‑linear, and least‑squares methods · Incremental vs. batch update methods · Deep Q‑Learning analysis and stability considerations · Deep Q‑Network (DQN) architecture
- Unit 4 · Short notes Advanced Q‑Learning, Policy Gradient, Hierarchical and POMDPs Fitted Q‑iteration and its applications · Advanced Q‑learning variants and handling deep Q‑learning problems · Imitation learning from optimal controllers · Integration of DQN with policy‑gradient approaches · Policy‑gradient algorithms for full reinforcement learning · Hierarchical reinforcement learning concepts
- Unit 5 · Short notes Actor‑Critic, Inverse RL and Current Trends Actor‑Critic methods and their variants · Inverse reinforcement learning (IRL) · Maximum entropy deep inverse reinforcement learning · Generative adversarial imitation learning (GAIL) · Recent trends in reinforcement learning architectures