AD-802-B Reinforcement Learning - Jun 2025 Question Paper

Under each question, How to answer lists the points an answer should cover, outlined by CampusPrep from the syllabus. Check the details against your textbook.

1. 7 Marks
   aWhat is Reinforcement learning? State one appropriate real-life example.
   How to answer

   • Define reinforcement learning as learning by interaction with reward signal
   • Describe agent, environment, action, reward loop
   • State one real-life example e.g. game playing, robotics, self-driving
2. 7 Marks
   bExplain various approaches to implement reinforcement learning.
   How to answer

   • List major approaches: value-based, policy-based, model-based vs model-free
   • Briefly explain each approach
   • Mention examples like Q-learning, policy gradients, actor-critic
3. 7 Marks
   aDifferentiate between Reinforcement Learning and Supervised Learning.
   How to answer

   • Define supervised learning with labeled data
   • Define reinforcement learning with reward feedback
   • Compare on basis of feedback, goal, data and training
4. 7 Marks
   bWhat is the key objective of bandit algorithms with reference to reinforcement learning?
   How to answer

   • Define multi-armed bandit problem
   • State objective: balance exploration vs exploitation
   • Explain minimizing regret / maximizing cumulative reward
5. 7 Marks
   aAfter 12 iterations of the UCB 1 algorithm applied on a 4-arm bandit problem, we have $n_1 = 3, n_2 = 4, n_3 = 3, n_4 = 2$ and $Q_{12}(1) = 0.55, Q_{12}(2) = 0.63, Q_{12}(3) = 0.61, Q_{12}(4) = 0.40$. Which arm should be played next.
   How to answer

   • Recall UCB1 formula $Q(a) + \sqrt{2\ln t / n_a}$ with $t=12$
   • Compute exploration bonus for each of 4 arms
   • Select arm with highest UCB value as next arm
6. 7 Marks
   bWhat is meant by passive and active reinforcement learning and how do we compare the two?
   How to answer

   • Define passive RL: learn value of fixed policy
   • Define active RL: learn optimal policy by acting
   • Compare on exploration, objective and methods
7. 14 Marks
   Explain the following term with respect to Bandit algorithm:

   1. i)Median Elimination
   2. ii)Upper Confidence Bound (UCB) algorithm
   3. iii)Probably Approximately Correct (PAC)
   How to answer

   • Explain median elimination principle for best-arm identification
   • Explain UCB algorithm with confidence bound formula
   • Explain PAC framework with probably approximately correct guarantee
8. 7 Marks
   aExplain the Q-Function and Q-Learning Algorithm.
   How to answer

   • Define Q-function $Q(s,a)$ as expected return
   • Present Q-learning update formula
   • Outline algorithm steps with epsilon-greedy and off-policy nature
9. 7 Marks
   bDiscuss Maximum Likelihood and Least Square Error Hypothesis.
   How to answer

   • Define maximum likelihood hypothesis
   • Define least square error hypothesis
   • Discuss relation and use in learning models
10. 7 Marks
    aExplain Fitted-Q and Deep Q-Learning Problems.
    How to answer

    • Explain fitted-Q iteration with function approximation
    • Explain deep Q-learning with neural networks
    • Highlight problems: instability, overestimation, data correlation
11. 7 Marks
    bExplain any one advanced Q-learning algorithms.
    How to answer

    • Name one advanced variant e.g. Double DQN, Dueling DQN
    • Explain its modification over vanilla Q-learning
    • State benefits and application
12. 7 Marks
    aExplain learning policies by imitating optimal controllers.
    How to answer

    • Define imitation learning / behavioral cloning
    • Explain learning from optimal controller demonstrations
    • Discuss steps and limitations like distribution shift
13. 7 Marks
    bExplain inverse reinforcement learning.
    How to answer

    • Define inverse RL as inferring reward from behavior
    • Contrast with forward RL
    • Outline methods e.g. max-entropy IRL and applications
14. 14 Marks
    Explain the following.

    1. i)DQN and Policy Gradient
    2. ii)Hierarchical RL
    3. iii)POMDPs.
    How to answer

    • Explain DQN and policy gradient methods
    • Explain hierarchical RL with options/skills
    • Explain POMDPs with partial observability and belief states