CS-601 Machine Learning - May 2024 Question Paper

Under each question, How to answer lists the points an answer should cover, outlined by CampusPrep from the syllabus. Check the details against your textbook.

1. 7 Marks
   aList the main types of machine learning algorithms and provide examples of applications for each type?
   How to answer

   • Supervised Learning: linear regression (house price prediction), classification (email spam detection), decision trees (loan approval)
   • Unsupervised Learning: clustering (customer segmentation using K-means), dimensionality reduction (PCA for feature extraction)
   • Reinforcement Learning: game playing (AlphaGo), robotics control, autonomous vehicles
   • Semi-supervised Learning: text classification with limited labels
2. 7 Marks
   bDescribe common evaluation metrics used for regression models of machine learning?
   How to answer

   • MSE: average of squared differences between predicted and actual values; sensitive to outliers
   • MAE: average absolute difference; robust to outliers
   • RMSE: square root of MSE; in same units as target variable
   • R²: proportion of variance explained by model; ranges from 0 to 1
   • MAPE: average percentage error; useful for comparison across scales
3. 7 Marks
   aExplain the importance of data normalization in machine learning for improving model convergence, stability and performance?
   How to answer

   • Normalization brings features to similar scales (e.g., 0-1 or mean 0, std 1) preventing large-scale features from dominating
   • Improves gradient descent convergence by creating more spherical loss surface; allows larger learning rates
   • Prevents numerical overflow/underflow in computations
   • Ensures fair feature contribution in distance-based algorithms (KNN, K-means, SVM)
4. 7 Marks
   bDiscuss the advantages and limitations of sigmoid and ReLU activation functions in terms vanishing gradient problem and output range?
   How to answer

   • Sigmoid: output range [0,1], smooth, but gradient near 0 at extremes causing vanishing gradient in deep networks
   • ReLU: output range [0,∞), gradient = 1 for positive inputs avoiding vanishing gradient, but zero gradient for negative inputs (dying ReLU)
   • Sigmoid advantages: smooth differentiable, interpretable as probability; disadvantages: slow convergence, vanishing gradient
   • ReLU advantages: computationally efficient, faster convergence; disadvantages: non-differentiable at 0, dead neurons
5. 7 Marks
   aDefine gradient descent in the context of optimization algorithms used in machine learning?
   How to answer

   • Gradient descent is an iterative optimization algorithm that minimizes a cost/loss function
   • At each iteration, parameters are updated in direction opposite to gradient: θ = θ - α∇J(θ)
   • Learning rate α controls step size; small α: slow convergence; large α: may overshoot minimum
   • Three variants: batch (all data), stochastic (one sample), mini-batch (subset) gradient descent
6. 7 Marks
   bExplain how L1 regularization (Lasso) and L2 regularization (Ridge) work to penalize the magnitude model parameters differently?
   How to answer

   • L1 adds penalty λ∑|θ| to loss function; encourages some coefficients to exactly zero, leading to sparse models and automatic feature selection
   • L2 adds penalty λ∑θ² to loss function; shrinks all coefficients but none to exactly zero, retaining all features with reduced magnitudes
   • L1 (Lasso): useful when feature selection needed, handles high-dimensional data; L2 (Ridge): better when all features important, more stable
   • L1 uses absolute values (non-differentiable at 0); L2 uses squares (smooth, differentiable everywhere)
7. 7 Marks
   aList and explain the types of padding commonly used in Convolutional Neural Networks (CNNs)?
   How to answer

   • Valid (No) Padding: no zeros added; output size = (input - kernel_size + 1); useful for decreasing spatial dimensions
   • Same Padding: zeros added to preserve input dimensions; output size = input size; commonly used for maintaining spatial resolution
   • Full (or Zero) Padding: additional zeros for maximum coverage; output size = (input + kernel_size - 1); rarely used
   • Padding prevents edge information loss and controls output dimensions
8. 7 Marks
   bDescribe the types of transfer learning, including feature extraction and fine-tuning?
   How to answer

   • Feature Extraction: use pre-trained model weights (frozen) as fixed feature extractor; train only final classifier layer; fast, requires less data
   • Fine-tuning: use pre-trained weights as initialization; unfreeze and retrain all or some layers with small learning rate; more effective, requires more data
   • Domain Adaptation: transfer knowledge from source to target domain with different data distributions
   • Choose feature extraction for similar domains with limited data; fine-tuning for sufficient target data
9. 7 Marks
   aExplain why high-dimensional data can be challenging for machine learning algorithms?
   How to answer

   • Curse of Dimensionality: as dimensions increase, volume of space grows exponentially; data becomes sparse and isolated
   • Overfitting: more features than samples increases overfitting risk; model memorizes noise rather than learning patterns
   • Computational Cost: algorithms become slower; distance metrics become less meaningful in high-dimensional spaces
   • Solutions: dimensionality reduction (PCA, t-SNE), feature selection, regularization
10. 7 Marks
    bDefine Recurrent Neural Networks (RNNs). Explain the types of architecture of an RNN?
    How to answer

    • RNNs are neural networks with feedback connections processing sequential data; maintain hidden state carrying temporal information
    • Basic RNN: simple recurrent structure with hidden state updated at each timestep; suffers from vanishing/exploding gradients
    • LSTM (Long Short-Term Memory): gated architecture with memory cell and gates (input, forget, output) for long-term dependencies
    • GRU (Gated Recurrent Unit): simplified LSTM with reset and update gates; fewer parameters, faster training
11. 7 Marks
    aDescribe the significance of n-gram precision and brevity penalty in the BLEU score calculation?
    How to answer

    • n-gram Precision: measures proportion of n-grams (words/phrases) in generated text matching reference translation; higher values indicate better word choice
    • Brevity Penalty: penalizes translations shorter than reference; prevents unfair high scores from short outputs; ratio-based exponential penalty
    • BLEU combines multiple n-gram precisions (1-gram through 4-gram) with geometric mean and applies brevity penalty
    • Significance: balances precision (correctness) with length appropriateness; prevents gaming by short but high-precision outputs
12. 7 Marks
    bExplain the roles of the actor and critic networks in the Actor-Critic model and how they interact to learn from experiences?
    How to answer

    • Actor: outputs probability distribution over actions (policy); learns to improve action selection based on critic feedback
    • Critic: estimates value function (expected cumulative reward); evaluates quality of state/state-action pairs
    • Interaction: actor takes action; critic evaluates result using temporal difference error; error signal guides both networks
    • Advantages: combines policy gradient (actor) with value-based learning (critic) for stable, sample-efficient training
13. 7 Marks
    aDescribe the concept of support vectors and their role in defining the decision boundary in Support Vector Machines (SVM)?
    How to answer

    • Support Vectors: data points closest to decision boundary; only these points determine hyperplane position and orientation
    • SVM finds hyperplane maximizing margin (distance between classes) subject to constraints; other points can be outside margin
    • Decision Boundary: optimal hyperplane determined by support vectors; non-support vectors don't affect boundary
    • Key advantage: computationally efficient (depends only on support vectors, not full dataset); robust to outliers far from boundary
14. 7 Marks
    bExplain how machine learning algorithms are utilized in speech processing?
    How to answer

    • Speech Recognition: convert audio to text using acoustic models (HMM, DNNs) + language models; end-to-end models like seq2seq and attention-based systems
    • Speaker Identification: identify speaker from voice using features like MFCCs; classification using SVM, GMM, or deep neural networks
    • Speech Enhancement: noise reduction using autoencoders or RNNs to improve audio quality
    • Emotion/Sentiment Detection: classify speaker emotional state from prosody and acoustic features
15. 14 Marks
    Write short notes on any two:

    1. i))Convex optimization
    2. ii))Linearity vs Non linearity
    3. iii))Q-learning
    4. iv))Markov Decision Process (MDP)
    How to answer

    • Select any 2 of the 4 topics
    • Provide concise but comprehensive coverage
    • Include key definitions, principles, or examples
    • Demonstrate understanding of core concepts