Machine Learning (AL-405) - Unit 3 Short Notes

UNIT 3: MACHINE LEARNING - COMPREHENSIVE SHORT NOTES

Based on rigorous analysis of RGPV past papers (2022-2025). Focus on definitions, formulas, algorithms, and comparisons.

1. FOUNDATIONS & CORE CONCEPTS

Definition & Importance

• Formal Definition: Machine Learning (ML) is a subset of AI that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. It builds a model from training data to make predictions or decisions.

• Importance: Solves complex, data-rich problems (image recognition, NLP, recommendation systems), automates decision-making, discovers patterns in big data.

• Perspectives:

  • AI: Broad field of creating intelligent agents.

  • ML: Approach to AI via learning from data.

  • Deep Learning (DL): Subset of ML using deep neural networks.

• Major Limitations: Requires large amounts of data, computational cost, "black box" interpretability issues, susceptibility to biased data, overfitting.

• Basic Design Issues: Choice of model, representation of data, handling noise, evaluation strategy, balancing complexity vs. generalization.

Types of Machine Learning

Data & Preprocessing

• Necessity: Real-world data is noisy, incomplete, inconsistent. Preprocessing improves convergence, stability, and model performance.

• Normalization/Standardization:

  • Min-Max Scaling: $$\displaystyle X' = \frac{X - X_{min}}{X_{max} - X_{min}} $$ (scales to [0,1]).

  • Standardization (Z-score): $$\displaystyle X' = \frac{X - \mu}{\sigma} $$ (mean=0, std=1).

• Handling Missing Data: Removal, imputation (mean/median/mode), or using algorithms that support missing values.

• Categorical Encoding:

  • Label Encoding: Assigns integer to each category. Risk: Imposes ordinal relationship.

  • One-Hot Encoding: Creates binary column for each category. Impact: Increases dimensionality (curse of dimensionality).

• Data Augmentation: Artificially increase training data size by creating modified copies (e.g., image rotation, flipping, adding noise). Crucial for CV to prevent overfitting.

Statistical & Probabilistic Foundations

• Role of Probability: Quantifies uncertainty, forms basis for probabilistic models (Naïve Bayes, Bayesian Networks), provides framework for inference and decision-making.

• Bayes' Theorem: $$\displaystyle P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} $$

  • Use in Naïve Bayes: Calculates posterior probability $P(Class|Features)$ for classification assuming feature independence.

  • Fundamental Importance: Allows updating beliefs with new evidence; core of Bayesian learning.

• Bayesian Networks: Directed acyclic graph (DAG) representing conditional dependencies among random variables. Each node has a Conditional Probability Table (CPT). Inference computes posterior probabilities given evidence.

• Entropy & Information Gain (Decision Trees):

  • Entropy: $$\displaystyle H(S) = -\sum_{i=1}^{c} p_i \log_2 p_i $$ (measure of impurity/uncertainty in set S).

  • Information Gain: $$\displaystyle IG(S,A) = H(S) - \sum_{v \in Values(A)} \frac{|S_v|}{|S|} H(S_v) $$ (reduction in entropy after splitting on attribute A). Used in ID3/C4.5.

Hypothesis Space & Learning Theory

• Hypothesis Space ($\mathcal{H}$): Set of all possible models/ functions that can be learned by the algorithm (e.g., all linear functions).

• Inductive Bias: Assumptions made by the learning algorithm to generalize from training examples to unseen data (e.g., "similar inputs have similar outputs" in KNN). Necessary because infinite functions fit finite data.

• Overfitting vs. Underfitting:

  • Overfitting: Model learns noise in training data, performs poorly on new data. Symptoms: High training accuracy, low validation/test accuracy. Causes: Model too complex, insufficient data, noise. Solutions: Regularization (L1/L2), cross-validation, pruning (trees), dropout (NN), more data/augmentation.

  • Underfitting: Model is too simple to capture underlying pattern. Symptoms: Low training & validation accuracy. Causes: Model complexity too low. Solutions: More features, increase model complexity, reduce regularization.

• Bias-Variance Trade-off:

  • Bias: Error from erroneous assumptions (underfitting). High bias → underfitting.

  • Variance: Error from sensitivity to small fluctuations in training set (overfitting). High variance → overfitting.

  • Trade-off: Decreasing bias increases variance and vice-versa. Goal: Find optimal complexity minimizing total error.

• Model Selection: Process of choosing the best model from candidate models (different algorithms or hyperparameters). Uses validation sets or cross-validation to estimate generalization error.

[!TIP] Exam Focus: Be prepared to define overfitting/underfitting with examples (e.g., in CNN: deep network with few images → overfitting). Know the bias-variance trade-off diagram conceptually.

2. CORE SUPERVISED LEARNING ALGORITHMS

Linear Models

• Linear Regression: Models target as linear combination of features. $$\displaystyle y = \beta_0 + \beta_1 x_1 + ... + \beta_n x_n + \epsilon $$.

  • Assumptions: Linearity, independence, homoscedasticity, normality of errors, no multicollinearity.

  • Cost Function (MSE): $$\displaystyle J(\theta) = \frac{1}{2m} \sum_{i=1}^{m} (h_\theta(x^{(i)}) - y^{(i)})^2 $$.

  • Optimization: Gradient Descent: $$\displaystyle \theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j} J(\theta) $$.

• Logistic Regression: Used for classification. Applies logistic (sigmoid) function to linear output: $$\displaystyle h_\theta(x) = \frac{1}{1 + e^{-\theta^T x}} $$.

  • Cost Function (Log Loss): $$\displaystyle J(\theta) = -\frac{1}{m} \sum_{i=1}^{m} [y^{(i)} \log(h_\theta(x^{(i)})) + (1-y^{(i)}) \log(1-h_\theta(x^{(i)}))] $$.

• Locally Weighted Linear Regression: Non-parametric method. Fits linear model weighted by proximity to query point (uses kernel, e.g., Gaussian). $$\displaystyle w(i) = \exp(-\frac{(x^{(i)}-x)^2}{2\tau^2}) $$.

Support Vector Machines (SVM)

• Goal: Find the optimal hyperplane that maximizes the margin (distance between hyperplane and nearest data points of any class).

• Linear SVM: For linearly separable data.

  • Decision Boundary: $$\displaystyle w^T x + b = 0 $$.

  • Margin: $$\displaystyle \frac{2}{\|w\|} $$. Maximizing margin is equivalent to minimizing $$\displaystyle \frac{1}{2}\|w\|^2 $$ subject to $$\displaystyle y^{(i)}(w^T x^{(i)} + b) \geq 1 $$.

  • Support Vectors: Training points that lie exactly on the margin boundaries (i.e., satisfy $$\displaystyle y^{(i)}(w^T x^{(i)} + b) = 1 $$). They define the decision boundary.

• Kernel Trick: Maps input features into higher-dimensional space implicitly using a kernel function $$\displaystyle K(x_i, x_j) = \phi(x_i)^T \phi(x_j) $$, allowing linear separation in that space without explicit transformation.

  • Common Kernels: Linear, Polynomial ($$\displaystyle K(x_i,x_j) = (x_i^T x_j + c)^d $$), RBF/Gaussian ($$\displaystyle K(x_i,x_j) = \exp(-\gamma \|x_i - x_j\|^2) $$).

• Applications: Bioinformatics (protein classification), Image recognition, Text categorization.

• Advantages: Effective in high dimensions, robust to overfitting (margin maximization), versatile via kernels.

• Limitations: Choice of kernel/parameters critical, slow training on large datasets, no direct probability estimates (requires Platt scaling).

Instance-Based Learning: K-Nearest Neighbors (KNN)

• Algorithm:

  1. Store all training examples.

  2. For a new query point, compute distance (Euclidean, Manhattan, Minkowski) to all stored points.

  3. Identify k nearest neighbors.

  4. Classification: Majority vote among neighbors.

  5. Regression: Average of neighbors' target values.

• Lazy vs. Eager Learning: KNN is lazy (no explicit training, generalizes at query time). Most other algorithms are eager (build model during training).

• Key Issues: Choice of k (small → noise sensitivity, large → smoother boundary), distance metric, computational cost at prediction.

Decision Trees

• Algorithms: ID3 (uses Information Gain), C4.5 (uses Gain Ratio, handles continuous/discrete), CART (uses Gini Impurity, supports regression).

• Splitting Criteria:

  • Entropy/Information Gain: (ID3/C4.5). Prefer attributes with high IG.

  • Gini Impurity: $$\displaystyle Gini(S) = 1 - \sum_{i=1}^{c} p_i^2 $$. (CART). Prefer attributes that minimize weighted Gini of children.

• Construction: Recursive partitioning until stopping condition (pure node, max depth, min samples).

• Issues: Overfitting (pruning needed), handling continuous attributes (find optimal split point), missing values (fractional instance splitting), bias towards multi-valued attributes.

• Appropriate Problems: When features are mixture of types, when interpretability is key, when data has missing values.

Ensemble Methods

• Goal: Combine multiple weak learners to create a strong learner, improving accuracy and robustness.

• Bagging (Bootstrap Aggregating):

  • Create B bootstrap samples (sampling with replacement).

  • Train independent base learners (e.g., decision trees) on each sample.

  • Aggregate: Majority vote (classification) or average (regression).

  • Random Forest: Bagging + feature randomness (each split considers random subset of features). Reduces correlation between trees.

• Boosting:

  • Train learners sequentially, each new learner focuses on errors of previous ones.

  • AdaBoost: Increases weight of misclassified instances. Final prediction: weighted majority vote.

  • Gradient Boosting: Fits each new learner to the residuals (negative gradient of loss function) of previous ensemble. (e.g., XGBoost, LightGBM).

• Stacking: Train multiple base learners, then use their predictions as input features for a final meta-learner.

• Bagging vs. Boosting:

  | Feature | Bagging | Boosting | | :--- | :--- | :--- | | Goal | Reduce variance | Reduce bias | | Training | Parallel, independent | Sequential, dependent | | Weights | Equal weight for samples | Re-weight samples (focus on errors) | | Robustness | Robust to overfitting (if base learners weak) | Can overfit if too many rounds/noisy data |

[!TIP] Exam Focus: Know the difference between Bagging and Boosting clearly. Random Forest is a specific type of Bagging. Be ready to explain how Gradient Boosting uses residuals.

3. UNSUPERVISED LEARNING & DIMENSIONALITY REDUCTION

Clustering

• Goals: Group similar objects together, discover hidden structures, summarize data.

• Requirements: Scalability, ability to handle different attribute types, discovery of arbitrary-shaped clusters, minimal domain knowledge, robustness to noise/outliers.

• K-Means Clustering:

  • Algorithm:

    1. Initialize K centroids (randomly or k-means++).

    2. Repeat until convergence:

       • Assignment: Assign each point to nearest centroid (using Euclidean distance).

       • Update: Recalculate centroids as mean of assigned points.

  • Properties: Sensitive to initial centroids, assumes spherical clusters of similar size, requires K specified.

• Hierarchical Clustering:

  • AGNES (AGglomerative NESting): Bottom-up. Start with each point as cluster, merge closest pairs. Linkage criteria: Single (min distance), Complete (max distance), Average.

  • DIANA (DIvisive ANAlysis): Top-down. Start with one cluster, split recursively.

  • Dendrogram: Tree diagram showing cluster merges/splits.

• Density-Based: DBSCAN (Density-Based Spatial Clustering of Applications with Noise):

  • Core Point: Has ≥ min_samples points within eps radius.

  • Border Point: Within eps of core point but not core itself.

  • Noise Point: Neither core nor border.

  • Advantages: Finds arbitrary-shaped clusters, robust to outliers, no need to specify K.

• Expectation-Maximization (EM) for Gaussian Mixture Models (GMM):

  • E-step: Estimate "responsibilities" (probability that each point belongs to each cluster) given current parameters (means, variances, weights).

  • M-step: Re-estimate parameters (means, covariances, mixing coefficients) to maximize expected complete-data log-likelihood.

  • Handles: Soft clustering, elliptical clusters, different sizes.

• Adaptive Hierarchical Clustering: Dynamically determines number of clusters by analyzing density or stability in dendrogram cuts.

• Applications: Market segmentation, social network analysis, image segmentation, anomaly detection.

Dimensionality Reduction

• Need & Benefits: Curse of dimensionality (sparsity, distance concentration), visualization (2D/3D), improve efficiency (less storage/computation), enhance model performance by removing noise/redundant features.

• Feature Selection vs. Feature Extraction:

  • Selection: Choose subset of original features (e.g., filter methods, wrapper methods like backward elimination).

  • Extraction: Create new features from original ones (e.g., PCA, LDA).

• Principal Component Analysis (PCA):

  • Goal: Find orthogonal axes (principal components) that maximize variance in data.

  • Algorithm:

    1. Standardize data.

    2. Compute covariance matrix $\Sigma$.

    3. Compute eigenvectors and eigenvalues of $\Sigma$.

    4. Sort eigenvectors by decreasing eigenvalues.

    5. Choose top k eigenvectors as principal components.

    6. Transform data: $$\displaystyle Z = X W_k $$ (where $$\displaystyle W_k $$ is matrix of top k eigenvectors).

  • Key: Preserves maximum variance; first PC has largest variance.

• Linear Discriminant Analysis (LDA): Supervised method. Finds axes that maximize separation between classes (maximize between-class scatter / minimize within-class scatter).

• t-SNE (t-Distributed Stochastic Neighbor Embedding): Non-linear, primarily for visualization. Preserves local structure (neighborhoods) by minimizing KL divergence between point distributions in high and low-dim space.

• Partial Least Squares (PLS): Supervised. Finds components that maximize covariance between features and target variable.

• Backward Elimination (Feature Selection):

  1. Start with all features.

  2. Train model, evaluate performance (e.g., using p-values or validation error).

  3. Remove the least significant feature.

  4. Repeat until performance degrades significantly.

[!TIP] Exam Focus: Be able to derive/state the PCA steps and objective. Know the difference between PCA (unsupervised, variance) and LDA (supervised, class separation). Practice a simple K-means calculation (past papers frequently ask).

4. NEURAL NETWORKS & DEEP LEARNING

Artificial Neural Networks (ANN) Fundamentals

• Biological vs. Artificial Neuron: Biological neuron receives signals via dendrites, processes in soma, sends output via axon. Perceptron mimics this: weighted sum of inputs + bias → activation function → output.

• Multi-Layer Perceptron (MLP): Input layer → one or more hidden layers → output layer. Fully connected. Universal approximator.

• Activation Functions:

  | Function | Formula | Range | Pros | Cons | | :--- | :--- | :--- | :--- | :--- | | Sigmoid | $$\displaystyle \sigma(z) = \frac{1}{1+e^{-z}} $$ | (0,1) | Smooth, outputs probability | Vanishing gradient, not zero-centered, slow | | Tanh | $$\displaystyle \tanh(z) = \frac{e^z - e^{-z}}{e^z + e^{-z}} $$ | (-1,1) | Zero-centered, steeper than sigmoid | Vanishing gradient | | ReLU | $$\displaystyle f(z) = \max(0,z) $$ | [0, ∞) | Computationally cheap, mitigates vanishing gradient (for +ve) | Dying ReLU (neurons stuck at 0) | | Leaky ReLU | $$\displaystyle f(z) = \max(\alpha z, z) $$ | (-∞, ∞) | Fixes dying ReLU | Results may vary |

• Parallel Processing: Neural networks perform massive parallel computations (matrix multiplications) across layers and neurons, making them efficient on GPUs.

Training Neural Networks

• Gradient Descent (GD): Iteratively adjust weights to minimize loss.

  • Batch GD: Use entire dataset per update (stable, slow).

  • Stochastic GD (SGD): Use one sample per update (noisy, fast, can escape minima).

  • Mini-batch GD: Use small random subset (compromise, standard).

• Backpropagation Algorithm:

  1. Forward Pass: Compute output and loss for a mini-batch.

  2. Backward Pass (Backprop):

     • Compute gradient of loss w.r.t. output layer activations.

     • Apply chain rule to propagate gradients backward through layers.

     • Compute $$\displaystyle \frac{\partial L}{\partial W^{(l)}} = \delta^{(l)} (a^{(l-1)})^T $$, where $$\displaystyle \delta^{(l)} $$ is error term for layer l.

  3. Update Weights: $$\displaystyle W^{(l)} := W^{(l)} - \eta \frac{\partial L}{\partial W^{(l)}} $$ (using chosen optimizer).

• Loss Functions:

  • Regression: Mean Squared Error (MSE): $$\displaystyle L = \frac{1}{N} \sum (y - \hat{y})^2 $$.

  • Classification: Cross-Entropy (Log Loss): $$\displaystyle L = -\frac{1}{N} \sum [y \log(\hat{y}) + (1-y)\log(1-\hat{y})] $$ (binary).

  • Huber: Combines MSE and MAE; less sensitive to outliers.

• Optimizers:

  | Optimizer | Key Idea | Benefit | | :--- | :--- | :--- | | SGD | Basic GD with mini-batch. | Simple, can find minima. | | Momentum | Adds fraction of previous update: $$\displaystyle v_t = \gamma v_{t-1} + \eta \nabla J(\theta) $$. | Accelerates in consistent direction, dampens oscillations. | | RMSprop | Adapts learning rate per parameter using moving average of squared gradients. | Handles non-stationary objectives. | | Adam | Combines Momentum and RMSprop (adaptive moment estimation). | Default choice, fast convergence, low memory. |

Convolutional Neural Networks (CNN)

• Architecture: [Input] → [Convolution → Activation] × N → [Pooling] × M → [Flatten] → [Fully Connected] → [Output]

• Convolution Operation:

  • Filter/Kernel: Small matrix (e.g., 3x3, 5x5) that slides over input feature map.

  • Feature Map: Output of applying a filter. Each filter detects a specific feature (edge, texture).

  • Stride: Step size of filter movement. Larger stride → smaller output.

  • Output Size: $$\displaystyle O = \frac{W - K + 2P}{S} + 1 $$, where $W$=input width, $K$=kernel size, $P$=padding, $S$=stride.

• 1x1 Convolution:

  • Purpose: Change depth (number of filters/channels) without affecting spatial dimensions. Acts as a bottleneck or network-in-network for feature reduction/combination, computational efficiency.

  • Example: In Inception modules, used before expensive 3x3/5x5 convs to reduce depth.

• Padding:

  | Type | Formula | Effect | Use Case | | :--- | :--- | :--- | :--- | | Valid | $$\displaystyle P=0 $$ | Output shrinks. | When border info less important. | | Same | $P$ chosen so $$\displaystyle O=W $$ | Preserves spatial size. | Common in early layers to retain info. | | Full | $$\displaystyle P=K-1 $$ | Output larger than input. | Rare, for specific signal processing. |

• Pooling/Sub-sampling (Downscaling): Reduces spatial dimensions (width/height), increases receptive field of subsequent layers, provides translation invariance, reduces parameters/computation.

  • Max Pooling: Takes max value in window. Most common.

  • Average Pooling: Takes average. Used in some architectures (e.g., Inception).

• Flattening: Converts pooled feature maps into a single 1D vector for input to fully connected layers.

• Why Downscale & Increase Filters? As network deepens, spatial resolution decreases (pooling/stride), but number of filters increases to capture more complex, abstract features. Maintains roughly constant computational load per layer.

• Identifying Overfitting/Underfitting in CNNs:

  • Overfitting: Training accuracy ↑, validation accuracy ↓/plateaus. Solutions: Data augmentation, Dropout (randomly deactivate neurons), L2 regularization, Batch Normalization, reduce model capacity.

  • Underfitting: Both accuracies low. Solutions: Increase model depth/width, reduce regularization, more training epochs.

• Advanced CNN Architectures (Short Notes):

  • Inception Module (GoogLeNet): Parallel convolutions (1x1, 3x3, 5x5) + pooling, concatenate outputs. 1x1 convs used as bottlenecks to reduce depth before expensive filters, improving efficiency. Captures multi-scale features.

  • ImageNet Competition: Catalyst for deep learning revolution.

    • AlexNet (2012): First deep CNN (8 layers), used ReLU, Dropout, GPU training. Proved deep CNNs work.

    • VGG (2014): Very deep (16-19 layers), uniform 3x3 convs. Showed depth matters.

    • ResNet (2015): Introduced skip connections (residual blocks) to train very deep networks (100+ layers) by mitigating vanishing gradient. Won with 152 layers.

• CNN Implementation (TensorFlow/Keras Overview):

  
  model = Sequential([
  
      Conv2D(filters=32, kernel_size=(3,3), activation='relu', input_shape=(img_h, img_w, channels)),
  
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(64, (3,3), activation='relu'),
  
      MaxPooling2D((2,2)),
  
      Flatten(),
  
      Dense(128, activation='relu'),
  
      Dropout(0.5),
  
      Dense(num_classes, activation='softmax')
  
  ])
  
  model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
  
  

Recurrent Neural Networks (RNN) & LSTM

• RNN Architecture: Has hidden state $$\displaystyle h_t $$ that propagates through time steps. $$\displaystyle h_t = f(W_{xh} x_t + W_{hh} h_{t-1} + b_h) $$. Output $$\displaystyle y_t = g(W_{hy} h_t + b_y) $$.

  • Types: One-to-one (vanilla), One-to-many (e.g., captioning), Many-to-one (e.g., sentiment), Many-to-many (e.g., translation).

• Vanishing/Exploding Gradient Problem: In vanilla RNNs, gradients multiplied by $$\displaystyle W_{hh} $$ repeatedly during backprop through time. If eigenvalues of $$\displaystyle W_{hh} < 1 $$ → vanish; if >1 → explode. Makes learning long-term dependencies difficult.

• LSTM (Long Short-Term Memory): Designed to address vanishing gradient.

  • Cell State ($$\displaystyle C_t $$): "Highway" carrying information unchanged across time steps.

  • Gates (Sigmoid + Tanh):

    • Forget Gate: Decides what to drop from cell state. $$\displaystyle f_t = \sigma(W_f \cdot [h_{t-1}, x_t] + b_f) $$

    • Input Gate: Decides what new info to store. $$\displaystyle i_t = \sigma(W_i \cdot [h_{t-1}, x_t] + b_i) $$, $$\displaystyle \tilde{C}_t = \tanh(W_C \cdot [h_{t-1}, x_t] + b_C) $$

    • Update Cell State: $$\displaystyle C_t = f_t \odot C_{t-1} + i_t \odot \tilde{C}_t $$

    • Output Gate: Decides what to output based on cell state. $$\displaystyle o_t = \sigma(W_o \cdot [h_{t-1}, x_t] + b_o) $$, $$\displaystyle h_t = o_t \odot \tanh(C_t) $$

• GRU (Gated Recurrent Unit): Simplified LSTM. Merges forget/input gates into update gate $$\displaystyle z_t $$, and has reset gate $$\displaystyle r_t $$. No separate cell state; hidden state serves both roles. Fewer parameters, often similar performance.

• Differences:

  | Model | Key Feature | Handles Long-Term Dependencies? | | :--- | :--- | :--- | | Feed-Forward | No recurrence, fixed-size input/output. | No | | Vanilla RNN | Simple recurrence, shared weights. | Poorly (vanishing gradient) | | LSTM | Cell state + gates. | Yes (designed for it) | | GRU | Update/reset gates, no separate cell state. | Yes (similar to LSTM) |

• Role in NLP Generation (e.g., ChatGPT context): LSTMs/GRUs were foundational for sequence modeling (language modeling, machine translation) before Transformers. They process tokens sequentially, maintaining context via hidden state. However, they struggle with very long sequences due to inherent sequential processing and limited memory, which Transformers (with self-attention) overcame.

Autoencoders

• Architecture: Encoder (f) → Bottleneck (latent code z) → Decoder (g). Trained to reconstruct input: $$\displaystyle \hat{x} = g(f(x)) $$.

• Use in Unsupervised Learning:

  • Dimensionality Reduction: Latent space z has lower dimension than input x.

  • Denoising: Train with corrupted input, reconstruct clean output (Denoising Autoencoder).

  • Anomaly Detection: Train on normal data; high reconstruction error on anomalous input.

  • Feature Learning: Latent representations can be used as features for other tasks.

Regularization Techniques

• L1 Regularization (Lasso): Adds penalty $$\displaystyle \lambda \sum |w_j| $$ to loss. Drives some weights exactly to zero, performing feature selection. Produces sparse models.

• L2 Regularization (Ridge): Adds penalty $$\displaystyle \lambda \sum w_j^2 $$ to loss. Shrinks weights towards zero but rarely to zero. Handles multicollinearity, improves generalization.

• How They Prevent Overfitting: Constrain weight magnitude, reduce model complexity, discourage learning noise.

• Dropout: During training, randomly "drop" (set to zero) a fraction p of neurons in a layer. Forces network to learn redundant representations, reduces co-adaptation. Not used at test time (use all neurons, scale activations).

• Batch Normalization:

  • Purpose: Normalize layer inputs (activations from previous layer) to have zero mean and unit variance per mini-batch.

  • Operation: For each feature: $$\displaystyle \hat{x}^{(k)} = \frac{x^{(k)} - \mu_B}{\sqrt{\sigma_B^2 + \epsilon}} $$, then scale/shift: $$\displaystyle y^{(k)} = \gamma^{(k)} \hat{x}^{(k)} + \beta^{(k)} $$.

  • Benefits: Reduces internal covariate shift, allows higher learning rates, acts as regularizer (noise from batch statistics), improves gradient flow.

[!TIP] Exam Focus: Be able to draw and explain LSTM cell with gates. Know the exact formulas for L1/L2 penalties. Explain why BatchNorm helps (internal covariate shift, smoother loss landscape). Contrast 1x1 conv purpose vs. standard convolution.

5. REINFORCEMENT LEARNING (RL)

Fundamentals

• Difference from Supervised/Unsupervised:

  • Supervised: Learn from labeled (input, target) pairs.

  • Unsupervised: Find structure in unlabeled data.

  • RL: Learn by interacting with an environment through trial-and-error, receiving scalar rewards. No explicit dataset; data is generated sequentially.

• Key Components:

  • Agent: Learner/decision maker.

  • Environment: World agent interacts with.

  • State (s): Representation of environment at time t.

  • Action (a): Move agent can make.

  • Reward (r): Immediate scalar feedback from environment.

  • Policy ($\pi$): Strategy mapping states to actions (deterministic: $$\displaystyle a = \pi(s) $$, stochastic: $\pi(a|s)$).

• Markov Decision Process (MDP): Formal framework for RL.

  • Components: $(S, A, T, R, \gamma)$

    • $S$: Set of states.

    • $A$: Set of actions.

    • $T(s'|s,a)$: Transition probability to state $s'$ from $s$ taking $a$.

    • $R(s,a,s')$: Reward received.

    • $\gamma \in [0,1]$: Discount factor (future reward importance).

  • Markov Property: Future state depends only on current state and action, not history.

Core Algorithms

• Q-Learning (Off-policy):

  • Learns action-value function $Q(s,a)$: expected cumulative discounted reward taking a in s then following optimal policy.

  • Update Rule: $$\displaystyle Q(s,a) \leftarrow Q(s,a) + \alpha [r + \gamma \max_{a'} Q(s',a') - Q(s,a)] $$

  • Exploration vs. Exploitation: Use $\epsilon$-greedy: with prob $\epsilon$ choose random action (explore), else choose $$\displaystyle \arg\max_a Q(s,a) $$ (exploit).

• SARSA (On-policy):

  • Learns $Q(s,a)$ for the policy being followed (often $\epsilon$-greedy).

  • Update Rule: $$\displaystyle Q(s,a) \leftarrow Q(s,a) + \alpha [r + \gamma Q(s',a') - Q(s,a)] $$, where $a'$ is action actually taken in $s'$.

  • Difference from Q-learning: Q-learning uses $$\displaystyle \max_{a'} Q(s',a') $$ (optimistic, off-policy), SARSA uses next action $a'$ from current policy (on-policy). SARSA is more conservative, considers exploration.

• Value Iteration vs. Policy Iteration:

  | Aspect | Value Iteration | Policy Iteration | | :--- | :--- | :--- | | Idea | Directly compute optimal value function $$\displaystyle V^* $$ via Bellman optimality backup. | Alternate: Policy Evaluation (compute $$\displaystyle V^\pi $$) → Policy Improvement (greedy w.r.t $$\displaystyle V^\pi $$). | | Convergence | Usually faster per iteration, but each iteration requires full sweeps. | Often fewer iterations, but each policy evaluation can be slow. | | Guarantee | Converges to $$\displaystyle V^* $$. | Converges to optimal policy. |

Advanced Actor-Critic Models

• Concept: Combine value-based (critic) and policy-based (actor) methods.

  • Actor: Policy function $\pi(a|s)$ (neural network). Outputs action probabilities.

  • Critic: Value function $V(s)$ or $Q(s,a)$ (neural network). Evaluates state/action value.

• Interaction:

  1. Actor selects action based on current policy.

  2. Environment returns reward and next state.

  3. Critic evaluates the (s,a) pair (or new state) to compute advantage $$\displaystyle A(s,a) = Q(s,a) - V(s) $$ (how much better was this action than average?).

  4. Actor Update: Use critic's advantage as a low-variance gradient signal to update policy (increase prob of good actions).

  5. Critic Update: Update value function towards observed returns (TD error: $$\displaystyle \delta = r + \gamma V(s') - V(s) $$).

• Examples:

  • A2C (Advantage Actor-Critic): Synchronous, multiple parallel actors.

  • A3C (Asynchronous A2C): Multiple actors in parallel threads, asynchronous updates (faster, less stable).

  • DDPG (Deep Deterministic Policy Gradient): For continuous action spaces. Actor outputs deterministic action, critic evaluates it.

  • PPO (Proximal Policy Optimization): Uses clipped objective to limit policy update size, stable, popular.

• Rise: Actor-critic methods balance bias/variance better than pure policy gradient (high variance) or pure Q-learning (overestimation bias), leading to more stable and sample-efficient learning in complex environments.

Practical Aspects

• Popular RL Frameworks:

  • OpenAI Gym: Standard API for environments (Atari, robotics, classic control).

  • Stable-Baselines3: Reliable implementations of common algorithms (PPO, SAC, etc.) based on PyTorch.

  • TensorFlow Agents (TF-Agents): Flexible library for RL with TensorFlow.

• One-Shot Learning:

  • Definition: Learning a new concept or task from very few examples (often one or a handful), unlike traditional supervised learning requiring hundreds/thousands.

  • Difference: Traditional SL learns decision boundaries from many examples; one-shot must leverage prior knowledge (from similar tasks) or learn representations that generalize extremely well.

  • Beneficial Applications: Rare species identification, new manufacturing defect detection, personalized medicine with limited patient data, few-shot image classification.

[!TIP] Exam Focus: Memorize Q-learning and SARSA update rules. Be able to contrast on-policy vs. off-policy. Draw simple Actor-Critic diagram showing interaction. Define one-shot learning clearly.

6. ADVANCED TOPICS & EMERGING TRENDS

Transfer Learning

• Purpose: Leverage knowledge (features, weights) from a pre-trained model (on large dataset like ImageNet) for a new, related task with limited data.

• Types:

  • Feature Extraction: Use pre-trained model as fixed feature extractor. Freeze all layers, replace and train only the final classifier/regressor head.

  • Fine-Tuning: Unfreeze some (or all) of the pre-trained layers and continue training with new data (often with lower learning rate). Allows adaptation of higher-level features.

• Benefits: Faster convergence, better performance with small data, reduces need for massive datasets/computation.

Generative Models & AI

• Generative AI: Models that generate new data samples (text, images, audio) resembling training data.

• Generative Adversarial Networks (GANs):

  • Generator (G): Takes random noise, generates fake samples.

  • Discriminator (D): Classifies real vs. fake samples.

  • Training: Adversarial game. G tries to fool D, D tries to distinguish. Objective: $$\displaystyle \min_G \max_D V(D,G) = \mathbb{E}_{x\sim p_{data}}[\log D(x)] + \mathbb{E}_{z\sim p_z}[\log(1-D(G(z)))] $$.

• Variational Autoencoders (VAEs):

  • Encoder: Outputs parameters ($\mu, \sigma$) of latent distribution (usually Gaussian).

  • Reparameterization Trick: Sample $$\displaystyle z = \mu + \sigma \odot \epsilon $$, $\epsilon \sim \mathcal{N}(0,I)$ to allow backprop.

  • Decoder: Reconstructs input from sampled z.

  • Loss: Reconstruction loss + KL divergence between learned distribution and prior (usually $\mathcal{N}(0,I)$). Encourages smooth latent space.

Self-Supervised Learning

• Paradigm: Create pretext tasks from unlabeled data, where the task itself is the supervision signal.

• Common Pretext Tasks:

  • Image: Predict rotation angle, solve jigsaw puzzles, colorization, contrastive learning (SimCLR: bring augmented views of same image closer, push different images apart).

  • Text: Masked Language Modeling (BERT: predict masked words), Next Sentence Prediction.

• Benefit: Learns powerful, transferable representations without human annotation, reducing labeling cost.

Convex Optimization (Short Note)

• Definition: Optimization problem where objective function is convex (bowl-shaped) and constraint set is convex. Any local minimum is global minimum.

• Importance in ML: Many ML problems (linear regression with L2, SVM with linear kernel, logistic regression) have convex loss functions, guaranteeing global optimum via gradient-based methods.

• Non-convex: Deep neural networks have non-convex loss surfaces, prone to local minima/saddle points, but empirical success despite this.

Linearity vs. Non-linearity (Short Note)

• Linearity: Model is linear in parameters (e.g., $$\displaystyle y = w_1 x_1 + w_2 x_2 + b $$). Decision boundary is straight line/hyperplane. Simple, interpretable, but limited expressiveness.

• Non-linearity: Model captures complex relationships (e.g., neural networks with activation functions, SVM with RBF kernel). Can model any continuous function (universal approximation). More powerful but risk of overfitting, harder to interpret.

• Impact on Gradient Descent: Linear models have convex loss → GD converges to global optimum. Non-linear models (deep NNs) have non-convex loss → GD finds local minima/saddle points; careful initialization, architecture, and optimization needed.

[!TIP] Exam Focus: Distinguish transfer learning types clearly. For GANs, know generator/discriminator roles. For self-supervised, give one example pretext task. Define convex vs. non-convex in ML context.

7. APPLICATIONS & DOMAIN-SPECIFIC ML

Natural Language Processing (NLP)

• Pipeline: Text → Tokenization → Cleaning (lowercase, remove stopwords) → Stemming/Lemmatization → Vectorization (Bag-of-Words, TF-IDF, Word Embeddings like Word2Vec, GloVe) → Model (RNN/LSTM/Transformer).

• Applications:

  • Machine Translation: Sequence-to-sequence models (Seq2Seq with attention, Transformers).

  • Sentiment Analysis: Classify text polarity (positive/negative).

  • ChatGPT-like Generation: Autoregressive language models (GPT series) predict next token.

• BLEU Score (for translation/generation):

  • n-gram Precision: Fraction of candidate n-grams that appear in reference(s). Modified precision clips counts to max in any reference.

  • Brevity Penalty (BP): Penalizes short candidates. $$\displaystyle BP = 1 $$ if $$\displaystyle c > r $$, else $$\displaystyle e^{(1-r/c)} $$ (where c=candidate length, r=reference length).

  • Formula: $$\displaystyle BLEU = BP \cdot \exp(\sum_{n=1}^{N} w_n \log p_n) $$, typically $$\displaystyle N=4 $$, $$\displaystyle w_n=1/4 $$.

Computer Vision (CV)

• Applications:

  • Image Classification: Assign label to entire image (CNNs: AlexNet, VGG, ResNet).

  • Object Detection: Locate and classify objects (R-CNN, YOLO, SSD).

  • Semantic Segmentation: Classify each pixel (U-Net, FCN).

• Role of CNNs: Hierarchical feature learning (edges → textures → object parts). Translation invariance via pooling.

• ImageNet Impact: Large-scale dataset (14M images, 22K categories). Annual competition (ILSVRC) drove breakthroughs (AlexNet 2012 → deep learning revolution).

Speech Processing

• Speech-to-Text (ASR):

  • Frontend: Pre-emphasis, framing, windowing, MFCC/Filterbank extraction.

  • Model: Hybrid (HMM + DNN) or End-to-End (CTC with RNN/Transformer, Seq2Seq).

  • Output: Text transcript.

• Speaker Identification/Verification:

  • Task: Who is speaking? (Identification) or Is it person X? (Verification).

  • Features: MFCC, i-vectors, x-vectors.

  • Model: GMM-UBM, DNN embeddings + cosine scoring.

• Speech Synthesis (TTS):

  • Pipeline: Text → Linguistic features → Acoustic features → Waveform.

  • Models: Concatenative, Parametric (HTS), Neural (Tacotron, WaveNet, FastSpeech).

  • Output: Natural-sounding speech.

• ML Utilization: Feature extraction (hand-crafted or learned), sequence modeling (RNN/LSTM/Transformer for temporal dependencies), classification/regression for frames, generative models for waveform.

[!TIP] Exam Focus: Know BLEU components (n-gram precision, BP). List CV applications with CNN examples. For speech, differentiate ASR, speaker ID, TTS tasks and typical ML models used.

8. MODEL EVALUATION, VALIDATION & EXPERIMENTATION

Performance Metrics

• Regression:

  • MAE: $$\displaystyle \frac{1}{n}\sum |y_i - \hat{y}_i| $$ (robust to outliers).

  • MSE: $$\displaystyle \frac{1}{n}\sum (y_i - \hat{y}_i)^2 $$ (punishes large errors).

  • RMSE: $\sqrt{MSE}$ (same units as target).

  • R² (Coefficient of Determination): $$\displaystyle 1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2} $$. Proportion of variance explained.

• Classification:

  • Confusion Matrix:

    | | Predicted + | Predicted - | | :--- | :--- | :--- | | Actual + | TP | FN | | Actual - | FP | TN |

  • Derived Metrics:

    • Accuracy: $$\displaystyle \frac{TP+TN}{Total} $$ (misleading if imbalanced).

    • Precision: $$\displaystyle \frac{TP}{TP+FP} $$ (of predicted positives, how many correct?).

    • Recall (Sensitivity): $$\displaystyle \frac{TP}{TP+FN} $$ (of actual positives, how many found?).

    • F1-Score: $$\displaystyle 2 \cdot \frac{Precision \cdot Recall}{Precision + Recall} $$ (harmonic mean).

    • Specificity: $$\displaystyle \frac{TN}{TN+FP} $$.

    • ROC-AUC: Area under ROC curve (TPR vs FPR). Threshold-invariant measure of separability.

Validation Techniques

• Train/Test Split: Simple split (e.g., 70/30). Risk: high variance in estimate if data small.

• Cross-Validation (k-fold):

  • Split data into k equal folds.

  • Train on k-1 folds, validate on held-out fold. Repeat k times.

  • Average validation metrics. Stratified k-fold: Maintains class distribution in each fold (for classification).

  • Purpose: More robust performance estimation, uses all data for training/validation, helps detect overfitting.

• Hold-out Validation Set: Separate set not used in training or hyperparameter tuning (final test set).

Experimental Design

1. Define Problem & Metrics: What to predict? Which metric(s) matter?

2. Data Collection & Preprocessing: Gather, clean, split (train/val/test).

3. Model Selection: Choose candidate algorithms.

4. Hyperparameter Tuning: Use validation set/Cross-validation. Methods:

   • Grid Search: Exhaustive over parameter grid.

   • Random Search: Sample random combinations; often more efficient.

5. Training & Evaluation: Train on train set, tune on val set, evaluate final model on held-out test set.

6. Analysis & Interpretation: Compare metrics, analyze errors, check assumptions.

Resampling Methods

• Bootstrap: Sample n observations with replacement from original data of size n. Creates many "bootstrap samples". Used to estimate sampling distribution of statistic (e.g., confidence intervals).

• Cross-Validation: As above, used for model assessment and hyperparameter tuning.

[!TIP] Exam Focus: Memorize formulas for Precision, Recall, F1, R². Explain why cross-validation is better than single train/test split. Differentiate validation set from test set. Contrast grid vs. random search.