Machine Learning (AL-405) - Unit 4 Short Notes

I. FOUNDATIONS & OVERARCHING CONCEPTS

Definition & Importance

Machine Learning (ML) is a subset of AI where algorithms learn patterns from data to make predictions or decisions without explicit programming.
Core paradigm: Data → Model → Prediction.
Importance: Solves complex real-world problems (healthcare diagnostics, financial fraud detection, NLP chatbots, CV autonomous vehicles).
Perspectives:

• Computational: Algorithm design, scalability.

• Statistical: Generalization, uncertainty.

• Engineering: System integration, deployment.

Types of Machine Learning

Core Issues & Challenges

• Overfitting: Model memorizes training data (high variance). Solutions: Regularization, more data, dropout.

• Underfitting: Model too simple (high bias). Solutions: Increase complexity, feature engineering.

• Bias-Variance Tradeoff: Increasing model complexity reduces bias but increases variance.

• Curse of Dimensionality: High-dim data sparse, distance metrics lose meaning. Solution: Dimensionality reduction.

• Data Quality: Missing values, noise, class imbalance (use SMOTE, class weights).

Hypothesis Space & Inductive Bias

• Hypothesis Space: Set of all possible models/algorithms considered (e.g., all linear functions).

• Inductive Bias: Assumptions to guide learning (e.g., Occam’s razor: prefer simpler models).

• No Free Lunch Theorem: No single algorithm works best for all problems; performance depends on data distribution.

II. DATA PREPROCESSING & FEATURE ENGINEERING

Data Preprocessing

• Why needed? Ensures convergence, stability, and performance of ML models.

• Normalization/Standardization:

  • Min-Max: $$\displaystyle x' = \frac{x - x_{\min}}{x_{\max} - x_{\min}} $$ → [0,1]

  • Z-score: $$\displaystyle x' = \frac{x - \mu}{\sigma} $$ → mean 0, std 1.

• Missing Data: Imputation (mean/median/mode) or deletion (if few missing).

• Feature Encoding:

  • One-Hot Encoding: Creates binary columns for each category → increases dimensionality.

  • Label Encoding: Assigns integer to each category → preserves dimensionality but implies ordinality (use only for tree-based models).

Dimensionality Reduction

• Purpose: Reduce computational cost, avoid overfitting, visualize data.

• Principal Component Analysis (PCA):

  1. Standardize data.

  2. Compute covariance matrix $\Sigma$.

  3. Compute eigenvectors/eigenvalues of $\Sigma$.

  4. Select top $k$ eigenvectors (principal components) preserving desired variance (e.g., 95%).

  5. Project data: $$\displaystyle X_{\text{reduced}} = X W_k $$, where $$\displaystyle W_k $$ = top $k$ eigenvectors.

  \boxed{\text{Variance preserved} = \frac{\sum_{i=1}^k \lambda_i}{\sum_{i=1}^n \lambda_i}}

• Feature Selection vs. Extraction:

  • Selection: Choose subset of original features (e.g., backward elimination, forward selection).

  • Extraction: Create new features (e.g., PCA, PLS).

• Partial Least Squares (PLS): Supervised dimensionality reduction; maximizes covariance between features and target.

Data Augmentation

• Artificially increase dataset size (especially for images): rotation, flipping, cropping, color jittering. Improves generalization.

III. EVALUATION METRICS & MODEL SELECTION

Regression Metrics

• MSE: $$\displaystyle \text{MSE} = \frac{1}{n}\sum_{i=1}^n (y_i - \hat{y}_i)^2 $$

• RMSE: $\sqrt{\text{MSE}}$

• MAE: $$\displaystyle \frac{1}{n}\sum_{i=1}^n |y_i - \hat{y}_i| $$

• R²: $$\displaystyle 1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2} $$ (proportion of variance explained)

Classification Metrics (from Confusion Matrix)

Clustering Evaluation

• Internal: Silhouette Score ($$\displaystyle \frac{b-a}{\max(a,b)} $$, where $a$ = intra-cluster distance, $b$ = nearest-cluster distance).

• External: Adjusted Rand Index (ARI) – compares with ground truth labels.

Model Selection & Validation

• Train/Validation/Test Split: Typical 60/20/20 or 70/15/15.

• k-fold Cross-Validation: Split data into $k$ folds; train on $k-1$, validate on 1; repeat $k$ times.

  \boxed{\text{CV Score} = \frac{1}{k}\sum_{i=1}^k \text{Score}_i}

  Stratified k-fold: Preserves class distribution in each fold (for imbalanced data).

• Bootstrap: Resample with replacement; estimate performance stability.

NLP-Specific Metrics

• BLEU Score: For machine translation.

  \boxed{\text{BLEU} = \text{BP} \cdot \exp\left(\sum_{n=1}^N w_n \log p_n\right)}

  where $$\displaystyle p_n $$ = n-gram precision, BP = brevity penalty (penalize short translations).

IV. OPTIMIZATION & REGULARIZATION

Gradient Descent Family

Optimizers

• Momentum: $$\displaystyle v_t = \gamma v_{t-1} + \eta \nabla J(\theta) $$; $$\displaystyle \theta \leftarrow \theta - v_t $$. Accelerates convergence.

• RMSProp: Adapts learning rate per parameter: $$\displaystyle s_t = \beta s_{t-1} + (1-\beta) (\nabla J(\theta))^2 $$; $$\displaystyle \theta \leftarrow \theta - \frac{\eta}{\sqrt{s_t} + \epsilon} \nabla J(\theta) $$.

• Adam (Adaptive Moment Estimation): Combines Momentum & RMSProp.

  \boxed{\begin{aligned} m_t &= \beta_1 m_{t-1} + (1-\beta_1) g_t \ v_t &= \beta_2 v_{t-1} + (1-\beta_2) g_t^2 \ \hat{m}_t &= \frac{m_t}{1-\beta_1^t},\ \hat{v}t = \frac{v_t}{1-\beta_2^t} \ \theta_t &= \theta{t-1} - \eta \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon} \end{aligned}}

  Default: $$\displaystyle \beta_1=0.9 $$, $$\displaystyle \beta_2=0.999 $$, $$\displaystyle \eta=0.001 $$. Robust, widely used.

Loss Functions

• MSE: Regression.

• Cross-Entropy: Classification (binary: $$\displaystyle -\frac{1}{n}\sum y\log\hat{y} + (1-y)\log(1-\hat{y}) $$; multi-class: $$\displaystyle -\frac{1}{n}\sum_{i=1}^n \sum_{c=1}^C y_{ic}\log\hat{y}_{ic} $$).

• Hinge Loss: SVM: $\max(0, 1 - y \cdot f(x))$.

• Huber Loss: Robust to outliers; quadratic for small errors, linear for large.

Regularization Techniques

[!TIP] Common Pitfall: L1 produces sparse models; L2 does not. Use L1 for feature selection, L2 for generalization.

Convex Optimization

• If loss function is convex, gradient descent converges to global optimum (no local minima). Many ML losses (e.g., linear regression MSE) are convex; neural networks are non-convex.

V. NEURAL NETWORKS & DEEP LEARNING ARCHITECTURES

A. Fundamentals

Artificial Neuron & Perceptron

• Structure: Inputs $$\displaystyle x_i $$, weights $$\displaystyle w_i $$, bias $b$, weighted sum $$\displaystyle z = \sum w_i x_i + b $$, activation $$\displaystyle a = f(z) $$.

• Perceptron Learning Algorithm (for linearly separable data):

  Initialize weights randomly. For each sample:

  $$\displaystyle \text{if } y \neq \hat{y}: w \leftarrow w + \eta (y - \hat{y}) x $$, $$\displaystyle b \leftarrow b + \eta (y - \hat{y}) $$.

  Repeat until convergence.

Multilayer Perceptron (MLP)

• Architecture: Input layer → one or more hidden layers (non-linear activations) → output layer.

• Universal Approximation Theorem: MLP with one hidden layer (sufficient neurons) can approximate any continuous function.

Activation Functions

[!TIP] Exam Tip: ReLU is default for hidden layers; Softmax for multi-class output; Sigmoid/Tanh for RNNs/gating.

Backpropagation Algorithm

1. Forward pass: Compute loss $L$ for given input.

2. Backward pass: Compute gradients $$\displaystyle \frac{\partial L}{\partial w} $$ using chain rule.

3. Update weights: $$\displaystyle w \leftarrow w - \eta \frac{\partial L}{\partial w} $$.
   Key idea: Error propagates backward; each layer’s gradient depends on downstream gradients.
   Example (2-layer network with sigmoid):

Let $$\displaystyle a^1 = \sigma(z^1) $$, $$\displaystyle z^1 = W^1 x + b^1 $$; $$\displaystyle a^2 = \sigma(z^2) $$, $$\displaystyle z^2 = W^2 a^1 + b^2 $$; loss $L$.

Then $$\displaystyle \delta^2 = \nabla_a L \odot \sigma'(z^2) $$, $$\displaystyle \delta^1 = (W^2)^T \delta^2 \odot \sigma'(z^1) $$.

Gradients: $$\displaystyle \frac{\partial L}{\partial W^2} = \delta^2 (a^1)^T $$, etc.

Parallel Processing

• GPUs accelerate matrix operations (convolutions, dense layers) via massive parallelism. Essential for training deep CNNs/RNNs.

B. Convolutional Neural Networks (CNNs)

Core Architecture & Operation

• Convolutional Layer: Applies filters/kernels $K$ to input volume, producing feature maps.

  Output size: $$\displaystyle \frac{W - K + 2P}{S} + 1 $$, where $W$ = input width, $K$ = kernel size, $P$ = padding, $S$ = stride.

• Why downsample (pooling) & increase filters deeper?

  • Downsampling (pooling): Reduces spatial dimensions → fewer parameters, translation invariance, larger receptive field.

  • Increase filters: Deeper layers capture more complex, abstract features (edges → textures → objects).

Convolution Types & Parameters

• Standard Convolution: Full spatial extent.

• 1×1 Convolution:

  \boxed{\text{Purpose: Channel reduction/expansion, feature combination, bottleneck (e.g., Inception, ResNet).}}

  • Reduces computational cost by reducing channels before expensive 3×3/5×5 convs.

  • Adds non-linearity without spatial change.

• Padding:

  • Same: Output size = input size ($$\displaystyle P = \frac{K-1}{2} $$). Preserves spatial dimensions.

  • Valid: No padding ($$\displaystyle P=0 $$); output shrinks.

  • Full: Rare; pads so all pixels covered.

• Stride: Step size; larger stride → smaller output.

Pooling Layers

• Max Pooling: Takes max in window → preserves dominant feature, translation invariance.

• Average Pooling: Averages values → smoother, used in some modern architectures (e.g., Inception).

• Sub-sampling: Role in reducing spatial size, increasing receptive field, providing robustness to small translations.

Key CNN Architectures & Modules

• Inception Module (GoogLeNet):

  Parallel convolutions (1×1, 3×3, 5×5) + max pooling; concatenate outputs.

  Why efficient? 1×1 convs reduce channels before expensive convs → multi-scale features with lower cost.

• ResNet: Skip connections (identity mapping: $$\displaystyle y = F(x) + x $$). Solves vanishing gradient, enables very deep networks (100+ layers).

• Flattening Layer: Converts 3D feature maps (height × width × channels) to 1D vector for fully connected layers.

CNN in Computer Vision

• Applications: Image classification (ImageNet winners: AlexNet 2012 → ReLU, Dropout, GPU; VGG; ResNet), object detection (YOLO, R-CNN), segmentation (U-Net).

• Transfer Learning:

  • Feature Extraction: Freeze pre-trained CNN, use as fixed feature extractor.

  • Fine-tuning: Unfreeze some top layers, train on new data.

C. Recurrent Neural Networks (RNNs) & LSTMs

Vanilla RNN

• Architecture: Looped connections; hidden state $$\displaystyle h_t = f(W_{xh} x_t + W_{hh} h_{t-1} + b) $$.

• Limitations: Vanishing/exploding gradients (due to repeated multiplication), short-term memory (struggles with long sequences).

Long Short-Term Memory (LSTM)

• Cell Structure:

  • Cell state $$\displaystyle C_t $$: "Conveyor belt" – carries long-term info with additive updates.

  • Gates:

    • Forget gate $$\displaystyle f_t = \sigma(W_f [h_{t-1}, x_t] + b_f) $$: What to remove from $$\displaystyle C_{t-1} $$.

    • Input gate $$\displaystyle i_t = \sigma(W_i [h_{t-1}, x_t] + b_i) $$: What new info to store.

    • Candidate $$\displaystyle \tilde{C}_t = \tanh(W_C [h_{t-1}, x_t] + b_C) $$.

    • Update: $$\displaystyle C_t = f_t \odot C_{t-1} + i_t \odot \tilde{C}_t $$.

    • Output gate $$\displaystyle o_t = \sigma(W_o [h_{t-1}, x_t] + b_o) $$; $$\displaystyle h_t = o_t \odot \tanh(C_t) $$.

• How LSTMs handle long-term dependencies: Additive cell state allows gradients to flow unchanged over many time steps, mitigating vanishing gradient.

Gated Recurrent Unit (GRU)

• Simplified LSTM: no cell state; reset gate $$\displaystyle r_t $$ and update gate $$\displaystyle z_t $$.

  $$\displaystyle h_t = (1-z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t $$, where $$\displaystyle \tilde{h}_t = \tanh(W [r_t \odot h_{t-1}, x_t]) $$.

  Fewer parameters, faster training, similar performance.

RNN Applications

• Time series forecasting, NLP (language modeling, machine translation).

• Role in ChatGPT: Transformers (not RNNs) power ChatGPT, but LSTMs were foundational for sequence modeling before attention.

D. Other Neural Network Types

Autoencoders

• Architecture: Encoder (compress input to latent space) → Bottleneck → Decoder (reconstruct input).

• Unsupervised uses: Dimensionality reduction (latent space), denoising (train with noisy input, clean output), anomaly detection (high reconstruction error).

Generative Models

• Generative AI Overview:

  • GANs: Generator vs Discriminator adversarial training.

  • VAEs: Learn latent distribution; sample from it to generate.

  • Diffusion Models: Add noise, learn to reverse; state-of-the-art for images (Stable Diffusion).

• Self-Supervised Learning: Pretext tasks from unlabeled data (e.g., predict missing patch, contrastive learning like SimCLR). Reduces need for labeled data.

VI. SUPPORT VECTOR MACHINES (SVM)

Core Objective & Hyperplane

• Find optimal hyperplane that maximizes margin between classes.

• Margin: Distance between hyperplane and nearest points (support vectors).

• Support Vectors: Training points on margin boundaries; define decision boundary; only they affect final model.

Mathematical Formulation

• For linearly separable data:

  \boxed{\begin{aligned} &\text{Minimize: } \frac{1}{2} |w|^2 \ &\text{Subject to: } y_i (w \cdot x_i + b) \geq 1 \quad \forall i \end{aligned}}

• Use Lagrange multipliers → dual problem: maximize $$\displaystyle \sum \alpha_i - \frac{1}{2}\sum_i\sum_j \alpha_i \alpha_j y_i y_j x_i \cdot x_j $$ s.t. $$\displaystyle \alpha_i \geq 0 $$, $$\displaystyle \sum \alpha_i y_i = 0 $$.

• Decision function: $$\displaystyle f(x) = \text{sign}\left(\sum \alpha_i y_i x_i \cdot x + b\right) $$.

Kernel Functions

• Kernel Trick: Compute inner products in high-dimensional space without explicit mapping.

  $$\displaystyle K(x_i, x_j) = \phi(x_i) \cdot \phi(x_j) $$.

• Common kernels:

  • Linear: $$\displaystyle K(x_i, x_j) = x_i \cdot x_j $$

  • Polynomial: $$\displaystyle K(x_i, x_j) = (\gamma x_i \cdot x_j + r)^d $$

  • RBF (Gaussian): $$\displaystyle K(x_i, x_j) = \exp(-\gamma \|x_i - x_j\|^2) $$ – maps to infinite-dim, very flexible.

SVM in High-Dimensional Spaces

• Effective when features >> samples (e.g., bioinformatics, text classification) because:

  1. Kernel trick avoids explicit high-dim computation.

  2. Max-margin principle generalizes well despite high dim.

• Soft Margin SVM: For non-separable data, introduce slack variables $$\displaystyle \xi_i \geq 0 $$:

  Minimize $$\displaystyle \frac{1}{2}\|w\|^2 + C \sum \xi_i $$ s.t. $$\displaystyle y_i(w\cdot x_i + b) \geq 1 - \xi_i $$.

  $C$ controls trade-off: large $C$ → hard margin (less tolerant), small $C$ → softer.

VII. REINFORCEMENT LEARNING (RL)

Fundamental Framework

• Markov Decision Process (MDP): $(S, A, P, R, \gamma)$

  • $S$: states, $A$: actions, $P(s'|s,a)$: transition probability, $R(s,a,s')$: reward, $\gamma$: discount factor (0≤γ<1).

• Policy $\pi(a|s)$: probability of taking action $a$ in state $s$.

• Value Functions:

  • State-value: $$\displaystyle V^\pi(s) = \mathbb{E}_\pi\left[\sum_{t=0}^\infty \gamma^t R_{t+1} \mid S_0=s\right] $$

  • Action-value: $$\displaystyle Q^\pi(s,a) = \mathbb{E}_\pi\left[\sum_{t=0}^\infty \gamma^t R_{t+1} \mid S_0=s, A_0=a\right] $$

• Bellman Equation:

  \boxed{V^\pi(s) = \sum_a \pi(a|s) \sum_{s'} P(s'|s,a) \left[ R(s,a,s') + \gamma V^\pi(s') \right]}

  Optimal $$\displaystyle V^*(s) = \max_a \sum_{s'} P(s'|s,a) \left[ R(s,a,s') + \gamma V^*(s') \right] $$.

RL vs. Supervised/Unsupervised Learning

• RL: Reward signal (sparse/delayed), sequential decisions, exploration-exploitation tradeoff, environment interaction.

• Supervised: Labeled data, immediate feedback, independent samples.

• Unsupervised: Unlabeled data, find structure, no reward.

Dynamic Programming Methods (assume known $P$, $R$)

• Policy Iteration:

  1. Policy Evaluation: Compute $$\displaystyle V^\pi $$ for current $\pi$ (solve linear system).

  2. Policy Improvement: Update $\pi$ greedily: $$\displaystyle \pi'(s) = \arg\max_a \sum_{s'} P(s'|s,a)[R + \gamma V^\pi(s')] $$.

  Repeat until $\pi$ stable.

• Value Iteration:

  Directly update $V$: $$\displaystyle V_{k+1}(s) = \max_a \sum_{s'} P(s'|s,a)[R + \gamma V_k(s')] $$.

  Faster than policy iteration; no explicit policy until end.

Model-Free Learning (unknown $P$, $R$)

• Q-Learning (off-policy):

  \boxed{Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a'} Q(s',a') - Q(s,a) \right]}

  • Learns optimal $$\displaystyle Q^* $$ independent of exploration policy.

  • Use $\epsilon$-greedy: with prob $\epsilon$ random action, else $$\displaystyle \arg\max_a Q(s,a) $$.

• SARSA (on-policy):

  \boxed{Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma Q(s',a') - Q(s,a) \right]}

  where $a'$ is action actually taken (from current policy).

  Difference: Q-learning uses $$\displaystyle \max_{a'} Q(s',a') $$ (optimistic); SARSA uses $Q(s',a')$ (actual). SARSA is more conservative, considers exploration.

Actor-Critic Methods

• Architecture:

  • Actor: Policy network $$\displaystyle \pi_\theta(a|s) $$; selects actions.

  • Critic: Value network $$\displaystyle V_w(s) $$ or $$\displaystyle Q_w(s,a) $$; evaluates state/action.

• Interaction:

  1. Actor generates action $$\displaystyle a_t \sim \pi_\theta(\cdot|s_t) $$.

  2. Environment returns $$\displaystyle r_t, s_{t+1} $$.

  3. Critic computes TD error: $$\displaystyle \delta_t = r_t + \gamma V_w(s_{t+1}) - V_w(s_t) $$.

  4. Critic updates: $$\displaystyle w \leftarrow w + \beta \delta_t \nabla_w V_w(s_t) $$.

  5. Actor updates: $$\displaystyle \theta \leftarrow \theta + \alpha \delta_t \nabla_\theta \log \pi_\theta(a_t|s_t) $$.

• Advanced Models:

  • A2C (Advantage Actor-Critic): Synchronous, uses advantage $$\displaystyle A(s,a) = Q(s,a) - V(s) $$.

  • A3C (Asynchronous): Multiple parallel actors.

  • PPO (Proximal Policy Optimization): Clips policy update to ensure small steps.

  • DDPG (Deep Deterministic Policy Gradient): For continuous actions, deterministic policy.

RL Frameworks & Tools

• OpenAI Gym: Standardized environments (Atari, MuJoCo).

• Stable Baselines3: Reliable implementations of PPO, SAC, etc.

• TensorFlow Agents: RL algorithms in TensorFlow.

Applications of RL

• Games: AlphaGo (board games), DQN (Atari).

• Robotics: Control policies.

• Recommendation systems: Sequential user engagement.

VIII. PROBABILITY & STATISTICAL FOUNDATIONS

Role of Probability in ML

• Models uncertainty, Bayesian inference, probabilistic predictions (e.g., Naive Bayes, Bayesian networks, Gaussian processes).

Bayes' Theorem

\boxed{P(A|B) = \frac{P(B|A) P(A)}{P(B)}}

• Use in Classification (Naive Bayes):

  $$\displaystyle P(\text{class}|\text{features}) \propto P(\text{class}) \prod_{i} P(\text{feature}_i|\text{class}) $$

  Assumes feature independence given class.

• Fundamental to probabilistic models: Updates prior belief with evidence (likelihood) to get posterior.

Bayesian Learning

• Prior $P(\theta)$: Belief about parameters before data.

• Likelihood $P(D|\theta)$: Probability of data given parameters.

• Posterior $$\displaystyle P(\theta|D) = \frac{P(D|\theta) P(\theta)}{P(D)} $$.

• MAP (Maximum A Posteriori): $$\displaystyle \hat{\theta}_{\text{MAP}} = \arg\max_\theta P(\theta|D) = \arg\max_\theta P(D|\theta) P(\theta) $$.

• MLE (Maximum Likelihood Estimation): $$\displaystyle \hat{\theta}_{\text{MLE}} = \arg\max_\theta P(D|\theta) $$ (assumes uniform prior).

Bayesian Networks

• Directed acyclic graph (DAG); nodes = random variables, edges = conditional dependencies.

• Encodes conditional independence; enables efficient inference (e.g., via variable elimination).

IX. SPECIFIC ALGORITHMS & TECHNIQUES

Decision Trees

• Splitting Criteria:

  • Entropy: $$\displaystyle H(p) = -\sum_{i=1}^c p_i \log_2 p_i $$

  • Gini Impurity: $$\displaystyle G = 1 - \sum_{i=1}^c p_i^2 $$

  • Information Gain: $$\displaystyle \text{IG} = H(\text{parent}) - \sum_{\text{children}} \frac{n_{\text{child}}}{n_{\text{parent}}} H(\text{child}) $$.

• Algorithms: CART (binary splits, Gini), ID3 (entropy), C4.5 (gain ratio).

• Issues: Overfitting (prune), instability (small data changes → different tree). Solved by Random Forests.

Clustering Algorithms

• K-Means:

  1. Initialize $k$ centroids randomly.

  2. Assign each point to nearest centroid (Euclidean distance).

  3. Update centroids as mean of assigned points.

  4. Repeat 2–3 until convergence.

  • Elbow Method: Plot within-cluster sum of squares (WCSS) vs $k$; choose $k$ at "elbow".

• Expectation-Maximization (EM):

  • E-step: Compute responsibilities $$\displaystyle \gamma(z|x) = P(z|x,\theta^{\text{old}}) $$ (posterior probability of cluster given data).

  • M-step: Update parameters $$\displaystyle \theta^{\text{new}} = \arg\max_\theta \sum_x \sum_z \gamma(z|x) \log P(x,z|\theta) $$.

  • Used for Gaussian Mixture Models (GMM); handles missing data naturally.

• Hierarchical Clustering:

  • Agglomerative (bottom-up): Start with each point as cluster; merge closest pairs.

  • DIANA (Divisive Analysis): Top-down; start with one cluster, split recursively.

• Requirements of clustering algorithms: Scalability, ability to handle noise, insensitivity to order, ability to deal with arbitrary shapes.

Instance-Based Learning

• K-Nearest Neighbors (KNN):

  • Lazy learning: No explicit training; store all data.

  • Prediction: For query $x$, find $k$ nearest neighbors (distance metric: Euclidean, Manhattan), predict majority class (classification) or average (regression).

  • Example: Given training data with features (speed, agility) and label (drafted yes/no), for $$\displaystyle x=(6.75, 3) $$ with $$\displaystyle k=3 $$, find 3 nearest neighbors, take majority vote.

Ensemble Methods

• Bagging (Bootstrap Aggregating): Train multiple models on bootstrapped samples; average/vote. Reduces variance. Example: Random Forest (decision trees + feature randomness).

• Boosting: Train sequentially; each model focuses on errors of previous. Reduces bias. Examples: AdaBoost, Gradient Boosting.

• Stacking: Train meta-learner on outputs of base models.

X. NATURAL LANGUAGE PROCESSING (NLP) & APPLICATIONS

NLP Pipeline & Steps

1. Tokenization: Split text into words/subwords.

2. Stop-word removal: Remove common words (the, is).

3. Stemming/Lemmatization: Reduce to root form (Porter stemmer, WordNet lemmatizer).

4. Vectorization:

   • Bag-of-Words (BoW): Count word frequencies.

   • TF-IDF: $$\displaystyle \text{TF-IDF}(t,d) = \text{tf}(t,d) \cdot \log\frac{N}{\text{df}(t)} $$, where $\text{tf}$ = term frequency, $\text{df}$ = document frequency.

   • Word Embeddings: Dense vectors (Word2Vec, GloVe) capture semantics.

5. Sequence Models: RNNs/LSTMs, Transformers.

Sequence Models & Metrics

• LSTMs in NLP: Language generation, machine translation (historically; now Transformers dominate).

• Attention Mechanism: Weighted sum of context vectors; allows model to focus on relevant parts (foundation for Transformers).

• BLEU Score: For machine translation evaluation.

  \boxed{\text{BLEU} = \text{BP} \cdot \exp\left(\sum_{n=1}^N w_n \log p_n\right)}

  where $$\displaystyle p_n $$ = n-gram precision (clipped), BP = brevity penalty ($$\displaystyle \min(1, e^{1-l_{\text{ref}}/l_{\text{hyp}}}) $$).

Speech Processing Applications

• Speech-to-Text (ASR):

  • Feature extraction: MFCC (Mel-Frequency Cepstral Coefficients).

  • Acoustic model: Maps MFCC to phonemes (HMM + GMM historically, now DNN/RNN).

  • Language model: Predicts word sequences (n-grams, RNNs).

• Speaker Identification/Verification:

  • i-vectors/x-vectors: Fixed-dimensional embeddings from speech.

  • Compare embeddings using cosine distance.

• Synthesis (Text-to-Speech):

  • Tacotron: Sequence-to-sequence with attention.

  • WaveNet: Autoregressive raw audio generation.

XI. COMPUTER VISION (CV) & ADVANCED TOPICS

CNN Applications in CV

• Image Classification: AlexNet (2012), VGG, ResNet.

• Object Detection:

  • YOLO (You Only Look Once): Single-stage, fast.

  • R-CNN family: Two-stage (region proposal + classification).

• Segmentation: U-Net (encoder-decoder with skip connections) for medical images.

Historical Benchmark: ImageNet Competition

• ILSVRC (ImageNet Large Scale Visual Recognition Challenge).

• Impact:

  • 2012: AlexNet (ReLU, Dropout, GPU training) dramatically reduced error (15.3% → 10.2%).

  • Spurred deep learning revolution: VGG (2014), ResNet (2015, residual blocks, 3.57% error).

  • Demonstrated power of deep CNNs with large data.

Advanced Topics

• One-Shot Learning: Learn from very few examples (e.g., one per class).

  • How differs from supervised? Traditional SL needs many examples; one-shot uses metric learning (Siamese networks) or meta-learning (learn to learn).

  • Applications: Face recognition (new person with one photo), rare event detection.

• Generative AI in CV:

  • GANs: StyleGAN for high-fidelity face synthesis.

  • Diffusion Models: Stable Diffusion, DALL-E 2 for text-to-image generation.

XII. FRAMEWORKS & IMPLEMENTATION

Popular ML/DL Frameworks

• TensorFlow/Keras: High-level API, production-ready, extensive tools.

• PyTorch: Dynamic computation graph, research-friendly, Pythonic.

• scikit-learn: Traditional ML algorithms, preprocessing, evaluation.

• RL-specific:

  • OpenAI Gym: Standard environments (Atari, robotics).

  • Stable Baselines3: Reliable RL algorithm implementations (PPO, SAC).

  • TensorFlow Agents: RL in TensorFlow.

Implementation Note: CNN in TensorFlow/Keras

model = Sequential([

    Conv2D(filters=32, kernel_size=(3,3), activation='relu', input_shape=(28,28,1)),

    MaxPool2D(pool_size=(2,2)),

    Conv2D(64, (3,3), activation='relu'),

    MaxPool2D((2,2)),

    Flatten(),  # Convert 3D to 1D

    Dense(128, activation='relu'),

    Dense(10, activation='softmax')

])

PRIORITY RECAP FOR EXAM:

• 🔴 Very High: CNN arch (1×1 conv, padding), RL (Q-learning, Actor-Critic, MDP), SVM (kernel, margin), Regularization (L1/L2), Evaluation Metrics, RNNs/LSTMs, NLP/Speech apps.

• 🟠 High: Backpropagation, Activation functions, Transfer learning, Inception, Bayes' Theorem, Decision Trees, Clustering (K-means, EM), Cross-validation, Loss functions.

• 🟡 Medium: Autoencoders, One-shot learning, Self-supervised learning, Generative AI, ImageNet, Attention, Batch Norm, Perceptron, PCA, BLEU, Sub-sampling.

• 🟢 Low: Specific frameworks (TF Agents), Flattening, Convex optimization, Linearity vs non-linearity, DIANA, Policy iteration.

[!TIP] Exam Strategy: For 7-mark questions, define clearly, give formulas, explain with simple examples, and mention applications. For derivations (backpropagation, Q-learning), show step-by-step. Always connect theory to real-world use cases (e.g., "LSTMs handle long-term dependencies in machine translation").