AI & Signal Processing (EC-802 (A)) - Unit 1 Short Notes

UNIT 1: Artificial Intelligence and Signal Processing Fundamentals

I. ARTIFICIAL INTELLIGENCE (AI)

A. Foundations of AI

Definition & Scope:

Artificial Intelligence is the science and engineering of creating intelligent machines that can perform tasks requiring human-like cognition—such as learning, reasoning, problem-solving, perception, and language understanding.

AI vs. ML vs. DL Hierarchy:

Natural (Human) Intelligence vs. Artificial Intelligence:

[!TIP] Exam Focus:

• Be ready to differentiate AI/ML/DL with examples.

• Compare Human vs AI intelligence on at least 4 parameters (common question in May 2023).

B. Knowledge Representation and Reasoning

Role of Knowledge in AI:

Knowledge (facts, rules, relationships) enables AI systems to reason, make decisions, and solve problems. It bridges raw data and intelligent behavior.

Declarative vs. Procedural Knowledge:

Utility Theory in AI:

• Concept: Quantifies the "usefulness" or payoff of outcomes to make optimal decisions under uncertainty.

• Importance: Enables rational choice by maximizing expected utility.

• Example: A self-driving car assigns utilities: collision = -100, safe stop = 0, smooth ride = +10. It chooses actions with highest expected utility.

Inference Mechanisms:

1. Forward Chaining

• Method: Start with known facts, apply rules to derive new facts until goal is reached (data-driven).

• Example:

  Rules:

  R1: If fever and cough → possible flu

  R2: If flu → take rest

  Facts: fever = true, cough = true

  → Derive possible flu → take rest.

• Advantages: Good for exploration, real-time monitoring.

• Disadvantages: May derive irrelevant facts (inefficient if goal is known).

2. Backward Chaining

• Method: Start with goal, work backward to find supporting facts (goal-driven).

• Example:

  Goal: take rest? Check R2: need flu.

  Check R1: need fever and cough → both true → goal achieved.

• Advantages: Efficient for specific queries, avoids irrelevant facts.

• Disadvantages: May fail if goal is unreachable, poor for open-ended exploration.

[!TIP] Common Pitfall:

• Confusing forward (facts → conclusion) with backward (conclusion → facts).

• Always state which chaining is used in expert systems (often backward).

C. Probabilistic Graphical Models

Markov Models:

• Structure: States with transition probabilities; next state depends only on current state (Markov property).

• Types:

  • Markov Chain: Observable states.

  • Hidden Markov Model (HMM): Hidden states, observable emissions.

• AI Applications:

  • Speech recognition (HMM: phonemes → audio signals).

  • Stock prediction (Markov chain).

  • Part-of-speech tagging (HMM).

Bayesian Networks (Belief Networks):

• Representation: Directed acyclic graph (DAG) where nodes = random variables, edges = conditional dependencies.

• Inference: Compute posterior probabilities using Bayes' theorem (e.g., P(A|B) = P(B|A)P(A)/P(B)).

• Example: Medical diagnosis:

  Nodes: Flu, Cough, Fever.

  Flu → Cough, Flu → Fever.

  Given Cough=true, Fever=true, compute P(Flu=true).

• Practical Importance: Handles uncertainty, combines evidence, used in diagnostics, risk analysis.

[!TIP] Exam Tip:

• For HMM, remember three problems: evaluation (forward algorithm), decoding (Viterbi), learning (Baum-Welch).

• Bayesian Networks require understanding of conditional independence.

D. Machine Learning Fundamentals

Learning Models & Paradigms:

Factors Affecting Learning Process:

1. Data Quality: Noise, missing values, bias → poor generalization.

2. Algorithm Selection: Complexity vs. interpretability (e.g., deep net vs. linear regression).

3. Evaluation Metrics: Accuracy, precision, recall, F1-score—choose based on task (e.g., medical: recall > accuracy).

4. Overfitting/Underfitting: Balance model complexity with data size.

Decision Trees vs. Naive Bayes Comparison:

[!TIP] Past Paper Link:

• May 2023 asked: "Compare Decision Tree with Naive Bayes" → use table above.

• Always mention Naive Bayes' independence assumption as key weakness.

II. SIGNAL PROCESSING

A. Discrete Fourier Transform (DFT)

Definition:

For a sequence \( x(n) \) of length \( N \),

$$ X(k) = \sum_{n=0}^{N-1} x(n) e^{-j\frac{2\pi}{N}kn} = \sum_{n=0}^{N-1} x(n) W_N^{kn}, \quad W_N = e^{-j\frac{2\pi}{N}} $$

Inverse DFT:

$$ x(n) = \frac{1}{N} \sum_{k=0}^{N-1} X(k) W_N^{-kn} $$

Key Properties & Theorems:

1. Circular Convolution Property

• Statement: Circular convolution of two length-\(N\) sequences in time domain equals product of their DFTs:

$$ x_1(n) \circledast x_2(n) \xrightarrow{\text{DFT}} X_1(k) \cdot X_2(k) $$

• Proof:

  Let \( y(n) = x_1(n) \circledast x_2(n) = \sum_{m=0}^{N-1} x_1(m) x_2(n-m \mod N) \).

  DFT:

$$ Y(k) = \sum_{n=0}^{N-1} y(n) W_N^{kn} = \sum_{n=0}^{N-1} \sum_{m=0}^{N-1} x_1(m) x_2(n-m) W_N^{kn} $$

Substitute \( l = n-m \):

$$ = \sum_{m=0}^{N-1} x_1(m) W_N^{km} \sum_{l=0}^{N-1} x_2(l) W_N^{kl} = X_1(k) X_2(k) \quad \boxed{} $$

2. DFT of Real and Even Sequence

• Statement: If \( x(n) \) is real and even (\( x(n) = x(N-n) \)), then \( X(k) \) is real and even.

• Proof:

  \( X(k) = \sum_{n=0}^{N-1} x(n) \cos\left(\frac{2\pi}{N}kn\right) \) (since sine terms cancel due to evenness).

  Thus \( X(k) \) is real. Also, \( X(k) = X(N-k) \) → even.

Modulation Property (Time Multiplication):

If \( g(n) = W_N^{-kn} x(n) \), then

$$ G(m) = X(m+k \mod N) $$

Effect: Multiplication by \( W_N^{-kn} \) circularly shifts DFT by \( k \).

[!TIP] Past Paper Example (May 2023):

Given \( x_1(n) = \sin(3\pi n/8) \), \( x_2(n) = \sin(3\pi n/8) \), compute 8-point DFT circular convolution.

Solution Sketch:

1. Compute \( X_1(k) \) for 8-point DFT of \( \sin(3\pi n/8) \).

2. Use property: \( \sin(\theta) = \frac{e^{j\theta} - e^{-j\theta}}{2j} \) → DFT has impulses at \( k=3 \) and \( k=5 \).

3. \( Y(k) = X_1(k) \cdot X_1(k) \) → non-zero only at \( k=3,5 \).

4. Compute \( y(n) = \text{IDFT}[Y(k)] \).

B. Multi-rate Signal Processing

Filter Banks:

• Concept: Decompose signal into subbands using analysis filters, process separately, reconstruct with synthesis filters.

• Quadrature Mirror Filter Banks (QMF):

  • Analysis filters: \( H_0(z) \) (lowpass), \( H_1(z) \) (highpass).

  • QMF Condition: \( H_1(z) = H_0(-z) \) → alias cancellation possible.

  • Application: Subband coding (e.g., JPEG2000, speech coding).

• Types:

  | Type | Description | Sampling | |------------------------|----------------------------------------------|----------------------| | Analysis/Synthesis | Split/recombine signal | Decimation/Interpolation | | Critically Sampled | Total output samples = input samples | No redundancy | | Oversampled | More output samples → robustness to aliasing | >1x sampling |

Sampling Rate Conversion:

• Efficient Structure: Polyphase Implementation

  For rate change by \( M \):

$$ x(n) \downarrow M \quad \text{or} \quad x(n) \uparrow L $$

Polyphase decomposition:

$$ H(z) = \sum_{i=0}^{M-1} z^{-i} E_i(z^M) \quad \text{(for decimation)} $$

Advantage: Reduces computation by factor \( M \), avoids unnecessary filtering.

[!TIP] Exam Focus:

• QMF often asked with alias cancellation condition.

• Polyphase structure is key for efficient converters—draw block diagram.

C. Digital Filter Design and Structures

FIR vs IIR Basic Structures:

Parallel All-Pass Realization for IIR:

Any stable IIR transfer function \( H(z) \) can be written as sum of all-pass filters plus a constant:

$$ H(z) = \sum_{i=1}^{K} \frac{\alpha_i + z^{-1}}{1 + \alpha_i z^{-1}} + C $$

Advantage: Guarantees stability if all \( |\alpha_i| < 1 \), useful for phase-sensitive applications.

[!TIP] Past Paper Link:

• May 2023 asked: "Parallel all-pass realization of IIR" → write the sum-of-all-pass form.

D. Interpolation Techniques

Spline Interpolation:

• Principle: Fit piecewise polynomials (splines) between data points with continuity constraints.

• Types:

  • Linear Spline: Piecewise linear, \( C^0 \) continuity. Simple but not smooth.

  • Cubic Spline: Piecewise cubic, \( C^2 \) continuity (smooth first/second derivatives). Most common.

• Applications in Signal Processing:

  • Signal reconstruction from samples (e.g., audio upsampling).

  • Image scaling (cubic spline preserves edges better than linear).

  • Smooth curve fitting in biomedical signals.

Mathematical Form (Cubic Spline):

For interval \([x_i, x_{i+1}]\),

$$ S_i(x) = a_i + b_i(x-x_i) + c_i(x-x_i)^2 + d_i(x-x_i)^3 $$

Coefficients determined by:

1. Function values match at knots.

2. First/second derivatives continuous at interior knots.

3. Boundary conditions (e.g., natural: \( S''(x_0)=S''(x_N)=0 \)).

[!TIP] Short Note Format:

• Define spline → types → one application → mention \( C^0, C^1, C^2 \) continuity levels.

Final Exam Strategy:

1. AI Section: Focus on definitions, comparisons, and examples (past papers heavily test these).

2. Signal Processing: Derivations (circular convolution proof) and numerical problems (DFT computation) are frequent.

3. Always box final formulas and use tables for comparisons.

4. For inference mechanisms (chaining), draw a small example—examiners love diagrams.