UNIT 3: DIGITAL COMMUNICATION - EXAM-FOCUSED SHORT NOTES

1.0 PULSE MODULATION TECHNIQUES

Pulse Modulation converts a continuous-time analog signal into a discrete-time pulse train where one or more pulse parameters (amplitude, width, position) vary with the message signal.

1.1 Pulse Amplitude Modulation (PAM)

• Generation: Sample-and-hold circuit. The amplitude of a rectangular pulse train follows the instantaneous amplitude of the message signal $m(t)$ at sampling instants.

• Detection: Simple low-pass filter (reconstruction filter) to recover $m(t)$.

• Spectral Characteristics: Spectrum is a scaled and periodic repetition of the original message spectrum, scaled by the sampling frequency $$\displaystyle f_s $$.

• Key Point: Most basic form; susceptible to noise as noise directly affects pulse amplitude.

1.2 Pulse Width Modulation (PWM)

• Generation: Compare $m(t)$ with a high-frequency sawtooth/triangular wave $c(t)$ using a comparator.

  • When $$\displaystyle m(t) > c(t) $$ → output high.

  • When $$\displaystyle m(t) < c(t) $$ → output low.

  • Result: Pulse width (duration) is proportional to $m(t)$.

  • DiagramSEARCH: PWM generation comparator circuit

• Detection: Use a sample-and-hold circuit synchronized with the pulse train, followed by a low-pass filter. The sample-and-hold captures the average voltage over each pulse, which is proportional to its width.

1.3 Pulse Position Modulation (PPM)

• Generation:

  1. Generate a fixed-width pulse train (clock).

  2. Use a monostable multivibrator triggered by each clock pulse.

  3. The delay from the clock edge to the leading edge of the PPM pulse is controlled by $m(t)$.

  • Result: Pulse position (timing) varies with $m(t)$.

  • DiagramSEARCH: PPM generation monostable multivibrator

• Detection: Use a correlator or matched filter synchronized to the expected pulse positions. The output peak indicates the position.

1.4 Comparison of PAM, PWM, and PPM

[!TIP] Exam Focus: Be prepared to draw generation circuits for PWM and PPM and explain why PPM is most power-efficient and noise-robust.

2.0 SAMPLING THEORY & ALIASING

2.1 Sampling Theorem for Lowpass Signals (Nyquist Theorem)

• Statement: A bandlimited signal with maximum frequency $$\displaystyle f_m $$ can be reconstructed perfectly from its samples if the sampling frequency $$\displaystyle f_s \geq 2f_m $$.

• Condition: $$\displaystyle f_s \geq 2f_{\text{max}} \quad \text{(Nyquist Rate)} $$

• Derivation Intuition: Sampling in time domain replicates the signal's spectrum at multiples of $$\displaystyle f_s $$. To avoid overlap (aliasing), the spectral replicas must not overlap. This requires $$\displaystyle f_s - f_m \geq f_m \Rightarrow f_s \geq 2f_m $$.

• Critical Frequency: $$\displaystyle f_N = 2f_m $$ is the Nyquist rate.

2.2 Sampling Theorem for Bandpass Signals

• Statement: For a bandpass signal with bandwidth $B$ and center frequency $$\displaystyle f_c $$, the minimum sampling rate is $$\displaystyle f_s \geq 2B $$.

• Condition: The signal's spectrum consists of non-zero bands only within $$\displaystyle [f_c - B/2, f_c + B/2] $$ and symmetric around $$\displaystyle f_c $$. Sampling at $2B$ can capture all information if the bands are "isolated" and $$\displaystyle f_c \gg B $$.

• Key: Exploits spectral gaps; allows undersampling (below $$\displaystyle 2f_{\text{max}} $$) for bandpass signals.

2.3 Types of Sampling

• Ideal (Impulse) Sampling:

  • Message $m(t)$ multiplied by an impulse train $$\displaystyle \sum_{n=-\infty}^{\infty} \delta(t - nT_s) $$.

  • Sampled Signal: $$\displaystyle g(t) = m(t) \cdot \sum_{n=-\infty}^{\infty} \delta(t - nT_s) = \sum_{n=-\infty}^{\infty} m(nT_s) \delta(t - nT_s) $$.

  • Spectrum: $$\displaystyle G(f) = f_s \sum_{k=-\infty}^{\infty} M(f - k f_s) $$.

  • Very High-Frequency Derivation Topic.

• Natural Sampling: Top of each pulse follows $m(t)$ during the pulse duration $\tau$. Pulse amplitude is $m(t)$ at the start of the pulse.

• Flat-Top (Practical) Sampling: Pulse amplitude is held constant (sample-and-hold) for duration $\tau$. Most common in practical A/D converters.

• Aperture Effect: In flat-top sampling, the finite pulse width $\tau$ acts as a low-pass filter, attenuating high-frequency components of $m(t)$. Causes temporal blurring.

2.4 Aliasing

• Definition: Distortion that occurs when $$\displaystyle f_s < 2f_{\text{max}} $$, causing spectral replicas to overlap. High-frequency components "fold back" into the baseband, appearing as false lower frequencies.

• Cause: Violation of Nyquist criterion.

• Anti-aliasing Filter: A low-pass analog filter placed before the sampler to bandlimit $m(t)$ to $$\displaystyle f_m \leq f_s/2 $$. Essential in all practical sampling systems.

2.5 Time Division Multiplexing (TDM)

• Principle: Multiple low-rate signals share a single high-bandwidth channel by taking turns in time. Each signal is sampled, and samples are interleaved into a single high-rate composite frame.

• Frame: Contains one sample from each channel (and often a sync pulse).

• Bit Rate: $$\displaystyle R_b = n \cdot f_s \cdot \log_2(L) $$, where $n$ = number of channels, $$\displaystyle f_s $$ = sampling rate per channel, $L$ = quantization levels.

[!TIP] Common Pitfall: Aliasing is irreversible. Once samples are taken with $$\displaystyle f_s < 2f_{\text{max}} $$, the original high frequencies cannot be recovered. Always apply anti-aliasing filter.

3.0 QUANTIZATION & PULSE CODE MODULATION (PCM)

3.1 Quantization

• Need: Sampling produces a continuous-amplitude signal. To convert to digital, amplitude must be mapped to a finite set of discrete levels.

• Process: Partition the amplitude range $$\displaystyle [V_{\min}, V_{\max}] $$ into $L$ intervals (quantization cells) of size $\Delta$.

  • $$\displaystyle L = 2^n $$, where $n$ = number of bits.

  • $$\displaystyle \Delta = \frac{V_{\max} - V_{\min}}{L} $$.

• Uniform Quantization: $\Delta$ constant. Used for signals with uniform amplitude distribution.

• Non-uniform Quantization: $\Delta$ varies (smaller for small amplitudes, larger for large). Exploits human perception (e.g., audio). Achieved via companding.

3.2 Quantization Noise & SQNR

• Quantization Error/Noise: $$\displaystyle e_q = x_q - x $$, where $x$ is original sample, $$\displaystyle x_q $$ is quantized level. Assumed uniformly distributed in $[-\Delta/2, \Delta/2]$ for uniform quantizer with high resolution.

• Noise Power (Uniform Quantizer):

$$P_q = \frac{\Delta^2}{12}$$

• Signal Power (for full-scale sine wave $$\displaystyle x(t) = A \sin(2\pi f t) $$):

$$P_s = \frac{A^2}{2} = \frac{(L\Delta/2)^2}{2} = \frac{L^2 \Delta^2}{8}$$

• Signal-to-Quantization-Noise Ratio (SQNR):

$$\text{SQNR} = \frac{P_s}{P_q} = \frac{L^2 \Delta^2 / 8}{\Delta^2 / 12} = \frac{3}{2} L^2 = 3 \cdot 2^{2n}$$

In **dB**:

$$\text{SQNR}_{\text{dB}} \approx 10 \log_{10}(3 \cdot 2^{2n}) \approx 1.76 + 6.02n \ \text{dB}$$

> **\boxed{\text{SQNR}_{\text{dB}} \approx 1.76 + 6.02n \ \text{dB}}**

*   **Very High-Frequency Derivation.**

3.3 Companding

• Purpose: Achieve non-uniform quantization effect using a uniform quantizer by compressing the signal before quantization and expanding after.

• A-law (Europe, ITU-T):

$$F(x) = \frac{\text{sgn}(x)}{1 + \ln(A)} \cdot \begin{cases} \frac{A|x|}{1+\ln A}, & |x| \leq 1/A \\ 1 + \ln(A|x|), & |x| > 1/A \end{cases} \quad (A=87.6)$$

• μ-law (USA, ITU-T):

$$F(x) = \frac{\text{sgn}(x) \ln(1+\mu|x|)}{\ln(1+\mu)} \quad (\mu=255)$$

• High-Frequency: Know the shapes (compressive for small $|x|$).

3.4 Pulse Code Modulation (PCM)

• Complete Transmitter Block Diagram:

  Anti-aliasing Filter → Sampler → Quantizer → Encoder → Channel

• Complete Receiver Block Diagram:

  Channel → Decoder → Reconstruction Filter (LPF) → Hold Circuit

• Output Bit Rate:

$$R_b = f_s \cdot n \ \text{bits/sec}$$

where $$\displaystyle f_s $$ = sampling rate, $n$ = bits/sample.

• Very High-Frequency: Be able to explain each block's function in detail.

3.5 DPCM & Delta Modulation (DM)

• DPCM (Differential PCM): Encodes the difference between current sample and a predicted value (from past samples). Reduces bits if signal is correlated.

• Delta Modulation (DM): Simplest DPCM. 1-bit quantizer.

  • Encoder: Compares $m(t)$ with a locally reconstructed signal $\hat{m}(t)$. Outputs 1 if $$\displaystyle m(t) > \hat{m}(t) $$ (step up), 0 if $$\displaystyle m(t) < \hat{m}(t) $$ (step down).

  • Decoder: Integrates the 1/0 stream to reconstruct $\hat{m}(t)$.

  • DiagramSEARCH: Delta modulator block diagram

• Distortions:

  • Slope Overload: Occurs when signal changes faster than the step size $\Delta$ can follow. Solution: Increase $\Delta$ or use Adaptive DM (ADM) where $\Delta$ adjusts based on signal slope.

  • Granular Noise: Occurs when signal is nearly constant but output toggles due to $\Delta$. Solution: Decrease $\Delta$.

• High-Frequency: Understand the trade-off between $\Delta$, slope overload, and granular noise.

4.0 BASEBAND TRANSMISSION & LINE CODING

4.1 Inter-Symbol Interference (ISI)

• Definition: Overlap of adjacent symbols in time at the decision instant, caused by channel dispersion (limited bandwidth, multipath).

• Cause: Channel impulse response $h(t)$ has duration > symbol period $$\displaystyle T_b $$. Received signal is convolution of transmitted pulse train with $h(t)$.

• Nyquist Criterion for Zero ISI: The overall system response (transmit filter + channel + receive filter) must satisfy:

$$H(f) = \frac{1}{f_s} \sum_{k=-\infty}^{\infty} X\left(f - \frac{k}{T_s}\right)$$

For baseband, **time-domain condition**:

$$h_{\text{total}}(t = kT_s) = \begin{cases} 1, & k=0 \\ 0, & k \neq 0 \end{cases}$$

• Methods to Minimize ISI:

  • Raised Cosine Filter: Has frequency response:

$$H(f) = \begin{cases} 1, & |f| \leq \frac{1-\alpha}{2T_s} \\ \frac{1}{2}\left[1 + \cos\left(\frac{\pi T_s}{\alpha}\left(|f| - \frac{1-\alpha}{2T_s}\right)\right)\right], & \frac{1-\alpha}{2T_s} < |f| \leq \frac{1+\alpha}{2T_s} \\ 0, & |f| > \frac{1+\alpha}{2T_s} \end{cases}$$

    where $\alpha$ = roll-off factor (0 to 1). $$\displaystyle \alpha=0 $$ → ideal brick-wall (not realizable). $$\displaystyle \alpha=1 $$ → full cosine roll-off.

4.2 Line Coding

• Polar Signaling (NRZ-L): 1 = +V, 0 = -V (or 0). No return to zero. DC component present, no inherent synchronization.

• Polar Signaling (NRZ-I): 1 = transition at beginning of bit interval, 0 = no transition. Differential encoding inherent.

• Return-to-Zero (RZ): Pulse returns to zero mid-bit. 1 = +V for first half, 0 for second; 0 = 0 throughout. Has DC component, requires more bandwidth than NRZ.

• Bipolar Signaling (AMI): 1 = alternating +V/-V, 0 = 0. No DC component, error detection (violation: two non-zero pulses of same polarity in a row).

• Manchester Coding: Very High-Frequency.

  • 1 = transition from High to Low in middle of bit.

  • 0 = transition from Low to High in middle of bit.

  • Example: 01101001 → 0↓1↑0↓1↑0↓0↑0↓1↑ (representing transitions).

  • Self-clocking (transition every bit), no DC component. Used in Ethernet (10BASE-T).

4.3 Differential Encoding

• How Achieved: Output depends on current input bit AND previous output bit.

  • Rule: d_k = b_k \oplus d_{k-1} (XOR). d_k = encoded bit, b_k = data bit.

  • Physical Realization: In phase modulation (DPSK), a 1 causes a 180° phase shift relative to previous symbol; 0 causes no phase change.

• Advantages:

  1. Immunity to phase ambiguity (180° ambiguity in carrier recovery doesn't matter).

  2. Simple carrier recovery (transition detection).

• Disadvantages:

  1. Error propagation: One bit error in demodulation causes two bit errors in decoded output (since it flips the reference for next bit).

• Very High-Frequency.

4.4 Other Baseband Concepts

• Eye Pattern: Oscilloscope display of overlayed received signal over multiple bit intervals.

  • Use: Measures system performance.

    • Eye Opening Vertical Height: Noise margin.

    • Eye Opening Horizontal Width: Timing jitter tolerance (ISI).

    • Slope of Eye Edges: Sensitivity to timing errors.

  • DiagramSEARCH: Eye pattern diagram ISI noise jitter

• Crosstalk: Unwanted coupling between adjacent transmission lines (near-end, far-end).

• On-off Signaling: 1 = pulse (on), 0 = no pulse (off). Simple but poor power efficiency and DC component.

5.0 DIGITAL PASSBAND MODULATION SCHEMES

5.1 Binary Modulation

• ASK (Amplitude Shift Keying):

  • 1 = $$\displaystyle A \cos(2\pi f_c t) $$, 0 = 0.

  • Simple but poor noise performance (amplitude sensitive).

• BPSK (Binary Phase Shift Keying):

  • 1 = $$\displaystyle A \cos(2\pi f_c t) $$, 0 = $$\displaystyle A \cos(2\pi f_c t + \pi) = -A \cos(2\pi f_c t) $$.

  • Coherent Detection: Multiply by local carrier $$\displaystyle \cos(2\pi f_c t) $$, integrate over $$\displaystyle T_b $$, compare to threshold.

  • Probability of Bit Error (Coherent, AWGN):

$$P_e = Q\left(\sqrt{\frac{2E_b}{N_0}}\right)$$

    where $$\displaystyle E_b $$ = energy/bit, $$\displaystyle N_0/2 $$ = noise PSD.

    > **\boxed{P_{e,\text{BPSK}} = Q\left(\sqrt{\frac{2E_b}{N_0}}\right)}**

    *   **Very High-Frequency Derivation.**

• DPSK (Differential Phase Shift Keying):

  • Principle: Encode data as phase change between successive symbols. 1 → 180° phase shift, 0 → 0° shift.

  • Detection: Coherently demodulate current and previous symbols, compare phases (multiply and integrate). Avoids need for absolute carrier phase reference.

  • Comparison with BPSK: ~1 dB worse $$\displaystyle P_e $$ at high SNR, but avoids phase ambiguity problem. $$\displaystyle P_{e,\text{DPSK}} \approx \frac{1}{2} e^{-E_b/N_0} $$ for large SNR.

  • Very High-Frequency.

• BFSK (Binary Frequency Shift Keying):

  • 1 = $$\displaystyle f_1 $$, 0 = $$\displaystyle f_2 $$. Orthogonal if $$\displaystyle |f_1 - f_2| = 1/T_b $$.

  • Coherent Detection: Two matched filters/correlators at $$\displaystyle f_1 $$ and $$\displaystyle f_2 $$, choose larger output.

  • Non-coherent Detection: Energy detection (envelope detectors) at each frequency. Simpler but worse performance.

  • $$\displaystyle P_e $$ (Coherent, Orthogonal): $$\displaystyle P_e = Q\left(\sqrt{\frac{E_b}{N_0}}\right) $$ (3 dB worse than BPSK).

  • $$\displaystyle P_e $$ (Non-coherent): $$\displaystyle P_e = \frac{1}{2} e^{-E_b/(2N_0)} $$.

5.2 M-ary Modulation

• M-ary PSK: $M$ symbols equally spaced on a circle of radius $$\displaystyle \sqrt{E_s} $$.

  • Constellation: Phase shifts $$\displaystyle \phi_k = \frac{2\pi k}{M} $$, $$\displaystyle k=0,1,...,M-1 $$.

  • Bandwidth Efficiency: $$\displaystyle \eta = \frac{\log_2 M}{2} $$ bits/s/Hz (for coherent PSK).

  • Trade-off: Increases data rate for same bandwidth but requires higher $$\displaystyle E_b/N_0 $$ for same $$\displaystyle P_e $$.

• M-ary FSK:

  • Orthogonal FSK: Frequencies $$\displaystyle f_k = f_c + k \Delta f $$, with $$\displaystyle \Delta f = 1/T_s $$ ($$\displaystyle T_s $$ = symbol duration). Minimum frequency separation for orthogonality.

  • Non-orthogonal FSK: Closer spacing, more bandwidth efficient but correlated detection needed.

5.3 Quadrature Modulation

• QPSK (Quadrature Phase Shift Keying):

  • Principle: Two independent BPSK streams on orthogonal carriers ($\cos$ and $\sin$). Each stream (I and Q) carries 1 bit/symbol.

  • Generation:

    1. Split data into two streams (odd/odd bits on I, even/even on Q).

    2. BPSK modulate each onto $$\displaystyle \cos(2\pi f_c t) $$ and $$\displaystyle \sin(2\pi f_c t) $$.

    3. Sum: $$\displaystyle s(t) = s_I(t)\cos(2\pi f_c t) - s_Q(t)\sin(2\pi f_c t) $$.

    • DiagramSEARCH: QPSK transmitter block diagram I Q channels

  • Constellation: 4 points at 45°, 135°, 225°, 315° (or 0°, 90°, 180°, 270° with Gray coding).

  • Detection: Coherent receiver with two matched filters (I and Q channels), sample at $$\displaystyle T_b $$, decide based on quadrant.

  • Advantages over BPSK:

    1. Bandwidth Efficiency: 2 bits/symbol vs 1 bit/symbol → half the bandwidth for same bit rate.

    2. Same $$\displaystyle E_b/N_0 $$ requirement as BPSK for same $$\displaystyle P_b $$ (since $$\displaystyle E_s = 2E_b $$ and distance between points is $$\displaystyle \sqrt{2E_s} = \sqrt{4E_b} $$).

  • Very High-Frequency.

• MSK (Minimum Shift Keying):

  • Definition: Special case of continuous-phase FSK (CPFSK) with modulation index $$\displaystyle h = 0.5 $$.

  • Frequency Separation: $$\displaystyle \Delta f = |f_1 - f_0| = \frac{1}{2T_b} $$ (minimum for orthogonality over $$\displaystyle 2T_b $$).

  • Spectral Properties: Very compact spectrum (main lobe narrower than QPSK), no side lobes, constant envelope. Excellent for non-linear channels (satellites).

  • Relation to QPSK: MSK can be viewed as offset QPSK (OQPSK) with sinusoidal pulse shaping. I and Q channels staggered by $$\displaystyle T_b $$ to avoid 180° phase transitions.

  • Spectral Properties of QPSK & MSK: MSK has ~25% narrower main lobe and lower out-of-band sidelobes than QPSK with rectangular pulses.

  • Very High-Frequency.

5.4 Quadrature Amplitude Modulation (QAM)

• Principle: Independent amplitude modulation of I and Q carriers. Combines ASK and PSK.

• Constellation: $$\displaystyle M = M_I \times M_Q $$ points on a rectangular grid.

• Generation/Detection: Similar to QPSK but with multiple amplitude levels on I and Q.

• Probability of Symbol Error (Square M-QAM, $$\displaystyle M=2^{2k} $$):

$$P_s \approx 4 \left(1 - \frac{1}{\sqrt{M}}\right) Q\left(\sqrt{\frac{3}{M-1} \frac{E_s}{N_0}}\right)$$

For Gray coding, $$\displaystyle P_b \approx \frac{P_s}{\log_2 M} $$.

> **\boxed{P_s \approx 4 \left(1 - \frac{1}{\sqrt{M}}\right) Q\left(\sqrt{\frac{3}{M-1} \frac{E_s}{N_0}}\right)}**

*   **High-Frequency Derivation.**

5.5 Comparative Analysis

[!TIP] Key Insight: Bandwidth-Power Trade-off: Higher M-ary schemes (QAM, high-PSK) give higher bandwidth efficiency but require significantly higher $$\displaystyle E_b/N_0 $$ (power) for same error rate.

6.0 RECEIVER STRUCTURES & PERFORMANCE ANALYSIS

6.1 Optimum Receiver Principle

• Correlator Receiver: Correlates received signal $r(t)$ with each possible signal $$\displaystyle s_i(t) $$ over symbol period $$\displaystyle T_s $$.

  • Output: $$\displaystyle r_i = \int_0^{T_s} r(t) s_i(t) \, dt $$.

  • Decide signal with maximum $$\displaystyle r_i $$.

• Matched Filter: Optimum linear filter for maximizing SNR at sampling time $$\displaystyle T_s $$ in AWGN.

  • Impulse Response: $$\displaystyle h(t) = s(T_s - t) $$ for $$\displaystyle 0 \leq t \leq T_s $$, 0 elsewhere. Time-reversed and delayed version of the signal.

  • Output at $$\displaystyle t=T_s $$: $$\displaystyle y(T_s) = \int_0^{T_s} r(t) s(T_s - t) \, dt = \int_0^{T_s} r(t) s(t) \, dt $$ if $s(t)$ is even/energy symmetric. Equivalent to correlator output.

  • High-Frequency: Derive $h(t)$ from $s(t)$.

6.2 Matched Filter Details

• Performance in AWGN: Maximizes $$\displaystyle SNR = \frac{(\int_0^{T_s} s^2(t) dt)^2}{N_0/2} = \frac{2E_s}{N_0} $$.

• Comparison with Correlator: Matched filter is the optimal linear receiver. Correlator is optimal if noise is white and we integrate over $$\displaystyle T_s $$. They are equivalent in performance for AWGN.

6.3 Signal Space Representation & Gram-Schmidt

• Purpose: Represent any set of signals $$\displaystyle \{s_i(t)\} $$ as vectors in an $N$-dimensional Euclidean space. Enables geometric probability of error calculation.

• Gram-Schmidt Orthogonalization Procedure (Very High-Frequency):

  1. Start with first basis vector: $$\displaystyle \phi_1(t) = s_1(t) / \|s_1(t)\| $$.

  2. Project $$\displaystyle s_2(t) $$ onto $$\displaystyle \phi_1(t) $$: $$\displaystyle proj = \langle s_2, \phi_1 \rangle \phi_1 $$.

  3. Subtract projection: $$\displaystyle u_2(t) = s_2(t) - proj $$.

  4. Normalize: $$\displaystyle \phi_2(t) = u_2(t) / \|u_2(t)\| $$.

  5. Repeat for $$\displaystyle s_3, s_4, ... $$ until all signals are expressed.

• Result: Any signal $$\displaystyle s_i(t) = \sum_{k=1}^{N} s_{ik} \phi_k(t) $$, where $$\displaystyle s_{ik} = \langle s_i, \phi_k \rangle $$ are coordinates.

• Use in $$\displaystyle P_e $$: Minimum distance between signal points in this space determines error probability. For equiprobable signals in AWGN, $$\displaystyle P_e \propto Q\left(\frac{d_{\min}}{\sqrt{2N_0}}\right) $$.

[!TIP] Exam Tip: Gram-Schmidt is often asked as "explain the procedure" or "orthogonalize given signals." Write steps clearly with projection formulas.

7.0 CHANNEL CAPACITY & INFORMATION THEORY

7.1 Information Measures

• Entropy $H(X)$ (bits): Average uncertainty of discrete random variable $X$.

$$H(X) = -\sum_{i} p(x_i) \log_2 p(x_i)$$

• Joint Entropy $H(X,Y)$: Uncertainty of pair $(X,Y)$.

• Conditional Entropy $H(Y|X)$: Uncertainty of $Y$ given $X$.

$$H(Y|X) = \sum_i p(x_i) H(Y|X=x_i)$$

• Mutual Information $I(X;Y)$ (bits): Information $X$ conveys about $Y$.

$$I(X;Y) = H(Y) - H(Y|X) = H(X) + H(Y) - H(X,Y)$$

Also: $$\displaystyle I(X;Y) = \sum_{x,y} p(x,y) \log_2 \frac{p(x,y)}{p(x)p(y)} $$.

7.2 Channel Capacity

• Definition: Maximum mutual information over all possible input distributions $p(x)$.

$$C = \max_{p(x)} I(X;Y) \quad \text{(bits per channel use)}$$

• Shannon-Hartley Theorem (AWGN Channel):

  For a continuous-time AWGN channel of bandwidth $B$ Hz and signal-to-noise ratio $S/N$,

$$C = B \log_2 \left(1 + \frac{S}{N}\right) \ \text{bits/sec}$$

> **\boxed{C = B \log_2 \left(1 + \frac{S}{N}\right)}**

*   **Very High-Frequency Derivation:** Starts from discrete-time model with $2B$ samples/sec (Nyquist), capacity per sample $$\displaystyle C_s = \frac{1}{2} \log_2(1+P/N_0) $$ where $$\displaystyle P=S/(2B) $$ is power per sample. Then $$\displaystyle C = 2B \cdot C_s $$.

• Shannon's Information Capacity Theorem: Reliable transmission (arbitrarily low error) is possible if and only if transmission rate $$\displaystyle R < C $$.

• Trade-off: Increasing $C$ requires increasing $B$ or $S/N$. Cannot overcome by clever modulation/coding alone.

7.3 Binary Symmetric Channel (BSC)

• Model: Binary input $X \in \{0,1\}$, binary output $Y \in \{0,1\}$. Crossover probability $P$ (probability 0→1 or 1→0).

• Conditional Probabilities: $$\displaystyle P(Y=1|X=0) = P $$, $$\displaystyle P(Y=0|X=1) = P $$.

• Capacity: Achieved with equiprobable input ($$\displaystyle p(0)=p(1)=0.5 $$).

$$C = 1 - H(P) \ \text{bits/channel use}$$

where $$\displaystyle H(P) = -P \log_2 P - (1-P) \log_2 (1-P) $$ is binary entropy.

> **\boxed{C_{\text{BSC}} = 1 - H(P)}**

• Numerical Example: For $$\displaystyle P=0.9 $$, $$\displaystyle H(0.9)=0.469 $$, $$\displaystyle C=0.531 $$ bits/use.

8.0 ERROR CONTROL CODING

8.1 Introduction

• Role: Add redundancy to transmitted data to enable detection and correction of errors at receiver without retransmission.

• Trade-off: Redundancy reduces effective data rate but improves reliability.

• High-Frequency: Know the basic trade-off diagram (Rate vs. $$\displaystyle E_b/N_0 $$).

8.2 Linear Block Codes

• Structure: $(n, k)$ code: $k$ information bits → $n$ transmitted bits ($n-k$ parity bits). Linear: code words form a vector subspace.

• Generator Matrix $\mathbf{G}$: $k \times n$ matrix. Encoding: $$\displaystyle \mathbf{c} = \mathbf{u} \mathbf{G} $$, where $\mathbf{u}$ is $1 \times k$ info vector.

• Parity-Check Matrix $\mathbf{H}$: $(n-k) \times n$ matrix. Satisfies $$\displaystyle \mathbf{c} \mathbf{H}^T = \mathbf{0} $$ for any code word $\mathbf{c}$.

• Syndrome Decoding:

  1. Compute syndrome: $$\displaystyle \mathbf{s} = \mathbf{r} \mathbf{H}^T $$ ($\mathbf{r}$ = received vector).

  2. If $$\displaystyle \mathbf{s} = \mathbf{0} $$ → assume no error (or undetectable).

  3. If $\mathbf{s} \neq \mathbf{0}$ → lookup in standard array to find error pattern $\mathbf{e}$ closest to $\mathbf{r}$.

  4. Correct: $$\displaystyle \mathbf{c} = \mathbf{r} - \mathbf{e} $$ (mod-2).

• Very High-Frequency: Be able to encode given $\mathbf{G}$ and decode given $\mathbf{H}$ and syndrome table.

• Distance Properties:

  • Hamming Weight $w(\mathbf{c})$: Number of non-zero elements in $\mathbf{c}$.

  • Hamming Distance $$\displaystyle d(\mathbf{c}_i, \mathbf{c}_j) $$: Number of positions where $$\displaystyle \mathbf{c}_i $$ and $$\displaystyle \mathbf{c}_j $$ differ.

  • Minimum Distance $$\displaystyle d_{\min} $$: Smallest distance between any two code words.

  • Error Detection: Can detect up to $$\displaystyle d_{\min}-1 $$ errors.

  • Error Correction: Can correct up to $$\displaystyle t = \left\lfloor \frac{d_{\min}-1}{2} \right\rfloor $$ errors.

8.3 Cyclic Codes

• Algebraic Structure: Code words are multiples of a generator polynomial $g(x)$ of degree $n-k$.

  • Code word polynomial $$\displaystyle c(x) = m(x) g(x) $$, where $m(x)$ is message polynomial of degree $$\displaystyle <k $$.

  • $g(x)$ must divide $$\displaystyle x^n + 1 $$ (for systematic codes).

• Encoding (Systematic):

  1. Multiply $m(x)$ by $$\displaystyle x^{n-k} $$.

  2. Divide by $g(x)$ → remainder $r(x)$.

  3. Code word: $$\displaystyle c(x) = x^{n-k} m(x) + r(x) $$.

• DiagramSEARCH: Cyclic encoder shift register circuit

• Syndrome Decoding: Compute syndrome polynomial $$\displaystyle s(x) = r(x) \mod g(x) $$. If $s(x) \neq 0$, error occurred. Use precomputed table of $s(x)$ → error pattern $e(x)$.

• Very High-Frequency: Understand polynomial representation and shift-register encoder.

8.4 Specific Codes

• Hamming Codes: Single-error correcting (SEC) linear block codes with $$\displaystyle d_{\min}=3 $$. Parameters: $$\displaystyle (2^m - 1, 2^m - m - 1) $$.

  • Example (7,4) Hamming Code:

    • $$\displaystyle n=7, k=4, n-k=3 $$ parity bits.

    • $$\displaystyle \mathbf{G} = \begin{bmatrix} 1 & 0 & 0 & 0 & 1 & 1 & 0 \\ 0 & 1 & 0 & 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 1 & 1 & 1 & 1 \end{bmatrix} $$

    • $$\displaystyle \mathbf{H} = \begin{bmatrix} 1 & 1 & 0 & 1 & 1 & 0 & 0 \\ 1 & 0 & 1 & 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 1 & 0 & 0 & 1 \end{bmatrix} $$ (columns are all non-zero 3-bit binary numbers).

  • Syndrome directly gives error location (if single error).

• Burst Error Correcting Codes: Cyclic codes (like Fire codes) are effective because burst errors often have polynomial representation divisible by $g(x)$.

8.5 Convolutional Codes

• Encoding:

  • Constraint Length $K$: Number of input bits that affect current output bits.

  • Code Rate $$\displaystyle R = k/n $$: $k$ input bits → $n$ output bits per shift.

  • Implemented with shift register of $K$ stages and modulo-2 adders.

  • State: Content of shift register ( $$\displaystyle 2^{K-1} $$ states).

• Graphical Representations:

  • State Diagram: Shows transitions between states with output labels.

  • Tree Diagram: Shows all possible output sequences.

  • Trellis Diagram: Compact state diagram showing time evolution. Used for Viterbi decoding.

• DiagramSEARCH: Convolutional encoder state diagram trellis

• Decoding - Viterbi Algorithm:

  • Concept: Maximum likelihood sequence estimation (MLSE) on trellis.

  • Process: Compute path metric (cumulative Hamming distance) for each state at each time step. Keep only survivor path (path with best metric) for each state. Trace back to get most likely transmitted sequence.

  • High-Frequency: Describe the algorithm steps (add-compare-select). Not required to derive.

[!TIP] Exam Focus: Linear Block Codes (encoding/decoding with G/H) and Convolutional Codes (encoder circuit, state diagram, Viterbi concept) are very high-frequency. Practice numerical problems for Hamming (7,4) code.

END OF UNIT 3 NOTES