DIGITAL COMMUNICATION (EC-502) - Unit 4 Short Notes

UNIT 4: DIGITAL COMMUNICATION

1. SAMPLING & SAMPLING THEOREM

Ideal Sampling (Instantaneous Sampling)

• Definition: Sampling where the signal is multiplied by an impulse train (Dirac comb). The sample value is the signal's instantaneous amplitude at the sampling instant.

• Mathematical Representation:

  Let $$\displaystyle x_a(t) $$ be the analog signal and $$\displaystyle s(t) = \sum_{n=-\infty}^{\infty} \delta(t - nT_s) $$ be the impulse train.

  The sampled signal is:

$$x_s(t) = x_a(t) \cdot s(t) = \sum_{n=-\infty}^{\infty} x_a(nT_s) \delta(t - nT_s)$$

• Frequency Domain: The spectrum $$\displaystyle X_s(f) $$ is a periodic replication of $$\displaystyle X_a(f) $$ with period $$\displaystyle f_s = 1/T_s $$.

$$X_s(f) = f_s \sum_{k=-\infty}^{\infty} X_a(f - kf_s)$$

Natural & Flat-Top (Practical) Sampling

• Natural Sampling: The top of the sample follows the analog signal during the sampling interval. The sample is a rectangular pulse of width $$\displaystyle T_s $$ whose height equals $$\displaystyle x_a(t) $$ at the start of the interval.

• Flat-Top Sampling: The top of the sample is flat (constant). Achieved by passing natural-sampled signal through a sample-and-hold circuit. Has an aperture effect—the effective sampling instant is delayed, causing a sinc-type attenuation of high frequencies.

• Comparison with Ideal: Practical sampling (especially flat-top) introduces a sin(x)/x (sinc) distortion due to the finite pulse width.

Nyquist-Shannon Sampling Theorem (Low-Pass)

• Statement: A bandlimited signal with maximum frequency $$\displaystyle f_m $$ can be perfectly reconstructed from its samples if the sampling frequency $$\displaystyle f_s \geq 2f_m $$.

• Nyquist Rate: $$\displaystyle f_N = 2f_m $$ (minimum required rate).

• Nyquist Interval: $$\displaystyle T_N = 1/f_N = 1/(2f_m) $$ (maximum time between samples).

• Perfect Reconstruction: Using an ideal low-pass filter (sinc interpolation).

Sampling Theorem for Band-Pass Signals

• For a signal with bandwidth $B$ and center frequency $$\displaystyle f_c $$ (where $$\displaystyle f_c >> B $$), the minimum sampling rate can be significantly less than $$\displaystyle 2f_c $$.

• Condition: $$\displaystyle f_s \geq 2B $$ (approximately), but exact condition depends on the signal's spectral structure. Allows undersampling (harmonic sampling).

Aliasing

• Definition: Overlapping of spectral replicas in the sampled signal's spectrum due to $$\displaystyle f_s < 2f_m $$.

• Cause: Inadequate sampling rate.

• Effect: High-frequency components fold back into the baseband, causing irreversible distortion in the reconstructed signal.

• Anti-aliasing Filter: A low-pass filter (with cutoff $$\displaystyle f_m $$) placed before the sampler to bandlimit the signal and prevent aliasing.

Time Division Multiplexing (TDM)

• Principle: Multiple signals share the same channel by taking turns in time. Each signal is sampled at its Nyquist rate, and samples are interleaved into a single high-rate composite signal.

• Relationship: The composite TDM bit rate must be at least the sum of the Nyquist rates of all individual channels.

[!TIP] Exam Focus: Be prepared to derive the expression for the sampled signal spectrum from the impulse train multiplication. Distinguish clearly between ideal, natural, and flat-top sampling waveforms and their spectral implications. Aliasing is a very common 7-mark question.

2. PULSE MODULATION

Pulse Amplitude Modulation (PAM)

• Generation: Sample the analog signal and hold the sample value constant for the duration of a pulse. The pulse amplitude is proportional to the sample value.

• Waveform: A train of rectangular pulses with varying amplitudes.

• Spectral Property: Bandwidth is approximately twice the maximum frequency of the original signal (similar to baseband transmission of samples).

Pulse Width Modulation (PWM)

• Generation:

  1. Integrate the analog signal.

  2. Compare the integrator output with a sawtooth (triangular) carrier.

  3. Output a pulse whose width is proportional to the instantaneous amplitude of the input signal.

• Detection: Pass the PWM signal through a low-pass filter (integrator). The average voltage over each bit period is proportional to the pulse width.

• Key Point: The amplitude of the PWM pulses is constant; only the width/duty cycle varies.

Pulse Position Modulation (PPM)

• Generation:

  1. Generate a timing pulse at the start of each bit period.

  2. Displace this timing pulse by an amount proportional to the amplitude of the input sample. The displaced pulse is the PPM signal.

• Detection: Use a correlator or a narrowband filter followed by an envelope detector, synchronized to the expected pulse positions.

Comparison of PAM, PWM, PPM

[!TIP] Exam Focus: Generation and detection block diagrams for PWM and PPM are frequently asked. Be able to sketch them and explain the role of the integrator and comparator. The comparison table is a high-frequency 5-7 mark question.

3. QUANTIZATION & PULSE CODE MODULATION (PCM)

Quantization

• Need: To convert continuous-amplitude PAM samples into discrete amplitude levels for digital transmission.

• Process: The amplitude range $$\displaystyle [V_{min}, V_{max}] $$ is divided into $L$ equal intervals (for uniform quantization). Each sample is approximated to the midpoint of its corresponding interval.

• Quantization Levels (L) & Step Size (Δ):

$$\Delta = \frac{V_{max} - V_{min}}{L}$$

For $n$ bits, $$\displaystyle L = 2^n $$.

• Example: Signal range -5V to +5V, 8-bit quantization.

$$V_{range} = 10V, \quad n=8 \Rightarrow L=256, \quad \Delta = \frac{10}{256} \approx 0.0391V$$

Quantization Noise & SQNR

• Quantization Noise: Error $$\displaystyle e_q(t) = x(t) - \hat{x}(t) $$ modeled as uniform noise over $[-\Delta/2, \Delta/2]$.

• Noise Power (for uniform quantizer, high-resolution):

$$P_q = \frac{\Delta^2}{12}$$

• Signal Power (for a full-scale sinusoid, peak amplitude $$\displaystyle A = (V_{max}-V_{min})/2 $$):

$$P_s = \frac{A^2}{2} = \frac{(L\Delta/2)^2}{2} = \frac{L^2 \Delta^2}{8}$$

• Signal-to-Quantization-Noise Ratio (SQNR):

$$SQNR = \frac{P_s}{P_q} = \frac{L^2 \Delta^2 / 8}{\Delta^2 / 12} = \frac{3}{2} L^2 = \frac{3}{2} \cdot 2^{2n}$$

In dB:

$$SQNR_{dB} \approx 10\log_{10}\left(\frac{3}{2} \cdot 2^{2n}\right) \approx (6.02n + 1.76) \text{ dB}$$

\boxed{SQNR_{dB} \approx 6.02n + 1.76 \text{ dB}}

Companding

• Concept: Compress signal dynamic range at transmitter (before quantization) using a non-uniform law, then expand it at receiver (after quantization) with the inverse law.

• Purpose: To improve SQNR for low-amplitude signals without increasing the number of bits $n$.

• μ-law (US/Japan): $$\displaystyle y = \frac{\ln(1+\mu|x|)}{\ln(1+\mu)} \cdot \text{sign}(x) $$, $$\displaystyle \mu=255 $$.

• A-law (Europe): Piecewise linear with $$\displaystyle A=87.6 $$.

• Result: More quantization levels (smaller Δ) for small signals, fewer levels for large signals.

Pulse Code Modulation (PCM) System

• Block Diagram:

  Analog Signal → Anti-aliasing Filter → Sampler → Quantizer → Encoder → Binary Channel → Decoder → Reconstruction Filter → Reconstructed Signal

• Bit Rate ($$\displaystyle R_b $$):

$$R_b = f_s \times n \text{ (bits/second)}$$

where $$\displaystyle f_s $$ = sampling rate, $n$ = bits per sample.

• Standard: Telephony uses $$\displaystyle f_s = 8000 $$ Hz, $$\displaystyle n=8 $$ bits → $$\displaystyle R_b = 64 $$ kbps.

Delta Modulation (DM) & Adaptive DM (ADM)

• DM Principle: Transmits only the change (1 bit per sample): +Δ if signal increases, -Δ if decreases.

• Generation: Compare input $x(t)$ with a locally generated staircase approximation $\hat{x}(t)$. Output 1 if $$\displaystyle x(t) > \hat{x}(t) $$, else 0.

• Detection: Integrate the 1/0 bit stream to reconstruct $\hat{x}(t)$.

• Distortions:

  • Granular Noise: Small step size $\Delta$ causes noise when signal is nearly constant.

  • Slope Overload: Large step size $\Delta$ cannot track rapid signal changes.

• ADM: The step size $\Delta$ is adapted based on recent signal changes to balance both distortions.

[!TIP] Exam Focus: Derivation of quantization noise power and SQNR formula is VERY HIGH FREQUENCY. Know the PCM block diagram and bit rate formula. For DM, clearly distinguish between granular noise and slope overload, and explain how ADM mitigates them.

4. BASEBAND SIGNALING & LINE CODING

Line Encoding (Waveform Coding)

• Purpose: Convert binary data (0s, 1s) into a baseband waveform suitable for the physical channel (e.g., wire, optical fiber).

• Key Properties Considered: DC component, bandwidth, synchronization (clock recovery), error detection capability.

Common Line Codes

Differential Encoding

• Principle: Encode based on transition (change) or no transition relative to the previous encoded bit.

$$d_n = b_n \oplus d_{n-1}$$

where $$\displaystyle b_n $$ is input bit, $$\displaystyle d_n $$ is output.

• Advantages:

  1. Polarity Insensitivity: A wire reversal (inverting all polarities) does not change the decoded data.

  2. Useful in phase-coherent systems where absolute phase is ambiguous (e.g., DPSK).

• Disadvantages:

  1. Error Propagation: A single bit error causes two decoded bit errors (current and next).

  2. More complex decoding.

Inter-Symbol Interference (ISI)

• Definition: Overlap of successive pulses in time, causing the received symbol to be corrupted by previous/next symbols.

• Cause: Bandwidth limitation of the channel or multipath propagation.

• Nyquist Criterion for Zero ISI:

  The overall system response $h(t)$ must satisfy:

$$h(kT_s) = \begin{cases} 1, & k=0 \\ 0, & k \neq 0 \end{cases}$$

where $$\displaystyle T_s $$ is the symbol period.

In frequency domain, this requires $H(f)$ to have a **sinc** shape or its periodic replications to sum to a constant.

• Minimization: Use pulse shaping (e.g., raised cosine filter) at transmitter and/or equalization at receiver.

Eye Pattern

• Concept: An oscilloscope display where the received signal is sampled over multiple bit periods and superimposed.

• Interpretation:

  • Eye Opening Height: Noise margin (vertical).

  • Eye Opening Width: Timing jitter tolerance (horizontal).

  • Slope of Eye Edges: Susceptibility to ISI (steeper slopes = less ISI).

  • Eye Closure: Indicates distortion, noise, or insufficient bandwidth.

  • Best Sampling Instant: Center of the eye opening.

[!TIP] Exam Focus: Manchester encoding (convert a given binary string) and advantages/disadvantages of differential encoding are VERY HIGH FREQUENCY. RZ signaling waveform and comparison with NRZ is also frequent. Be able to define ISI and state the Nyquist criterion. Eye pattern interpretation is a standard question.

5. DIGITAL MODULATION TECHNIQUES (Passband)

Binary Modulation

Quadrature Phase Shift Keying (QPSK)

• Principle: 2 bits per symbol ($$\displaystyle b_1, b_2 $$). 4 phase states: $\pm \pi/4, \pm 3\pi/4$ (or 0°, 90°, 180°, 270°).

• Generation (Offset QPSK):

  • Split data into I (in-phase) and Q (quadrature) streams (even/odd bits).

  • Modulate each onto orthogonal carriers $$\displaystyle \cos(\omega_c t) $$ and $$\displaystyle \sin(\omega_c t) $$ using BPSK modulators.

  • Sum the two BPSK signals.

$$s(t) = I(t)\sqrt{\frac{2E_s}{T_s}}\cos(\omega_c t) - Q(t)\sqrt{\frac{2E_s}{T_s}}\sin(\omega_c t)$$

• Detection: Coherent receiver with I and Q channels. Correlate with local $\cos$ and $\sin$ carriers, sample, and decide.

• Advantages over BPSK:

  • Same bandwidth as BPSK ($$\displaystyle \approx 2/T_s $$).

  • Double the bit rate (2 bits/symbol) → Bandwidth efficiency $$\displaystyle \eta = 2 $$ bits/s/Hz.

• Probability of Error (Coherent QPSK):

$$P_e(QPSK) = 2Q\left(\sqrt{\frac{2E_b}{N_0}}\right) - Q^2\left(\sqrt{\frac{2E_b}{N_0}}\right) \approx Q\left(\sqrt{\frac{2E_b}{N_0}}\right)$$

Same as **BPSK** for high SNR.

Differential Phase Shift Keying (DPSK)

• Principle: Encode based on phase change relative to previous symbol.

  • 0 → No phase change ($$\displaystyle \Delta\phi=0 $$)

  • 1 → Phase change ($$\displaystyle \Delta\phi=\pi $$ for DBPSK)

• Generation: Differential encoder (XOR) output drives a BPSK modulator.

• Detection (Delay & Subtract):

  1. Multiply received signal with delayed version (by $$\displaystyle T_s $$).

  2. Integrate over $$\displaystyle T_s $$.

  3. Decide sign.

• Comparison with Coherent PSK:

  • Adv: No carrier phase synchronization needed.

  • Dis: ~1 dB performance degradation (higher $$\displaystyle P_e $$) at high SNR.

M-ary Modulation

Minimum Shift Keying (MSK)

• Definition: Special case of Continuous-Phase FSK (CPFSK) with modulation index $$\displaystyle h = 0.5 $$.

$$f_{dev} = \pm \frac{1}{4T_b} \quad (\text{orthogonal})$$

• Generation: Offset QPSK with sinusoidal pulse shaping (half-sinusoid) on I and Q channels. Ensures phase continuity.

• Spectral Properties: Compact spectrum with low sidelobes (roll-off ~20 dB/decade). More bandwidth-efficient than QPSK.

• Comparison: Better spectral efficiency than QPSK, similar error performance, constant envelope (good for nonlinear amplifiers).

Quadrature Amplitude Modulation (QAM)

• Principle: Two orthogonal carriers ($\cos$, $\sin$) modulated by two independent PAM signals (I and Q).

• Constellation: Square grid (e.g., 16-QAM has 16 points).

• Generation/Detection: Same as QPSK, but I and Q channels have $M/2$-PAM levels.

• Probability of Error (Square M-QAM, high SNR):

$$P_s \approx 4\left(1 - \frac{1}{\sqrt{M}}\right) Q\left(\sqrt{\frac{3}{M-1} \cdot \frac{E_b}{N_0}}\right)$$

$$P_b \approx \frac{P_s}{\log_2 M}$$

Comparison of Digital Modulation Schemes

[!TIP] Exam Focus: QPSK generation/detection block diagram and advantages over BPSK are VERY HIGH FREQUENCY. Spectral properties of QPSK and MSK are also frequent. DPSK principle and performance is standard. Be able to compare schemes using the table above. Derivation of $$\displaystyle P_e $$ for BPSK, QPSK, and QAM is critical.

6. SIGNAL SPACE & PROBABILITY OF ERROR CALCULATION

Gram-Schmidt Orthogonalization

• Purpose: To represent any set of $M$ linearly independent signal waveforms $$\displaystyle \{s_1(t), s_2(t), ..., s_M(t)\} $$ in an orthogonal basis $$\displaystyle \{\phi_1(t), \phi_2(t), ..., \phi_M(t)\} $$.

• Procedure (Step-by-Step):

  1. $$\displaystyle \phi_1(t) = s_1(t) $$

  2. $$\displaystyle \phi_2(t) = s_2(t) - \frac{\langle s_2, \phi_1 \rangle}{\|\phi_1\|^2} \phi_1(t) $$

  3. $$\displaystyle \phi_3(t) = s_3(t) - \frac{\langle s_3, \phi_1 \rangle}{\|\phi_1\|^2} \phi_1(t) - \frac{\langle s_3, \phi_2 \rangle}{\|\phi_2\|^2} \phi_2(t) $$

  4. Continue...

  where $$\displaystyle \langle f,g \rangle = \int_0^T f(t)g(t) dt $$ is the inner product, and $$\displaystyle \|f\|^2 = \langle f,f \rangle $$.

• Result: Any signal $$\displaystyle s_i(t) $$ can be written as:

$$s_i(t) = \sum_{j=1}^{M} s_{ij} \phi_j(t)$$

where $$\displaystyle s_{ij} = \langle s_i, \phi_j \rangle $$ are the **signal coordinates**.

• Signal Vector: $$\displaystyle \mathbf{s}_i = (s_{i1}, s_{i2}, ..., s_{iM}) $$.

Signal Space Representation

• Concept: Each signal waveform $$\displaystyle s_i(t) $$ is represented by its M-dimensional vector $$\displaystyle \mathbf{s}_i $$ in the orthogonal basis.

• Euclidean Distance: The distance between two signal vectors $$\displaystyle \mathbf{s}_i $$ and $$\displaystyle \mathbf{s}_j $$ is:

$$d_{ij} = \|\mathbf{s}_i - \mathbf{s}_j\| = \sqrt{\sum_{k=1}^{M} (s_{ik} - s_{jk})^2}$$

This distance determines the error probability in AWGN.

• Use: For any modulation scheme, find the orthogonal basis via Gram-Schmidt, compute signal vectors, and use the minimum distance $$\displaystyle d_{min} $$ between constellation points to calculate $$\displaystyle P_e $$.

Probability of Error Calculation (General Approach)

1. Find Orthogonal Basis: Apply Gram-Schmidt to the set of transmitted signals.

2. Compute Signal Vectors: Find coordinates $$\displaystyle \mathbf{s}_i $$ for each signal.

3. Received Vector: In AWGN with two-sided PSD $$\displaystyle N_0/2 $$, the received vector is $$\displaystyle \mathbf{r} = \mathbf{s}_i + \mathbf{n} $$, where $\mathbf{n}$ is an M-dimensional Gaussian vector with independent components, each $$\displaystyle \sim \mathcal{N}(0, N_0/2) $$.

4. Decision Rule: Maximum Likelihood (ML) = Minimum Euclidean distance. The decision region for $$\displaystyle \mathbf{s}_i $$ is the set of points closer to $$\displaystyle \mathbf{s}_i $$ than to any other $$\displaystyle \mathbf{s}_j $$.

5. Compute $$\displaystyle P_e $$: For coherent detection, $$\displaystyle P_e $$ depends on the pairwise error probability $$\displaystyle P(\mathbf{s}_i \to \mathbf{s}_j) $$ and the geometry of decision boundaries. For symmetric constellations:

$$P_b \approx \frac{\text{Number of nearest neighbors}}{ \log_2 M } \cdot Q\left( \frac{d_{min}}{\sqrt{2N_0}} \right)$$

Specific Results (Derived via Signal Space)

• BPSK: Signals: $$\displaystyle s_1(t) = +\sqrt{E_b}\phi(t) $$, $$\displaystyle s_2(t) = -\sqrt{E_b}\phi(t) $$. $$\displaystyle d_{min} = 2\sqrt{E_b} $$.

  \boxed{P_e(BPSK) = Q\left(\sqrt{\frac{2E_b}{N_0}}\right)}

• QPSK: Signals: 4 points at $$\displaystyle (\pm\sqrt{E_b/2}, \pm\sqrt{E_b/2}) $$. $$\displaystyle d_{min} = \sqrt{2E_b} $$.

  \boxed{P_e(QPSK) \approx Q\left(\sqrt{\frac{2E_b}{N_0}}\right) \text{ (same as BPSK)}}

• Square M-QAM: $$\displaystyle d_{min} = 2\sqrt{\frac{E_b}{\log_2 M} \cdot \frac{2}{M-1}} $$ for average energy $$\displaystyle E_b $$.

  \boxed{P_e(M\text{-}QAM) \approx 4\left(1 - \frac{1}{\sqrt{M}}\right) Q\left(\sqrt{\frac{3}{M-1} \cdot \frac{E_b}{N_0}}\right)}

• M-ary Orthogonal (e.g., M-FSK): $$\displaystyle d_{min}^2 = 2E_b $$.

  \boxed{P_e(M\text{-}orthogonal) = (M-1)Q\left(\sqrt{\frac{E_b}{N_0}}\right)}

[!TIP] Exam Focus: Gram-Schmidt procedure (step-by-step for a given set of signals) and signal space representation for BPSK, QPSK, and QAM to derive their $$\displaystyle P_e $$ are VERY HIGH FREQUENCY. You must be able to draw the signal vectors in 2D for QPSK and QAM, identify $$\displaystyle d_{min} $$, and plug into the Q-function formula.

7. RECEIVER STRUCTURES & OPTIMAL DETECTION

Matched Filter

• Principle: The optimal linear filter for maximizing the output SNR at a specific sampling time in the presence of AWGN.

• Impulse Response: Time-reversed version of the transmitted signal shape $s(t)$.

$$h_{MF}(t) = s(T - t), \quad 0 \le t \le T$$

• Output: At $$\displaystyle t=T $$, the output is the correlation between $s(t)$ and the received signal $r(t)$.

$$y(T) = \int_0^T r(t) s(T-t) dt = \int_0^T r(t) s(t) dt \quad \text{(if $s(t)$ is even)}$$

• Block Diagram: Received Signal r(t) → Matched Filter → Sampler at t=T → Decision Device.

• Optimality: Provides the maximum possible SNR among all linear filters.

Correlator Detector

• Principle: Directly computes the correlation between $r(t)$ and a locally generated template $s(t)$ over the symbol period $T$.

$$y = \int_0^T r(t) s(t) dt$$

• Equivalence to Matched Filter: For a signal $s(t)$ that is not time-limited to $[0,T]$, the matched filter output at $$\displaystyle t=T $$ is identical to the correlator output. They are dual implementations.

• Comparison:

  • Matched Filter: Implemented as an analog filter (LTI system).

  • Correlator: Implemented as a multiplier and integrator (or sampler and sum for discrete-time).

  • Both are optimal for AWGN.

Coherent vs. Non-Coherent Detection

• Coherent Detection: Requires precise knowledge of the carrier phase and frequency at the receiver.

  • Examples: Coherent BPSK, QPSK, Coherent BFSK.

  • Performance: Better (lower $$\displaystyle P_e $$) for same $$\displaystyle E_b/N_0 $$.

  • Complexity: Higher (needs carrier synchronization circuit: PLL, Costas loop).

• Non-Coherent Detection: Does not require carrier phase reference.

  • Examples: Non-coherent BFSK (envelope detection), DPSK (delay & subtract), OOK (envelope detection).

  • Performance: Worse (higher $$\displaystyle P_e $$) due to phase ambiguity.

  • Complexity: Lower (simpler receiver).

[!TIP] Exam Focus: Matched filter impulse response derivation and block diagram are standard. Explain why it maximizes SNR. The equivalence between matched filter and correlator is important. Coherent vs. Non-coherent comparison (with examples and performance trade-off) is frequently asked.

8. INFORMATION THEORY & CHANNEL CAPACITY

Entropy, Information, Information Rate

• Entropy $H(X)$: Average uncertainty/information content of a discrete source $X$ with probabilities $$\displaystyle p(x_i) $$.

$$H(X) = -\sum_{i} p(x_i) \log_2 p(x_i) \text{ (bits/symbol)}$$

Maximum for uniform distribution.

• Information Content: Self-information of an outcome $$\displaystyle x_i $$:

$$I(x_i) = -\log_2 p(x_i) \text{ bits}$$

• Information Rate $R$: Average information per unit time.

$$R = H(X) \times \text{symbol rate} \quad \text{(bits/sec)}$$

Shannon-Hartley Theorem (AWGN Channel Capacity)

• Statement: The maximum error-free transmission rate (channel capacity $C$) for a bandlimited AWGN channel with bandwidth $B$ Hz and signal-to-noise ratio $S/N$ is:

  \boxed{C = B \log_2\left(1 + \frac{S}{N}\right) \text{ bits/sec}}

• Implications:

  1. $C$ increases with bandwidth $B$ and SNR $S/N$.

  2. Trade-off: For a fixed $C$, can trade bandwidth for power (and vice versa).

  3. Shannon Limit: To transmit reliably, we need $R \le C$. The minimum required $$\displaystyle E_b/N_0 $$ for a given bandwidth efficiency $$\displaystyle \eta = R/B $$ is:

$$\left(\frac{E_b}{N_0}\right)_{min} = \frac{2^{\eta} - 1}{\eta}$$

    As $\eta \to 0$, $$\displaystyle (E_b/N_0)_{min} \to \ln 2 \approx -1.59 $$ dB.

Shannon's Theorem (Discrete Memoryless Channel)

• Statement: Channel capacity $C$ is the maximum mutual information $I(X;Y)$ over all input distributions $p(x)$.

$$C = \max_{p(x)} I(X;Y) \text{ bits/channel use}$$

• Mutual Information $I(X;Y)$: Measures information gained about $X$ from observing $Y$.

$$I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)$$

• Binary Symmetric Channel (BSC): Crossover probability $P$ (probability 0→1 or 1→0).

  • Capacity:

    \boxed{C = 1 - H_b(P) \text{ bits/channel use}}

    where $$\displaystyle H_b(P) = -P\log_2 P - (1-P)\log_2(1-P) $$ is binary entropy.

  • Example: $$\displaystyle P=0.1 \Rightarrow C = 1 - H_b(0.1) \approx 0.531 $$ bits/use.

Information Capacity Theorem

• Relates bandwidth efficiency $$\displaystyle \eta = R/B $$ to the minimum required $$\displaystyle E_b/N_0 $$ for reliable communication:

$$\frac{E_b}{N_0} \ge \frac{2^{\eta} - 1}{\eta}$$

• This is the fundamental limit for any modulation/coding scheme over an AWGN channel.

• For very large bandwidth ($\eta \to 0$), the limit approaches $\ln 2 \approx -1.59$ dB.

[!TIP] Exam Focus: Shannon-Hartley theorem statement, formula, and implications are VERY HIGH FREQUENCY. Be able to calculate channel capacity for a BSC given $P$. Understand the trade-off between bandwidth and SNR. The information capacity theorem and the -1.59 dB limit are important theoretical concepts.

9. ERROR CONTROL CODING (ECC)

Need & Applications

• Need: To combat channel noise, detect and correct errors without retransmission, improving reliability.

• Trade-off: Adds redundancy (extra bits) → reduces effective data rate or requires more bandwidth/power.

• Applications: Deep space (Voyager), wireless (GSM, 4G/5G), storage (CD, DVD, SSD), digital TV (DVB), satellite.

Linear Block Codes

• Definition: $(n, k)$ code: $k$ information bits $\to$ $n$ coded bits ($$\displaystyle n>k $$). Linear: $$\displaystyle \mathbf{c}_1 \oplus \mathbf{c}_2 $$ is also a valid codeword.

• Generator Matrix $\mathbf{G}$: $k \times n$ matrix. Encoding: $$\displaystyle \mathbf{c} = \mathbf{u} \mathbf{G} $$, where $\mathbf{u}$ is $1 \times k$ message vector.

  $\mathbf{G}$ is often in systematic form: $$\displaystyle \mathbf{G} = [\mathbf{I}_k | \mathbf{P}] $$.

• Parity-Check Matrix $\mathbf{H}$: $(n-k) \times n$ matrix. Satisfies $$\displaystyle \mathbf{c} \mathbf{H}^T = \mathbf{0} $$ for any valid codeword $\mathbf{c}$.

  For systematic $\mathbf{G}$, $$\displaystyle \mathbf{H} = [-\mathbf{P}^T | \mathbf{I}_{n-k}] $$.

• Syndrome: $$\displaystyle \mathbf{S} = \mathbf{r} \mathbf{H}^T $$. $$\displaystyle \mathbf{S}=\mathbf{0} $$ suggests no error (or undetectable). $\mathbf{S} \neq \mathbf{0}$ indicates error pattern.

• Error Capability: Can correct up to $t$ errors where:

$$t = \left\lfloor \frac{d_{min} - 1}{2} \right\rfloor$$

$$\displaystyle d_{min} $$ = minimum Hamming distance between any two codewords.

• Error Detection: Can detect up to $$\displaystyle d_{min}-1 $$ errors.

Cyclic Codes

• Subclass of linear block codes with cyclic shift property: If $\mathbf{c}$ is a codeword, any cyclic shift of $\mathbf{c}$ is also a codeword.

• Generator Polynomial $g(x)$: A degree $n-k$ polynomial that divides $$\displaystyle x^n + 1 $$. All code polynomials $c(x)$ are multiples of $g(x)$.

$$c(x) = m(x) g(x)$$

where $m(x)$ is the message polynomial of degree $$\displaystyle <k $$.

• Encoding: Multiply $m(x)$ by $$\displaystyle x^{n-k} $$ (append zeros), divide by $g(x)$. Remainder is the parity. Code polynomial = dividend.

• Syndrome: $$\displaystyle S(x) = r(x) \mod g(x) $$ (remainder of division). Same as linear block code syndrome.

• Implementation: Simple shift register (divider circuit) for both encoder and syndrome calculator.

Hamming Codes

• Perfect single-error-correcting (SEC) codes.

• Parameters: For $(n, k)$ Hamming code, $$\displaystyle n = 2^r - 1 $$, $$\displaystyle k = n - r $$, where $r$ = number of parity bits.

• Example $(7,4)$ Hamming Code:

  • $$\displaystyle n=7 $$, $$\displaystyle k=4 $$, $$\displaystyle r=3 $$ parity bits.

  • $$\displaystyle d_{min}=3 $$ → $$\displaystyle t = \lfloor (3-1)/2 \rfloor = 1 $$ error-correcting capability.

  • Can correct 1 bit error per 7-bit codeword.

Convolutional Codes

• Concept: Encoding where current output bits depend on current AND previous input bits (memory).

• Parameters:

  • Constraint length $K$: Number of input bits that affect the current output.

  • Code rate $k/n$: $k$ input bits per $n$ output bits per time unit.

• Representation:

  • State Diagram: $$\displaystyle 2^{K-1} $$ states (contents of shift register).

  • Tree Diagram: Shows all possible output sequences.

  • Trellis Diagram: Compact representation of state transitions over time (most useful for decoding).

• Encoding: Shift register with $K$ stages, modulo-2 adders connecting inputs to outputs.

• Viterbi Algorithm (VA):

  • Maximum Likelihood Sequence Estimation (MLSE) decoder.

  • Works on trellis diagram.

  • Principle: At each time step, compute path metric (e.g., Hamming distance) for all state transitions, survivor path for each state, and traceback to find most likely sequence.

  • Complexity: Grows exponentially with $K$, but manageable for small $K$ (e.g., $K \le 9$).

Burst Error Correcting Codes

• Burst Error: A contiguous sequence of erroneous bits (length $b$).

• Fire Codes: A class of cyclic codes designed for burst error correction. Can correct any burst of length $\le b$, where $b$ is a design parameter.

• Encoder/Decoder: Uses generator polynomial $g(x)$ that is product of a primitive polynomial and a polynomial that ensures burst-error correction capability. Decoding often involves error trapping (shifting syndrome register until it becomes zero).

[!TIP] Exam Focus: Linear block codes (G, H matrices, syndrome calculation) and cyclic codes (generator polynomial, encoding circuit) are VERY HIGH FREQUENCY. Convolutional codes (state/trellis diagram, Viterbi principle) are also common. Be able to calculate parameters for a Hamming code. Burst error codes (Fire codes) concept is standard. Always relate $$\displaystyle d_{min} $$ to error correction capability $t$.

END OF UNIT 4 NOTES