Information Theory and Coding (EC-702 (B)) - Important Questions

A zero memory source has a source alphabet. $S = \{s_1, s_2, s_3\}$ with $P = \{0.5, 0.3, 0.2\}$. Find the entropy of the source. Find the entropy of its second extension and verify.

Define uncertainty, information and entropy and discuss its properties, including that entropy is maximum when all messages are equiprobable

The international Morse code uses a sequence of dots and dashes to transmit letters of the English alphabet. The dash is represented by a current pulse that has a duration of 3 units and the dot has a duration of 1 unit. The probability of occurrence of a dash is 1 of the probability of occurrence of a dot.

Prove that $I(X;Y) = H(X)+H(Y)-H(X,Y)$.

Consider a message ensembles $S = \{s_1, s_2, s_3, s_4, s_5, s_6, s_7\}$ with probabilities $P = \{0.45, 0.15, 0.12, 0.08, 0.08, 0.08, 0.04\}$, Construct a binary code and determine its efficiency using Shannon-Fano coding procedure.

Construct the Huffman code with minimum code variance for the following probabilities and also determine the code variance and code efficiency: {0.25, 0.25, 0.125, 0.125, 0.125, 0.0625, 0.0625}.

Discuss about arithmetic coding with suitable example.

Derive the expression for channel capacity when bandwidth tends to infinity from the channel capacity theorem and explain it.

State and prove Shannon Hartley Theorem.

Derive the channel capacity of the Binary Erasure Channel (BEC).

For a BSC with given crossover probability and a given set of equiprobable transmitted codewords, find the input probabilities $P(0), P(1)$, the efficiency of the code, and the channel capacity.

Explain the decoder for cyclic codes with the help of a block diagram and error correction procedure.

What is the significance of the syndromes of a (6, 3) systematic cyclic code? Give a complete list codes and syndromes.

Discuss the procedure for generation of a systematic cyclic code. Draw and explain the systematic cyclic encoder circuit for a (15, 9) cyclic code with generator polynomial $g(X) = 1 + X^3 + X^4 + X^5 + X^6$.

A channel encoder uses a (7, 4) linear systematic cyclic code in the systematic form, generator polynomial being $X^3 + X + 1$. Determine the correct codeword transmitted if the received word is -

What is a perfect code? Explain the features of the (7,4) Hamming code.

Explain the structure of Block Codes and Systematic Block Codes.

Define parity check polynomial and explain with suitable examples.

Define the minimum distance of a code and explain its importance in error detection and correction.

Give / design an example Hamming code with message length $k=4$ and $d_{min}=3$, give its generator matrix and parity-check matrix and comment on its error correction capability.

Find all code vectors of a (6, 3) block code from the given generator matrix $G$.

For a given (7,4) cyclic code with generator polynomial $g(X) = X^3 + X + 1$, draw encoder and syndrome calculator block diagrams and find generator and parity-check matrices in systematic form.

Explain maximum likelihood decoding of convolutional codes using the Viterbi algorithm.

Show how a 4 bit stage shift register can generate a convolution code for a input train 11001.

Explain the working of (2,1,3) Convolutional encoder using transform domain approach.

Explain BCH codes in details with taking a suitable example.

Write short notes on (any four) :

Write a short notes on any two: