AL-503-B Deep Learning - Nov 2022 Question Paper

Under each question, How to answer lists the points an answer should cover, outlined by CampusPrep from the syllabus. Check the details against your textbook.

1. 7 Marks
   aDiscuss the GPU implementation of randomized SVD and write its applications.
   How to answer

   • Outline randomized SVD algorithm steps
   • Explain GPU parallelization / implementation aspects
   • List applications e.g. dimensionality reduction, recommender, image processing
2. 7 Marks
   bWhat is the representation power of a multilayer network of sigmoid neurons? Explain in detail.
   How to answer

   • Define multilayer network of sigmoid neurons
   • State universal approximation / representation result
   • Explain proof idea and significance / limitations
3. 7 Marks
   aWhat is batch normalization? Explain how does it work and also write its advantages.
   How to answer

   • Define batch normalization
   • Explain normalization steps during training / inference
   • List advantages e.g. faster training, stability, regularization
4. 7 Marks
   bWhen should auto encoders be used instead of PCA/SVD for dimensionality reduction? Justify.
   How to answer

   • Define PCA/SVD vs autoencoder for dimensionality reduction
   • Compare linear vs non-linear, data size, complexity
   • Justify when autoencoders preferred
5. 7 Marks
   aExplain in brief sparse auto-encoder and contractive auto-encoder.
   How to answer

   • Define sparse auto-encoder and sparsity penalty
   • Define contractive auto-encoder and penalty on Jacobian
   • Compare purpose and effect
6. 7 Marks
   bWrite a short note on regularization of auto encoders.
   How to answer

   • Define need for regularization in autoencoders
   • List methods: sparse, contractive, denoising, weight decay
   • Explain effect on overfitting and features
7. 7 Marks
   aDiscuss in detail about ReLu activation function.
   How to answer

   • Define ReLU formula $f(x)=\max(0,x)$
   • Explain graph, derivative and non-saturation
   • Discuss advantages and issues like dying ReLU and variants
8. 7 Marks
   bIllustrate the CNN terms Padding and Pooling.
   How to answer

   • Define padding: types and purpose to preserve size
   • Define pooling: max/average and downsampling
   • Illustrate with example feature-map dimensions
9. 7 Marks
   aWhat is unit pruning? Discuss the need of unit pruning in deep learning.
   How to answer

   • Define unit pruning / network pruning
   • Explain need: compression, speed, reduce overfitting
   • Outline pruning criteria and fine-tuning steps
10. 7 Marks
    bExplain briefly about directed graphical model.
    How to answer

    • Define directed graphical model / Bayesian network
    • Explain DAG, conditional factorization
    • Give example of dependencies
11. 7 Marks
    aExplain briefly deep recurrent neural networks and its architecture with a neat diagram.
    How to answer

    • Define deep recurrent neural network and unfolding in time
    • Describe stacked recurrent layers architecture
    • Provide neat labelled diagram of deep RNN
12. 7 Marks
    bDiscuss in detail Long Short Term Memory (LSTM) and write the advantages of LSTM.
    How to answer

    • Describe LSTM cell, gates and cell state
    • Explain how it solves vanishing gradient
    • List advantages for long sequences
13. 7 Marks
    aExplain briefly about the following auto regressive models. NADE
    How to answer

    • Define autoregressive models
    • Explain NADE architecture and training
    • Mention application to density estimation
14. 7 Marks
    bExplain briefly about the following auto regressive models. MADE
    How to answer

    • Define MADE and masked autoencoder idea
    • Explain masking to enforce autoregressivity
    • Mention advantage for fast evaluation
15. 14 Marks
    Write a short note on any two of the following:

    1. a)Representation learning.
    2. b)Feed forward neural network
    3. c)Weight decay and also the benefits of using weight decay
    4. d)Deep belief networks
    How to answer

    • Select any two topics from list
    • Define each with key concepts and architecture
    • Give applications / advantages with diagram if relevant