Deep Learning (AL-503 (B)) - Important Questions

Explain feed-forward neural networks and sigmoid neurons with suitable diagram.

Explain PCA and SVD, how decomposition is performed, and their role in representation learning.

Explain the backpropagation algorithm and how it is used to update / optimize weights in a neural network during training

What is batch normalization, how does it work, and why is it used / what are its advantages in training deep networks?

Explain the historical evolution of deep learning, the McCulloch-Pitts neuron model, and the representation power of multilayer perceptrons.

Explain Multilayer perception with neat diagram.

Explain backpropagation algorithm with mathematical formulation, including weight initialization and batch normalization

Explain the representation power of multilayer perceptrons (multilayer networks of sigmoid neurons) and their capacity to learn complex relationships in data.

Explain the architecture of a simple neural network, activation functions and their purpose, and the role of backpropagation in training.

Explain the historical progression of deep learning highlighting key milestones and breakthroughs.

Discuss the GPU implementation of randomized SVD and write its applications.

Discuss regularization in autoencoders, including techniques and its role/impact on training.

Explain the Gradient Descent method in deep learning.

Compare gradient descent variants: GD, SGD and Momentum.

Explain denoising, sparse, contractive and variational autoencoders.

Explain the relationship of autoencoders with PCA and SVD and the role of dataset augmentation.

Explain about Nesterov Accelerated GD.

Explain the architecture and working principles of Deep Feed forward Neural Networks. Discuss their application areas and key advantages over other types of neural networks.

Investigate the theoretical foundations and practical implications of AdaGrad, Adam, and RMSProp optimization algorithms in the context of training deep neural networks.

When should autoencoders be used instead of PCA/SVD for dimensionality reduction, with justification.

Explain in brief sparse auto-encoder and contractive auto-encoder.

Define overfitting and underfitting and explain regularization techniques to prevent overfitting such as dropout, weight decay and early stopping.

Explain the architecture/concept of Convolutional Neural Networks (CNNs) and their role in image recognition, with neat diagram.

Define Stochastic pooling. Explain it with example.

Explain convolution operation, kernels, stride, padding and pooling in CNNs.

Explain architectures of VGGNet and ResNet.

Explain visualization of CNNs, Deep Dream and Deep Art.

Define the ReLU activation function and explain its properties and significance in neural networks/CNNs.

What are activation functions, why are they important in neural networks, and what are the different types used in deep learning?

Discuss the role of dropout in training deep neural networks and how it improves generalization.

Tabulate examples of different formats of data that can be used with a convolutional network system.

Explain the concept of Deep Dream and its role in generating surreal images based on neural network activations.

Illustrate the CNN terms Padding and Pooling.

Explain backpropagation through time (BPTT), including vanishing and exploding gradient problems and truncated BPTT.

Explain the fundamental concepts and architectures of Deep Recurrent Neural Networks with a neat diagram.

Explain directed graphical models.

Explain the working/architecture of Long Short-Term Memory (LSTM) networks and their advantages.

Explain LSTM and GRU architectures and their role in solving the vanishing gradient problem.

Explain speech recognition techniques in detail.

What is the vanishing gradient problem, how does it affect training of deep neural networks, and what strategies mitigate it?

What is a Recurrent Neural Network (RNN), why is it useful for sequence data, and how does it compare with a traditional feed-forward neural network?

Explain the vanishing and exploding gradient challenges during BPTT and their impact on learning long-range dependencies.

Discuss the challenges associated with encoding and decoding sequential data and how RNNs handle these challenges through their inherent architecture.

Write a short note on image recognition, video analytics, and weight decay.

Explain Restricted Boltzmann Machines with an example.

Define deep learning and describe its different applications.

Explain RBMs, Gibbs sampling and deep belief networks.

Explain GANs and applications of deep learning in vision, speech, and medical domains.

Discuss the differences between GANs and VAEs and when to choose one over the other.

Explain briefly about the following auto regressive models. NADE

Explain briefly about the following auto regressive models. MADE

Explain the difference between supervised and unsupervised learning. Provide examples where deep learning can be used for each type.

What is the difference between normalization and standardization in data preprocessing?

Discuss unsupervised learning with examples.

Explain how reinforcement learning differs from supervised learning and provide an example of how deep learning can be applied in reinforcement learning.

Explain the concept of value iteration in dynamic programming for solving Markov Decision Processes (MDPs).

Compare and contrast policy iteration with value iteration in terms of convergence and computational complexity.

Compare and contrast different least squares methods, such as LSPI (Least Squares Policy Iteration).

Explore advanced Q-learning algorithms, such as Double DQN and Dueling DQN.

Write a short note on any two of the following: