How unit 1 is examined
This unit covers the biological neuron and its artificial models, from the McCulloch-Pitts unit to the perceptron, feed-forward networks and backpropagation. Marks sit in the biological neuron, deep learning basics (ML vs DL, types), the neural network note and McCulloch-Pitts.
Biological Neuron
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Medium weight</span>
Definition. <mark>Deep Learning is a subset of machine learning that uses multi-layer (deep) neural networks to learn hierarchical features automatically from raw data.</mark> A biological neuron is the basic nerve cell of the brain that receives, integrates and transmits electrical signals.
Diagram.
<figure class="ds-fig" style="margin:1.4rem 0;overflow-x:auto"><svg xmlns="http://www.w3.org/2000/svg" id="dsfig-u1-01" viewBox="0 0 544.4 183.2" width="544.4" height="183.2" role="img" aria-label="Biological neuron. Dn = dendrites, Som = soma (cell body), Nuc = nucleus, Axn = axon, Mye = myelin sheath, Syn = synaptic terminals"><style>#dsfig-u1-01 .e{stroke:#454C5A;stroke-width:1.4;fill:none}#dsfig-u1-01 .e.hi{stroke:#2340B8;stroke-width:2.6}#dsfig-u1-01 .n{fill:#FFFFFF;stroke:#16181D;stroke-width:1.4}#dsfig-u1-01 .n.hi{fill:#E3E9FC;stroke:#2340B8;stroke-width:2.2}#dsfig-u1-01 .n.rb-b{fill:#16181D;stroke:#16181D}#dsfig-u1-01 .n.rb-r{fill:#BD3227;stroke:#BD3227}#dsfig-u1-01 text{font-family:"JetBrains Mono",ui-monospace,Menlo,Consolas,monospace;font-size:13px}#dsfig-u1-01 .t{fill:#16181D;font-weight:500}#dsfig-u1-01 .t.inv{fill:#FFFFFF;font-weight:700}#dsfig-u1-01 .kd{stroke:#16181D;stroke-width:1.2}#dsfig-u1-01 .dot{fill:#16181D}#dsfig-u1-01 .ann{fill:#2340B8;font-size:11px;font-weight:700}#dsfig-u1-01 .lbl{fill:#6F7787;font-family:system-ui,-apple-system,sans-serif;font-size:12px;font-weight:700}#dsfig-u1-01 .ptr{fill:#2340B8;font-size:12px;font-weight:700}#dsfig-u1-01 .ah{fill:#454C5A}#dsfig-u1-01 .ah.hi{fill:#2340B8}#dsfig-u1-01 .wl rect{fill:#FFFFFF;stroke:#DCE0E7}#dsfig-u1-01 .wl .t{font-size:12px;font-weight:700}#dsfig-u1-01 .wl.hi rect{fill:#2340B8;stroke:#2340B8}#dsfig-u1-01 .wl.hi .t{fill:#FFFFFF}html.dark #dsfig-u1-01 .e{stroke:#B1B7C3}html.dark #dsfig-u1-01 .e.hi{stroke:#8FA3FF}html.dark #dsfig-u1-01 .n{fill:#161920;stroke:#E6E8ED}html.dark #dsfig-u1-01 .n.hi{fill:#1E2748;stroke:#8FA3FF}html.dark #dsfig-u1-01 .n.rb-b{fill:#E6E8ED;stroke:#E6E8ED}html.dark #dsfig-u1-01 .n.rb-r{fill:#FF7E71;stroke:#FF7E71}html.dark #dsfig-u1-01 .t{fill:#E6E8ED}html.dark #dsfig-u1-01 .t.inv{fill:#0F1115}html.dark #dsfig-u1-01 .kd{stroke:#E6E8ED}html.dark #dsfig-u1-01 .dot{fill:#E6E8ED}html.dark #dsfig-u1-01 .ann{fill:#8FA3FF}html.dark #dsfig-u1-01 .lbl{fill:#858D9C}html.dark #dsfig-u1-01 .ptr{fill:#8FA3FF}html.dark #dsfig-u1-01 .ah{fill:#B1B7C3}html.dark #dsfig-u1-01 .ah.hi{fill:#8FA3FF}html.dark #dsfig-u1-01 .wl rect{fill:#161920;stroke:#2A2E37}html.dark #dsfig-u1-01 .wl.hi rect{fill:#8FA3FF;stroke:#8FA3FF}html.dark #dsfig-u1-01 .wl.hi .t{fill:#0F1115}</style><defs><marker id="ah1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah" d="M0,1 L9,5 L0,9 z"/></marker><marker id="ahh1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah hi" d="M0,1 L9,5 L0,9 z"/></marker></defs><path class="e" d="M59,40 L156.6,40" marker-end="url(#ah1)"/><path class="e" d="M177.6,124.2 L177.6,59"/><path class="e" d="M196.6,40 L277,40" marker-end="url(#ah1)"/><path class="e" d="M317,40 L363,40" marker-end="url(#ah1)"/><path class="e" d="M403,40 L483.4,40" marker-end="url(#ah1)"/><circle class="n" cx="40" cy="40" r="18"/><text class="t" x="40" y="40" dy=".35em" text-anchor="middle">Dn</text><circle class="n" cx="177.6" cy="40" r="18"/><text class="t" x="177.6" y="40" dy=".35em" text-anchor="middle">Som</text><circle class="n" cx="177.6" cy="143.2" r="18"/><text class="t" x="177.6" y="143.2" dy=".35em" text-anchor="middle">Nuc</text><circle class="n" cx="298" cy="40" r="18"/><text class="t" x="298" y="40" dy=".35em" text-anchor="middle">Axn</text><circle class="n" cx="384" cy="40" r="18"/><text class="t" x="384" y="40" dy=".35em" text-anchor="middle">Mye</text><circle class="n" cx="504.4" cy="40" r="18"/><text class="t" x="504.4" y="40" dy=".35em" text-anchor="middle">Syn</text></svg><figcaption style="font-size:.82em;opacity:.72;margin-top:.45rem">Biological neuron. Dn = dendrites, Som = soma (cell body), Nuc = nucleus, Axn = axon, Mye = myelin sheath, Syn = synaptic terminals</figcaption></figure>
Key points.
- Dendrites are branched fibres that receive signals from other neurons and carry them to the cell body (input reception).
- The soma (cell body) contains the nucleus and sums up the incoming signals (integration).
- If the summed signal crosses a threshold, the neuron fires an electrical impulse (activation).
- The axon is a long fibre that carries the impulse away from the soma, and the myelin sheath around it insulates it and speeds up conduction.
- Synaptic terminals at the axon end pass the signal to the next neuron's dendrites through chemical messengers (output transmission).
- Mapping to the artificial neuron: dendrites are inputs, synapse strength is the weight, soma is the summation plus activation, and axon is the output.
Biological vs artificial neural network.
| Aspect | Biological | Artificial |
|---|---|---|
| Unit | Neuron with dendrites, soma, axon | Node with inputs, weights, activation |
| Speed | Slow (milliseconds), massively parallel | Very fast, but largely sequential |
| Learning | Synaptic plasticity, few examples | Gradient descent, needs large data |
| Energy | About 20 W for the whole brain | Hundreds of watts for large models |
| Robustness | Graceful, generalises to new situations | Brittle outside training data |
| Size | About 86 billion neurons | Millions to billions of parameters |
Limitations of current deep learning against the brain: it needs huge labelled data, it has weak common-sense reasoning, it transfers poorly to new tasks, it is a black box, and it consumes far more energy.
Answer frame. Open with the definition of deep learning; draw the labelled neuron; develop points 1-5 then map to the artificial neuron (point 6); close with "the artificial neuron is a simplified model of this cell". For the comparison, use the table and end with the limitations line.
Asked: [8 marks] (May 2023) What is Deep Learning? Discuss the structure and units of Biological Neuron. Asked: [7 marks] (Jun 2025) Compare biological neural networks with artificial neural networks. What are the limitations of current deep learning models compared to the human brain?
Idea of computational units
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Low weight</span>
Definition. <mark>A neural network is a computational model of interconnected artificial neurons (computational units) that learns a mapping from input to output by adjusting weights.</mark>
Key points.
- Each unit computes a weighted sum of its inputs plus a bias, $z=\sum w_ix_i+b$, and passes it through an activation function, $y=f(z)$.
- Units are arranged in an input layer, one or more hidden layers and an output layer.
- Main types are feed-forward (data flows one way), recurrent (loops for sequences) and convolutional (for images).
- Applications include image recognition, speech recognition, translation and forecasting.
Machine Learning vs Deep Learning.
| Aspect | Machine Learning | Deep Learning |
|---|---|---|
| Features | Hand-crafted by engineers | Learned automatically, layer by layer |
| Data needed | Works with small data | Needs very large data |
| Hardware | CPU is enough | GPU/TPU needed |
| Interpretability | Easier to explain | Black box |
| Performance | Plateaus with more data | Keeps improving with data |
| Examples | Spam filter, decision trees | Image recognition, speech, translation |
Types of Deep Learning. CNN (convolutional networks) for images and video; RNN/LSTM (recurrent) for sequences, text and speech; autoencoders (compress and reconstruct data) for denoising and anomaly detection; GANs (generator versus discriminator) for generating realistic images; plus deep belief networks and transformers.
Answer frame. Open with the definition; list the components (weights, bias, activation), then types and applications; for ML vs DL draw the table and close with when to use each; for types, give one line each on characteristic and application.
Asked: [5 marks] (May 2023) Write a short note on neural network. Asked: [6 marks] (May 2023) Explain the difference between Machine Learning and Deep Learning. Asked: [8 marks] (May 2024) What do you understand by deep learning? Explain the different types of Deep Learning.
McCulloch-Pitts Neural Model
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Low weight</span>
Definition. A neuron is the basic processing unit of a neural network. <mark>The McCulloch-Pitts (1943) model is a binary threshold neuron: it sums binary inputs and outputs 1 if the sum reaches a threshold $\theta$, otherwise 0.</mark>
Diagram.
<figure class="ds-fig" style="margin:1.4rem 0;overflow-x:auto"><svg xmlns="http://www.w3.org/2000/svg" id="dsfig-u1-02" viewBox="0 0 424 209" width="424" height="209" role="img" aria-label="McCulloch-Pitts neuron. X1, X2 = binary inputs, Sum = summation with threshold, y = binary output"><style>#dsfig-u1-02 .e{stroke:#454C5A;stroke-width:1.4;fill:none}#dsfig-u1-02 .e.hi{stroke:#2340B8;stroke-width:2.6}#dsfig-u1-02 .n{fill:#FFFFFF;stroke:#16181D;stroke-width:1.4}#dsfig-u1-02 .n.hi{fill:#E3E9FC;stroke:#2340B8;stroke-width:2.2}#dsfig-u1-02 .n.rb-b{fill:#16181D;stroke:#16181D}#dsfig-u1-02 .n.rb-r{fill:#BD3227;stroke:#BD3227}#dsfig-u1-02 text{font-family:"JetBrains Mono",ui-monospace,Menlo,Consolas,monospace;font-size:13px}#dsfig-u1-02 .t{fill:#16181D;font-weight:500}#dsfig-u1-02 .t.inv{fill:#FFFFFF;font-weight:700}#dsfig-u1-02 .kd{stroke:#16181D;stroke-width:1.2}#dsfig-u1-02 .dot{fill:#16181D}#dsfig-u1-02 .ann{fill:#2340B8;font-size:11px;font-weight:700}#dsfig-u1-02 .lbl{fill:#6F7787;font-family:system-ui,-apple-system,sans-serif;font-size:12px;font-weight:700}#dsfig-u1-02 .ptr{fill:#2340B8;font-size:12px;font-weight:700}#dsfig-u1-02 .ah{fill:#454C5A}#dsfig-u1-02 .ah.hi{fill:#2340B8}#dsfig-u1-02 .wl rect{fill:#FFFFFF;stroke:#DCE0E7}#dsfig-u1-02 .wl .t{font-size:12px;font-weight:700}#dsfig-u1-02 .wl.hi rect{fill:#2340B8;stroke:#2340B8}#dsfig-u1-02 .wl.hi .t{fill:#FFFFFF}html.dark #dsfig-u1-02 .e{stroke:#B1B7C3}html.dark #dsfig-u1-02 .e.hi{stroke:#8FA3FF}html.dark #dsfig-u1-02 .n{fill:#161920;stroke:#E6E8ED}html.dark #dsfig-u1-02 .n.hi{fill:#1E2748;stroke:#8FA3FF}html.dark #dsfig-u1-02 .n.rb-b{fill:#E6E8ED;stroke:#E6E8ED}html.dark #dsfig-u1-02 .n.rb-r{fill:#FF7E71;stroke:#FF7E71}html.dark #dsfig-u1-02 .t{fill:#E6E8ED}html.dark #dsfig-u1-02 .t.inv{fill:#0F1115}html.dark #dsfig-u1-02 .kd{stroke:#E6E8ED}html.dark #dsfig-u1-02 .dot{fill:#E6E8ED}html.dark #dsfig-u1-02 .ann{fill:#8FA3FF}html.dark #dsfig-u1-02 .lbl{fill:#858D9C}html.dark #dsfig-u1-02 .ptr{fill:#8FA3FF}html.dark #dsfig-u1-02 .ah{fill:#B1B7C3}html.dark #dsfig-u1-02 .ah.hi{fill:#8FA3FF}html.dark #dsfig-u1-02 .wl rect{fill:#161920;stroke:#2A2E37}html.dark #dsfig-u1-02 .wl.hi rect{fill:#8FA3FF;stroke:#8FA3FF}html.dark #dsfig-u1-02 .wl.hi .t{fill:#0F1115}</style><defs><marker id="ah2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah" d="M0,1 L9,5 L0,9 z"/></marker><marker id="ahh2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah hi" d="M0,1 L9,5 L0,9 z"/></marker></defs><path class="e" d="M57.8,46.7 L192.3,97.1" marker-end="url(#ah2)"/><path class="e" d="M57.8,162.3 L192.3,111.9" marker-end="url(#ah2)"/><path class="e" d="M231,104.5 L363,104.5" marker-end="url(#ah2)"/><g class="wl"><rect x="112.8" y="63.3" width="26.4" height="18" rx="9"/><text class="t" x="126" y="72.3" dy=".35em" text-anchor="middle">w1</text></g><g class="wl"><rect x="112.8" y="127.8" width="26.4" height="18" rx="9"/><text class="t" x="126" y="136.8" dy=".35em" text-anchor="middle">w2</text></g><g class="wl"><rect x="288.4" y="95.5" width="19.2" height="18" rx="9"/><text class="t" x="298" y="104.5" dy=".35em" text-anchor="middle">y</text></g><circle class="n" cx="40" cy="40" r="18"/><text class="t" x="40" y="40" dy=".35em" text-anchor="middle">X1</text><circle class="n" cx="40" cy="169" r="18"/><text class="t" x="40" y="169" dy=".35em" text-anchor="middle">X2</text><circle class="n" cx="212" cy="104.5" r="18"/><text class="t" x="212" y="104.5" dy=".35em" text-anchor="middle">Sum</text><circle class="n" cx="384" cy="104.5" r="18"/><text class="t" x="384" y="104.5" dy=".35em" text-anchor="middle">Out</text></svg><figcaption style="font-size:.82em;opacity:.72;margin-top:.45rem">McCulloch-Pitts neuron. X1, X2 = binary inputs, Sum = summation with threshold, y = binary output</figcaption></figure>
Formula.
$$y=\begin{cases}1 & \sum_i w_ix_i \ge \theta\\ 0 & \text{otherwise}\end{cases}$$
Key points.
- Inputs and output are binary (0 or 1), and weights are fixed, not learned.
- Excitatory inputs have weight +1, and one active inhibitory input forces the output to 0.
- With $w=(1,1)$: $\theta=2$ gives AND (outputs 0,0,0,1) and $\theta=1$ gives OR (outputs 0,1,1,1); NOT uses weight $-1$ with $\theta=0$.
- Capability: it can build any Boolean function by combining gates.
- Limitation: no learning, no real-valued inputs, and a single unit cannot solve XOR.
Answer frame. Open with defining neuron, then the model; draw the diagram and formula; show AND/OR example; close with limitations.
Asked: [9 marks] (May 2024) Define Neuron. Explain McCulloch-Pitts Neural Model in detail.
Linear Perceptron
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>
Definition. ==A perceptron (Rosenblatt, 1958) is a single-layer neuron that computes a weighted sum plus bias and applies a step function: $y=1$ if $\mathbf{w}\cdot\mathbf{x}+b>0$, else 0.==
Key points.
- Unlike McCulloch-Pitts, its weights are learnable and inputs can be real numbers.
- It is a linear classifier: the boundary $\mathbf{w}\cdot\mathbf{x}+b=0$ is a straight line or hyperplane.
- It solves only linearly separable problems (AND, OR) and fails on XOR.
Perceptron Learning
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>
Definition. <mark>The perceptron learning rule corrects the weights after each error: $w_i \leftarrow w_i+\eta\,(t-y)\,x_i$ and $b \leftarrow b+\eta\,(t-y)$.</mark>
Key points.
- Here $t$ is the target, $y$ the output and $\eta$ the learning rate; no change is made when the output is correct.
- Example: $w=(0,0)$, $b=0$, $\eta=1$, input $(1,1)$, target 1 gives $y=0$, so the new $w=(1,1)$, $b=1$.
- The rule converges in finite steps if the data are linearly separable.
- It repeats over the training set until no errors remain.
Feed Forward Networks
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>
Definition. <mark>A feed-forward network is a neural network whose connections go only from input to output through hidden layers, with no loops.</mark>
Key points.
- Each layer computes $\mathbf{h}=f(W\mathbf{x}+\mathbf{b})$ and passes it to the next layer.
- With non-linear activations and hidden layers, it can learn non-linear functions such as XOR.
- The multilayer perceptron (MLP) is the standard example.
Back Propagation Networks
<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>
Definition. <mark>Backpropagation is the algorithm that trains a multi-layer network by propagating the output error backwards using the chain rule to compute gradients of the loss with respect to every weight.</mark>
Key points.
- A forward pass computes the output and the loss $E$.
- A backward pass computes $\partial E/\partial w$ layer by layer using the chain rule.
- Weights are updated by gradient descent: $w \leftarrow w-\eta\,\partial E/\partial w$.
- This is repeated over many epochs until the loss is small.
Last-minute revision
- Deep learning is a subset of ML using multi-layer neural networks that learn features automatically.
- Neuron parts: dendrites (input), soma (sum), axon (output), myelin sheath (insulation), synapse (link).
- A neural unit computes $y=f(\sum w_ix_i+b)$.
- ML needs hand-made features and less data; DL learns features and needs more data and GPUs.
- Types of DL: CNN, RNN/LSTM, autoencoder, GAN.
- McCulloch-Pitts: binary threshold neuron with fixed weights; AND has $\theta=2$, OR has $\theta=1$ for weights (1,1).
- A single perceptron cannot solve XOR.
- Perceptron rule: $w\leftarrow w+\eta(t-y)x$.
- Feed-forward networks have no loops; backpropagation uses the chain rule to get gradients.
Memory hooks
- DASA: Dendrites in, Axon out, Soma sums, Axon-end synapse passes on.
- MP = "Fixed and Fire": fixed weights, fires at threshold.
- CRAG: CNN, RNN, Autoencoder, GAN.
- Forward for the answer, backward for the blame.
Coverage checklist
- Biological Neuron: Q2 (May 2023), Q4 (Jun 2025)
- Idea of computational units: Q6 (May 2023), plus Q5 (May 2023) and Q3 (May 2024)
- McCulloch-Pitts Neural Model: Q1 (May 2024)
- Linear Perceptron: no past questions
- Perceptron Learning: no past questions
- Feed Forward Networks: no past questions
- Back Propagation Networks: no past questions