Skip to content
CampusPrep
AD-702 (C) · Computational Intelligence/Quick Revision Short Notes

Computational Intelligence (AD-702 (C)) - Unit 1 Short Notes

How unit 1 is examined

This unit defines computational intelligence, its types and components, learning and training, parametric and nonparametric models, and feed-forward and feedback networks; no past questions have been asked recently, so each topic is taught briefly but completely in case it appears.

Types of Computational Intelligence

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>Computational intelligence (CI) is a family of nature-inspired computing methods that solve complex, uncertain and imprecise problems by learning and adapting, where exact mathematical methods fail.</mark>

Key points.

  1. CI is also called soft computing because it tolerates imprecision, uncertainty and partial truth instead of demanding exact answers.
  2. The main types are artificial neural networks, fuzzy systems and evolutionary computation.
  3. Swarm intelligence (ant colony, particle swarm, bee colony) and rough set theory are also counted as types of CI.
  4. Each type copies nature: the brain (neural networks), human reasoning (fuzzy logic), evolution (genetic algorithms) and animal swarms (swarm intelligence).
  5. CI differs from hard computing, which needs a precise model, exact inputs and gives an exact output.
Type Inspired by Main use
Neural network Brain neurons Learning, classification
Fuzzy system Human reasoning Control under vagueness
Evolutionary computation Natural selection Optimisation
Swarm intelligence Ants, birds, bees Search, routing

Answer frame. Open with the definition; list the types with the natural source of each; give the table; close with the line that CI handles uncertainty where hard computing cannot.

Components of Computational Intelligence

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>The components of CI are artificial neural networks for learning, fuzzy logic for reasoning with vagueness, and evolutionary algorithms for optimisation.</mark>

Key points.

  1. Neural networks learn input-output patterns from data by adjusting connection weights.
  2. Fuzzy logic gives each element a degree of membership between 0 and 1, so vague human knowledge such as "warm" or "high" can be written as rules.
  3. Evolutionary algorithms, such as the genetic algorithm, search for good solutions using selection, crossover and mutation.
  4. The components are combined into hybrid systems, for example neuro-fuzzy (a network that tunes fuzzy rules) and genetic-fuzzy, because each covers the weakness of another.
  5. Neural networks learn but are hard to interpret; fuzzy systems are readable but cannot learn alone; evolutionary methods optimise but are slow.

<figure class="ds-fig" style="margin:1.4rem 0;overflow-x:auto"><svg xmlns="http://www.w3.org/2000/svg" id="dsfig-u1-01" viewBox="0 0 338 252" width="338" height="252" role="img" aria-label="Components of CI: NN neural network, FL fuzzy logic, EA evolutionary algorithms"><style>#dsfig-u1-01 .e{stroke:#454C5A;stroke-width:1.4;fill:none}#dsfig-u1-01 .e.hi{stroke:#2340B8;stroke-width:2.6}#dsfig-u1-01 .n{fill:#FFFFFF;stroke:#16181D;stroke-width:1.4}#dsfig-u1-01 .n.hi{fill:#E3E9FC;stroke:#2340B8;stroke-width:2.2}#dsfig-u1-01 .n.rb-b{fill:#16181D;stroke:#16181D}#dsfig-u1-01 .n.rb-r{fill:#BD3227;stroke:#BD3227}#dsfig-u1-01 text{font-family:"JetBrains Mono",ui-monospace,Menlo,Consolas,monospace;font-size:13px}#dsfig-u1-01 .t{fill:#16181D;font-weight:500}#dsfig-u1-01 .t.inv{fill:#FFFFFF;font-weight:700}#dsfig-u1-01 .kd{stroke:#16181D;stroke-width:1.2}#dsfig-u1-01 .dot{fill:#16181D}#dsfig-u1-01 .ann{fill:#2340B8;font-size:11px;font-weight:700}#dsfig-u1-01 .lbl{fill:#6F7787;font-family:system-ui,-apple-system,sans-serif;font-size:12px;font-weight:700}#dsfig-u1-01 .ptr{fill:#2340B8;font-size:12px;font-weight:700}#dsfig-u1-01 .ah{fill:#454C5A}#dsfig-u1-01 .ah.hi{fill:#2340B8}#dsfig-u1-01 .wl rect{fill:#FFFFFF;stroke:#DCE0E7}#dsfig-u1-01 .wl .t{font-size:12px;font-weight:700}#dsfig-u1-01 .wl.hi rect{fill:#2340B8;stroke:#2340B8}#dsfig-u1-01 .wl.hi .t{fill:#FFFFFF}html.dark #dsfig-u1-01 .e{stroke:#B1B7C3}html.dark #dsfig-u1-01 .e.hi{stroke:#8FA3FF}html.dark #dsfig-u1-01 .n{fill:#161920;stroke:#E6E8ED}html.dark #dsfig-u1-01 .n.hi{fill:#1E2748;stroke:#8FA3FF}html.dark #dsfig-u1-01 .n.rb-b{fill:#E6E8ED;stroke:#E6E8ED}html.dark #dsfig-u1-01 .n.rb-r{fill:#FF7E71;stroke:#FF7E71}html.dark #dsfig-u1-01 .t{fill:#E6E8ED}html.dark #dsfig-u1-01 .t.inv{fill:#0F1115}html.dark #dsfig-u1-01 .kd{stroke:#E6E8ED}html.dark #dsfig-u1-01 .dot{fill:#E6E8ED}html.dark #dsfig-u1-01 .ann{fill:#8FA3FF}html.dark #dsfig-u1-01 .lbl{fill:#858D9C}html.dark #dsfig-u1-01 .ptr{fill:#8FA3FF}html.dark #dsfig-u1-01 .ah{fill:#B1B7C3}html.dark #dsfig-u1-01 .ah.hi{fill:#8FA3FF}html.dark #dsfig-u1-01 .wl rect{fill:#161920;stroke:#2A2E37}html.dark #dsfig-u1-01 .wl.hi rect{fill:#8FA3FF;stroke:#8FA3FF}html.dark #dsfig-u1-01 .wl.hi .t{fill:#0F1115}</style><defs><marker id="ah1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah" d="M0,1 L9,5 L0,9 z"/></marker><marker id="ahh1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah hi" d="M0,1 L9,5 L0,9 z"/></marker></defs><path class="e" d="M157.6,55.2 L52.6,195.2" marker-end="url(#ah1)"/><path class="e" d="M169,59 L169,191" marker-end="url(#ah1)"/><path class="e" d="M180.4,55.2 L285.4,195.2" marker-end="url(#ah1)"/><circle class="n" cx="169" cy="40" r="18"/><text class="t" x="169" y="40" dy=".35em" text-anchor="middle">CI</text><circle class="n" cx="40" cy="212" r="18"/><text class="t" x="40" y="212" dy=".35em" text-anchor="middle">NN</text><circle class="n" cx="169" cy="212" r="18"/><text class="t" x="169" y="212" dy=".35em" text-anchor="middle">FL</text><circle class="n" cx="298" cy="212" r="18"/><text class="t" x="298" y="212" dy=".35em" text-anchor="middle">EA</text></svg><figcaption style="font-size:.82em;opacity:.72;margin-top:.45rem">Components of CI: NN neural network, FL fuzzy logic, EA evolutionary algorithms</figcaption></figure>

Answer frame. Open with the definition; draw the diagram with the three boxes; develop one point per component in the order learning, reasoning, optimisation; close with hybrid systems.

Concept of Learning, Training model

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>Learning is the process of adjusting a model's parameters from data so that its error on the task falls; training is the step in which this adjustment is done on the training set.</mark>

Key points.

  1. In supervised learning, the training data has inputs with known target outputs, and the model reduces the error between prediction and target.
  2. In unsupervised learning, there are no targets, and the model finds clusters or structure by itself.
  3. In reinforcement learning, an agent learns by trial, receiving reward or penalty for its actions.
  4. The data is split into a training set to fit the model and a test set of unseen data to measure generalisation.
  5. Training repeats until the error is small enough; a model that fits training data well but fails on test data is overfitted.
  6. A weight update in a neural network follows the rule $w_{new} = w_{old} + \eta \, (t - y)\, x$, where $\eta$ is the learning rate, $t$ the target and $y$ the output.

Example. For $w = 0.5$, $x = 1$, $t = 1$, $y = 0$ and $\eta = 0.1$: $w_{new} = 0.5 + 0.1(1-0)(1) = 0.6$, so the weight moves towards the target.

Answer frame. Open with the definition; state the three kinds of learning with one line each; add train-test split and overfitting; close with the weight-update rule.

Parametric Models

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>A parametric model assumes a fixed functional form with a fixed number of parameters, whatever the size of the data.</mark>

Key points.

  1. Linear regression is the standard example: $y = w_1 x + w_0$, with only two parameters.
  2. Training only estimates the parameters $w$; the data can then be discarded.
  3. It is fast and works with little data, but a wrongly assumed form gives high bias and underfitting.
  4. Logistic regression and a neural network of fixed size are also parametric.
  5. The parameters are chosen to minimise the squared error $E = \sum (t_i - y_i)^2$.

Example. Data $(1,2), (2,4), (3,5)$ gives the least-squares line $y = 1.5x + 0.67$, and its prediction at $x = 4$ is $6.67$. The whole data set is now summarised by two numbers.

Answer frame. Open with the definition; give the linear regression form; develop speed, small data need and bias; close with the contrast that nonparametric models grow with data.

Nonparametric Models

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>A nonparametric model makes no fixed-form assumption, so its complexity grows with the amount of training data.</mark>

Key points.

  1. k-nearest neighbours (kNN) classifies a point by the majority class of its $k$ closest training points, using the distance $d = \sqrt{\sum (x_i - q_i)^2}$.
  2. It keeps all the training data, so prediction is slow and memory use is high.
  3. It is flexible and fits any shape of data, but a small $k$ overfits and a large $k$ underfits.
  4. Decision trees and kernel methods are other examples.
  5. It needs more data than a parametric model to work well.

Example. Training points $(1,1)A$, $(2,1)A$, $(4,4)B$, $(5,4)B$, $(5,5)B$ and query $q = (3,2)$ with $k = 3$: the distances are $1.41$ for $(2,1)$, $2.24$ for $(1,1)$ and $2.24$ for $(4,4)$, so the nearest three are A, A, B and the vote is class A.

Point Distance to $q$ Class
(2,1) 1.41 A
(1,1) 2.24 A
(4,4) 2.24 B

Answer frame. Open with the definition; explain kNN with the distance formula; give the small example; close with a comparison to parametric models (fixed versus growing, fast versus slow prediction).

Feed Forward network

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>A feed-forward network is a neural network in which signals travel only from input to output, with no loops or cycles.</mark>

Key points.

  1. The layers are input, one or more hidden layers and output, and each neuron sends its output only to the next layer.
  2. A neuron computes $y = f\left(\sum_i w_i x_i + b\right)$, where $f$ is the activation function and $b$ the bias.
  3. The single-layer perceptron and the multilayer perceptron (MLP) are the examples.
  4. It is trained with backpropagation, which passes the error backwards to update the weights.
  5. It has no memory of past inputs, so the output depends only on the present input.
  6. It is used for classification and function approximation.

<figure class="ds-fig" style="margin:1.4rem 0;overflow-x:auto"><svg xmlns="http://www.w3.org/2000/svg" id="dsfig-u1-02" viewBox="0 0 424 252" width="424" height="252" role="img" aria-label="Feed-forward network: X inputs, H hidden neurons, Y output"><style>#dsfig-u1-02 .e{stroke:#454C5A;stroke-width:1.4;fill:none}#dsfig-u1-02 .e.hi{stroke:#2340B8;stroke-width:2.6}#dsfig-u1-02 .n{fill:#FFFFFF;stroke:#16181D;stroke-width:1.4}#dsfig-u1-02 .n.hi{fill:#E3E9FC;stroke:#2340B8;stroke-width:2.2}#dsfig-u1-02 .n.rb-b{fill:#16181D;stroke:#16181D}#dsfig-u1-02 .n.rb-r{fill:#BD3227;stroke:#BD3227}#dsfig-u1-02 text{font-family:"JetBrains Mono",ui-monospace,Menlo,Consolas,monospace;font-size:13px}#dsfig-u1-02 .t{fill:#16181D;font-weight:500}#dsfig-u1-02 .t.inv{fill:#FFFFFF;font-weight:700}#dsfig-u1-02 .kd{stroke:#16181D;stroke-width:1.2}#dsfig-u1-02 .dot{fill:#16181D}#dsfig-u1-02 .ann{fill:#2340B8;font-size:11px;font-weight:700}#dsfig-u1-02 .lbl{fill:#6F7787;font-family:system-ui,-apple-system,sans-serif;font-size:12px;font-weight:700}#dsfig-u1-02 .ptr{fill:#2340B8;font-size:12px;font-weight:700}#dsfig-u1-02 .ah{fill:#454C5A}#dsfig-u1-02 .ah.hi{fill:#2340B8}#dsfig-u1-02 .wl rect{fill:#FFFFFF;stroke:#DCE0E7}#dsfig-u1-02 .wl .t{font-size:12px;font-weight:700}#dsfig-u1-02 .wl.hi rect{fill:#2340B8;stroke:#2340B8}#dsfig-u1-02 .wl.hi .t{fill:#FFFFFF}html.dark #dsfig-u1-02 .e{stroke:#B1B7C3}html.dark #dsfig-u1-02 .e.hi{stroke:#8FA3FF}html.dark #dsfig-u1-02 .n{fill:#161920;stroke:#E6E8ED}html.dark #dsfig-u1-02 .n.hi{fill:#1E2748;stroke:#8FA3FF}html.dark #dsfig-u1-02 .n.rb-b{fill:#E6E8ED;stroke:#E6E8ED}html.dark #dsfig-u1-02 .n.rb-r{fill:#FF7E71;stroke:#FF7E71}html.dark #dsfig-u1-02 .t{fill:#E6E8ED}html.dark #dsfig-u1-02 .t.inv{fill:#0F1115}html.dark #dsfig-u1-02 .kd{stroke:#E6E8ED}html.dark #dsfig-u1-02 .dot{fill:#E6E8ED}html.dark #dsfig-u1-02 .ann{fill:#8FA3FF}html.dark #dsfig-u1-02 .lbl{fill:#858D9C}html.dark #dsfig-u1-02 .ptr{fill:#8FA3FF}html.dark #dsfig-u1-02 .ah{fill:#B1B7C3}html.dark #dsfig-u1-02 .ah.hi{fill:#8FA3FF}html.dark #dsfig-u1-02 .wl rect{fill:#161920;stroke:#2A2E37}html.dark #dsfig-u1-02 .wl.hi rect{fill:#8FA3FF;stroke:#8FA3FF}html.dark #dsfig-u1-02 .wl.hi .t{fill:#0F1115}</style><defs><marker id="ah2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah" d="M0,1 L9,5 L0,9 z"/></marker><marker id="ahh2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah hi" d="M0,1 L9,5 L0,9 z"/></marker></defs><path class="e" d="M59,40 L191,40" marker-end="url(#ah2)"/><path class="e" d="M53.4,53.4 L197.2,197.2" marker-end="url(#ah2)"/><path class="e" d="M53.4,198.6 L197.2,54.8" marker-end="url(#ah2)"/><path class="e" d="M59,212 L191,212" marker-end="url(#ah2)"/><path class="e" d="M229,48.5 L365.2,116.6" marker-end="url(#ah2)"/><path class="e" d="M229,203.5 L365.2,135.4" marker-end="url(#ah2)"/><circle class="n" cx="40" cy="40" r="18"/><text class="t" x="40" y="40" dy=".35em" text-anchor="middle">X1</text><circle class="n" cx="40" cy="212" r="18"/><text class="t" x="40" y="212" dy=".35em" text-anchor="middle">X2</text><circle class="n" cx="212" cy="40" r="18"/><text class="t" x="212" y="40" dy=".35em" text-anchor="middle">H1</text><circle class="n" cx="212" cy="212" r="18"/><text class="t" x="212" y="212" dy=".35em" text-anchor="middle">H2</text><circle class="n" cx="384" cy="126" r="18"/><text class="t" x="384" y="126" dy=".35em" text-anchor="middle">Y</text></svg><figcaption style="font-size:.82em;opacity:.72;margin-top:.45rem">Feed-forward network: X inputs, H hidden neurons, Y output</figcaption></figure>

Example. A perceptron with $w_1 = w_2 = 0.5$, $b = -0.3$ and inputs $(1,1)$ has net $= 0.5 + 0.5 - 0.3 = 0.7$, and with a step activation the output is 1.

Answer frame. Open with the definition; draw the diagram; develop the neuron equation, layers, backpropagation and no-memory points; close with the contrast to feedback networks.

Feedback network

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. <mark>A feedback (recurrent) network has connections that carry outputs back as inputs, so the output depends on both the present input and earlier states.</mark>

Key points.

  1. The loops give the network memory, so it suits sequences and time series.
  2. The Hopfield network is the classic example: fully connected, with symmetric weights $w_{ij} = w_{ji}$ and no self-connection.
  3. It is a dynamic system that moves through states until it settles to a stable state.
  4. It is used as an associative memory, to recall a stored pattern from a noisy one, and for optimisation.
Feed forward Feedback
Signal flows one way Outputs loop back
No memory Has memory
Static mapping Dynamic behaviour
MLP Hopfield, recurrent network

Answer frame. Open with the definition; state the loop and memory; describe the Hopfield network; close with the comparison table.

Last-minute revision

  • Computational intelligence is soft computing: neural networks, fuzzy logic and evolutionary computation.
  • Fuzzy membership lies between 0 and 1.
  • Supervised learning uses labelled data; unsupervised finds structure; reinforcement uses reward.
  • Weight update: $w_{new} = w_{old} + \eta (t - y) x$.
  • Parametric models have a fixed number of parameters, for example linear regression $y = w_1 x + w_0$.
  • Nonparametric models grow with the data, for example kNN.
  • kNN uses the majority vote of the $k$ nearest points by Euclidean distance.
  • Neuron output: $y = f\left(\sum w_i x_i + b\right)$.
  • Feed-forward networks have no loops; feedback networks have loops and memory.
  • The Hopfield network is a feedback network with symmetric weights and no self-connection.

Memory hooks

  • NFE: Neural, Fuzzy, Evolutionary are the three components.
  • Parametric means fixed: the parameters stay fixed.
  • Nonparametric means grows: it keeps the data, as kNN does.
  • Feed-forward is one-way; feedback loops back and remembers.
  • Small $k$ overfits, large $k$ underfits.

Coverage checklist

  • Types of Computational Intelligence: definition, types, table; no past questions.
  • Components of Computational Intelligence: three components, hybrids, diagram; no past questions.
  • Concept of Learning, Training model: three kinds of learning, train-test, update rule; no past questions.
  • Parametric Models: fixed form, linear regression example; no past questions.
  • Nonparametric Models: kNN, worked example; no past questions.
  • Feed Forward network: layers, neuron equation, diagram; no past questions.
  • Feedback network: loops, Hopfield, comparison; no past questions.
Go to where you left off?

Quick Add to Notes

Save questions, your own notes and screenshots into notes filed by unit. It takes a free account.

Create free account

Have an account? Log in