Pattern Recognition (CS-503 (B)) Unit 1 Notes

How unit 1 is examined

This unit defines pattern recognition, its design cycle, learning types (supervised, unsupervised, adaptation), discriminant functions with decision boundaries, and metric spaces; design principles, supervised versus unsupervised learning and the definition questions carry the marks.

Definitions

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Low weight</span>

Definition. <mark>Pattern recognition is the automatic assignment of an object or pattern, described by measured features, to one of a set of categories or classes.</mark>

Key points.

A pattern is any measurable description of an object, such as an image, a speech signal or a feature vector $x=(x_1,\dots,x_d)$.
A pattern recognition system has four stages: sensing (a sensor captures raw data), segmentation (the object is isolated from its background), feature extraction (informative measurements are computed) and classification (a decision assigns a class).
The design cycle is data collection, feature choice, model choice, training, evaluation, and it is repeated until performance is acceptable.
Applications include character recognition, speech recognition, fingerprint identification and medical diagnosis.

Asked: [7 marks] (Dec 2020) What is pattern recognition? Explain it.

Data sets for Pattern

Definition. A pattern data set is a collection of sample patterns, each stored as a feature vector, used to train and test a recognition system.

Key points.

Each sample is a row of feature values, and in supervised problems it also carries a class label.
The data is split into a training set, which builds the model, and a test set, which measures performance on unseen samples.
The samples must be representative of real conditions and large enough, otherwise the classifier does not generalise.

Application areas and examples of pattern recognition

Definition. Applications of pattern recognition are the practical problems in which objects must be automatically assigned to categories.

Key points.

Image and vision: optical character recognition, face and fingerprint recognition, and medical image analysis.
Speech and text: speech recognition, speaker identification and spam filtering.
Biomedical and industrial use: ECG classification, disease diagnosis and defect inspection.
Other uses include remote sensing, credit-card fraud detection and bioinformatics.

Design principles of pattern recognition system

Definition. <mark>A pattern recognition system is designed as a cycle of data collection, feature choice, model choice, training and evaluation, in which each stage is revised using feedback until the classifier generalises well.</mark>

Diagram. <figure class="ds-fig" style="margin:1.4rem 0;overflow-x:auto"><svg xmlns="http://www.w3.org/2000/svg" id="dsfig-u1-01" viewBox="0 0 492.8 252" width="492.8" height="252" role="img" aria-label="Design cycle. Env = environment, Sen = sensing and preprocessing, Fea = feature extraction, Mod = model or knowledge base, Trn = training, Evl = evaluation with feedback"><style>#dsfig-u1-01 .e{stroke:#454C5A;stroke-width:1.4;fill:none}#dsfig-u1-01 .e.hi{stroke:#2340B8;stroke-width:2.6}#dsfig-u1-01 .n{fill:#FFFFFF;stroke:#16181D;stroke-width:1.4}#dsfig-u1-01 .n.hi{fill:#E3E9FC;stroke:#2340B8;stroke-width:2.2}#dsfig-u1-01 .n.rb-b{fill:#16181D;stroke:#16181D}#dsfig-u1-01 .n.rb-r{fill:#BD3227;stroke:#BD3227}#dsfig-u1-01 text{font-family:"JetBrains Mono",ui-monospace,Menlo,Consolas,monospace;font-size:13px}#dsfig-u1-01 .t{fill:#16181D;font-weight:500}#dsfig-u1-01 .t.inv{fill:#FFFFFF;font-weight:700}#dsfig-u1-01 .kd{stroke:#16181D;stroke-width:1.2}#dsfig-u1-01 .dot{fill:#16181D}#dsfig-u1-01 .ann{fill:#2340B8;font-size:11px;font-weight:700}#dsfig-u1-01 .lbl{fill:#6F7787;font-family:system-ui,-apple-system,sans-serif;font-size:12px;font-weight:700}#dsfig-u1-01 .ptr{fill:#2340B8;font-size:12px;font-weight:700}#dsfig-u1-01 .ah{fill:#454C5A}#dsfig-u1-01 .ah.hi{fill:#2340B8}#dsfig-u1-01 .wl rect{fill:#FFFFFF;stroke:#DCE0E7}#dsfig-u1-01 .wl .t{font-size:12px;font-weight:700}#dsfig-u1-01 .wl.hi rect{fill:#2340B8;stroke:#2340B8}#dsfig-u1-01 .wl.hi .t{fill:#FFFFFF}html.dark #dsfig-u1-01 .e{stroke:#B1B7C3}html.dark #dsfig-u1-01 .e.hi{stroke:#8FA3FF}html.dark #dsfig-u1-01 .n{fill:#161920;stroke:#E6E8ED}html.dark #dsfig-u1-01 .n.hi{fill:#1E2748;stroke:#8FA3FF}html.dark #dsfig-u1-01 .n.rb-b{fill:#E6E8ED;stroke:#E6E8ED}html.dark #dsfig-u1-01 .n.rb-r{fill:#FF7E71;stroke:#FF7E71}html.dark #dsfig-u1-01 .t{fill:#E6E8ED}html.dark #dsfig-u1-01 .t.inv{fill:#0F1115}html.dark #dsfig-u1-01 .kd{stroke:#E6E8ED}html.dark #dsfig-u1-01 .dot{fill:#E6E8ED}html.dark #dsfig-u1-01 .ann{fill:#8FA3FF}html.dark #dsfig-u1-01 .lbl{fill:#858D9C}html.dark #dsfig-u1-01 .ptr{fill:#8FA3FF}html.dark #dsfig-u1-01 .ah{fill:#B1B7C3}html.dark #dsfig-u1-01 .ah.hi{fill:#8FA3FF}html.dark #dsfig-u1-01 .wl rect{fill:#161920;stroke:#2A2E37}html.dark #dsfig-u1-01 .wl.hi rect{fill:#8FA3FF;stroke:#8FA3FF}html.dark #dsfig-u1-01 .wl.hi .t{fill:#0F1115}</style><defs><marker id="ah1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah" d="M0,1 L9,5 L0,9 z"/></marker><marker id="ahh1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="ah hi" d="M0,1 L9,5 L0,9 z"/></marker></defs><path class="e" d="M59,126 L122.2,126" marker-end="url(#ah1)"/><path class="e" d="M162.2,126 L225.4,126" marker-end="url(#ah1)"/><path class="e" d="M265.4,126 L328.6,126" marker-end="url(#ah1)"/><path class="e" d="M364.2,113.8 L436.7,53.4" marker-end="url(#ah1)"/><path class="e" d="M452.8,59 L452.8,191" marker-end="url(#ah1)"/><path class="e" d="M435.3,204.7 L265.8,134.1" marker-end="url(#ah1)"/><circle class="n" cx="40" cy="126" r="18"/><text class="t" x="40" y="126" dy=".35em" text-anchor="middle">Env</text><circle class="n" cx="143.2" cy="126" r="18"/><text class="t" x="143.2" y="126" dy=".35em" text-anchor="middle">Sen</text><circle class="n" cx="246.4" cy="126" r="18"/><text class="t" x="246.4" y="126" dy=".35em" text-anchor="middle">Fea</text><circle class="n" cx="349.6" cy="126" r="18"/><text class="t" x="349.6" y="126" dy=".35em" text-anchor="middle">Mod</text><circle class="n" cx="452.8" cy="40" r="18"/><text class="t" x="452.8" y="40" dy=".35em" text-anchor="middle">Trn</text><circle class="n" cx="452.8" cy="212" r="18"/><text class="t" x="452.8" y="212" dy=".35em" text-anchor="middle">Evl</text></svg><figcaption style="font-size:.82em;opacity:.72;margin-top:.45rem">Design cycle. Env = environment, Sen = sensing and preprocessing, Fea = feature extraction, Mod = model or knowledge base, Trn = training, Evl = evaluation with feedback</figcaption></figure>

Key points.

The environment supplies the raw patterns, and a sensor with preprocessing converts them into clean, noise-reduced data.
Data collection must give enough representative samples covering all classes and conditions.
Feature extraction chooses measurements that are distinctive, low in number and insensitive to noise, because good features matter more than a clever classifier.
Model choice selects the classifier type, such as a linear, statistical or neural model, and its complexity.
Training uses the learning algorithm to estimate model parameters from the training data, which forms the knowledge base.
Evaluation measures error on unseen test data, and the result is fed back to revise features or model, which is the adaptation loop.
Trade-offs: a complex model fits training data but generalises poorly (overfitting), and richer features raise computational cost.

Answer frame. Open with the definition and name the components; draw the design cycle; then develop points 1-6 in order; close with the trade-off of complexity versus generalisation. For "components of a learning system" stress environment, sensing, feature extraction, learning algorithm, knowledge base and performance evaluation with feedback.

Asked: [7 marks] (Jun 2020) What are different components of a learning system? Asked: [7 marks] (Nov 2023) Explain various design principles of pattern recognition systems.

Classification and clustering

Definition. <mark>Classification assigns patterns to predefined classes using labelled training data, whereas clustering groups unlabelled patterns into clusters by similarity.</mark>

Basis	Classification	Clustering
Supervision	Supervised, uses labels	Unsupervised, no labels
Classes	Fixed and known beforehand	Discovered from the data
Output	Class label of each pattern	Cluster membership
Evaluation	Accuracy against known labels	Cluster validity indices
Example	Spam or not spam, k-NN	Customer segments, K-means

Asked: [7 marks] (Jun 2020) Write the difference between classification and clustering.

Supervised learning

Definition. <mark>Supervised learning builds a model from labelled training pairs $(x_i, y_i)$ so that it can predict the label of a new pattern; unsupervised learning finds structure in unlabelled data $\{x_i\}$.</mark>

Key points.

In supervised learning a teacher supplies the correct class for every training sample.
The goal is to learn a mapping $x \to y$ that generalises to unseen patterns.
Feedback is direct, since the prediction error is measured against the label and used to correct the model.
Typical algorithms are decision trees, naive Bayes, SVM and k-NN, and the tasks are classification and regression.
Unsupervised learning has no teacher, its goal is to discover groups or structure, and it uses clustering algorithms such as K-means.

Basis	Supervised	Unsupervised
Data	Labelled	Unlabelled
Goal	Predict labels	Discover structure
Feedback	Direct, from labels	None
Algorithms	SVM, k-NN, decision tree	K-means, hierarchical
Example	Classifying handwritten digits	Grouping customers by behaviour

Answer frame. Open with both definitions; draw no figure; give the five-row table above with one example each; close with classification versus clustering as the standard pair.

Asked: [7 marks] (Dec 2020, Nov 2023) Differentiate supervised learning and unsupervised learning, with suitable example.

Unsupervised learning and adaptation

Definition. <mark>Learning is the estimation of system parameters from a fixed set of training data, whereas adaptation is the adjustment of an already trained system to a changing environment.</mark>

Basis	Learning	Adaptation
Objective	Estimate parameters from data	Keep performance as conditions change
Time-scale	Offline, before use	Online, during use
Mechanism	Training algorithm	Incremental parameter update
Example	Training a digit classifier	Speech system tuning to a new speaker

Key points.

Unsupervised learning uses unlabelled samples, so the system forms its own groups.
Adaptation lets a deployed system track drift without retraining from scratch.

Asked: [7 marks] (Dec 2020) Differentiate learning and adaptation.

Pattern recognition approaches

Definition. The main approaches are statistical, syntactic (structural), neural and template matching, differing in how a pattern is represented and classified.

Key points.

Statistical PR represents a pattern as a feature vector and classifies it using probability models.
Syntactic PR represents a pattern as primitives combined by grammar rules and classifies it by parsing.
Neural approaches learn the mapping from examples using networks of simple units.
Template matching compares the input with a stored prototype using a similarity measure.

Decision boundaries

Definition. <mark>A discriminant function $g_i(x)$ is a function computed for each class $\omega_i$, and the pattern $x$ is assigned to the class with the largest value; the decision boundary is the surface where two discriminants are equal.</mark>

Formula. Decide $\omega_i$ if $g_i(x) > g_j(x)$ for all $j \ne i$. Boundary between two classes: $g_i(x)=g_j(x)$, or $g(x)=g_1(x)-g_2(x)=0$.

Key points.

For two classes a single function $g(x)$ suffices: choose $\omega_1$ if $g(x)>0$, otherwise $\omega_2$.
The linear case is $g(x)=w^{T}x+w_0$, and its boundary $w^{T}x+w_0=0$ is a hyperplane (a line in two dimensions).
For the Bayes classifier, $g_i(x)=P(\omega_i\mid x)$, which gives the minimum-error rule.

Asked: [7 marks] (Dec 2020) What do you mean by discriminant function?

Decision region

Definition. A decision region $R_i$ is the set of all points $x$ that the classifier assigns to class $\omega_i$.

Key points.

The regions are $R_i=\{x: g_i(x)>g_j(x)\ \forall j\ne i\}$, and together they partition the feature space.
Neighbouring regions are separated by decision boundaries.
Regions may be connected or disjoint, depending on the classifier.

Metric spaces

Definition. ==A metric space is a set $X$ with a distance function $d: X\times X\to\mathbb{R}$ satisfying, for all $x,y,z$: non-negativity $d(x,y)\ge 0$ with $d(x,y)=0$ iff $x=y$; symmetry $d(x,y)=d(y,x)$; and the triangle inequality $d(x,z)\le d(x,y)+d(y,z)$.==

Key points.

Clustering is the unsupervised grouping of patterns so that patterns in one cluster are more similar to each other than to those in other clusters, for example customer segmentation using K-means.
A metric gives a consistent measure of similarity, which clustering and nearest-neighbour methods require.
Examples of metrics are the Euclidean, Manhattan and Minkowski distances.

Asked: [7 marks] (Nov 2023) Define the following: i) Clustering ii) Metric spaces

Distances

Definition. A distance measures the dissimilarity of two patterns $x$ and $y$ in $d$ dimensions.

Formula. Euclidean: $d(x,y)=\sqrt{\sum_{i=1}^{d}(x_i-y_i)^2}$. Manhattan: $\sum_{i=1}^{d}|x_i-y_i|$. Minkowski: $\left(\sum_{i}|x_i-y_i|^{p}\right)^{1/p}$.

Key points.

For $x=(1,2)$ and $y=(4,6)$, Euclidean distance is $\sqrt{9+16}=5$ and Manhattan distance is $3+4=7$.
Minkowski gives Manhattan at $p=1$ and Euclidean at $p=2$.

Last-minute revision

Pattern recognition is the automatic assignment of patterns to classes from measured features.
System stages: sensing, segmentation, feature extraction, classification.
Design cycle: data collection, feature choice, model choice, training, evaluation.
Complex models overfit; good generalisation is the aim.
Classification is supervised with fixed classes; clustering is unsupervised with discovered groups.
Supervised learning uses labelled data with direct feedback; unsupervised uses none.
Learning is offline parameter estimation; adaptation is online adjustment to a changing environment.
Decide $\omega_i$ if $g_i(x)>g_j(x)$; the boundary is $g_i(x)=g_j(x)$.
Linear discriminant: $g(x)=w^{T}x+w_0$.
Metric axioms: non-negativity, symmetry, triangle inequality.
Euclidean distance between $(1,2)$ and $(4,6)$ is 5; Manhattan is 7.

Memory hooks

Design cycle: "Collect, Choose, Choose, Train, Test", then loop back.
Supervised means a teacher gives labels; unsupervised means no teacher.
Learning is before deployment; adaptation is during deployment.
Metric axioms: "Positive, Symmetric, Triangle" (PST).

Coverage checklist

Definitions: Dec 2020 what is pattern recognition.
data sets for Pattern: no past questions.
Application Areas and Examples of pattern recognition: no past questions.
Design principles of pattern recognition system: Jun 2020 components of a learning system; Nov 2023 design principles.
Classification and clustering: Jun 2020 classification versus clustering.
supervised Learning: Dec 2020, Nov 2023 supervised versus unsupervised.
unsupervised learning and adaptation: Dec 2020 learning versus adaptation.
Pattern recognition approaches: no past questions.
Decision Boundaries: Dec 2020 discriminant function.
Decision region: no past questions.
Metric spaces: Nov 2023 define clustering and metric spaces.
distances: no past questions.

Account

Pattern Recognition (CS-503 (B)) - Unit 1 Short Notes

How unit 1 is examined

Definitions

Data sets for Pattern

Application areas and examples of pattern recognition

Design principles of pattern recognition system

Classification and clustering

Supervised learning

Unsupervised learning and adaptation

Pattern recognition approaches

Decision boundaries

Decision region

Metric spaces

Distances

Last-minute revision

Memory hooks

Coverage checklist