Augmented & Virtual Reality (AD-803 (B)) Unit 4 Notes

How unit 4 is examined

This unit covers AR terminology, simple AR, applications, and fiducial markers (detection, pose, types, imperceptible and miniature markers); no past questions are on record, so learn each definition.

Terminology

Not asked since 2022

Definition. Augmented reality (AR) overlays computer-generated virtual objects on the real world, registered in 3D and interactive in real time.

Key points.

Milgram's reality-virtuality continuum runs from the real environment to a fully virtual one, and AR lies near the real end.
Registration means aligning virtual objects exactly with real-world positions.
Tracking follows the camera or user pose so the overlay stays fixed as they move.
A marker (fiducial) is a known visual pattern used for tracking.

Simple augmented reality

Definition. Simple AR captures a camera image, finds a known marker in it, computes its pose and renders a virtual object on top of the live video.

Key points.

The camera frame is the input, and video see-through display shows the real scene mixed with graphics.
The marker gives position and orientation without expensive sensors.
The loop repeats for every frame: capture, detect, estimate pose, render, display.
Occlusion and lighting are usually ignored, so the overlay simply sits on top.

Augmented reality as an emerging technology

Definition. AR is an emerging technology because cheap cameras, powerful mobile processors and sensors now let it run on ordinary phones.

Key points.

Smartphones and tablets carry camera, GPS, compass and gyroscope, so AR reaches mass users.
Head-mounted displays and smart glasses are improving in size, field of view and comfort.
Better computer vision and SDKs (ARToolKit, ARCore, ARKit) lower development effort.
Open issues are tracking accuracy, battery life, social acceptance and privacy.

Augmented reality applications

Definition. AR applications enrich real-world tasks with context-linked virtual information.

Key points.

Education and training show 3D models and step-by-step guidance over real equipment.
Medicine overlays scans on the patient during surgery planning.
Industry uses AR for assembly, maintenance and remote expert help.
Retail, games (Pokemon Go), navigation, advertising and interior design preview products in the user's own setting.

Marker detection

Definition. Marker detection finds and identifies a known fiducial pattern in the camera image.

Key points.

The image is converted to grayscale and thresholded to binary.
Connected regions are found and their contours extracted.
Contours that fit four straight lines with four corners are kept as quadrilateral candidates.
Candidates are rectified and matched against known patterns, and false ones are rejected.

Marker pose

Definition. Marker pose is the position and orientation of the marker relative to the camera.

Key points.

The four detected corners are matched to the known real marker corners.
A homography gives the 3D transformation, which is a rotation $R$ and translation $t$.
Pose is written as a $4\times4$ matrix $T=\begin{bmatrix}R&t\\0&1\end{bmatrix}$, with 6 degrees of freedom.
Rendering uses $T$ so the virtual object follows the marker, and corner error causes jitter.

Marker types and identification: template markers, 2D bar-code markers

Definition. Template markers carry a black-and-white image pattern matched against stored templates, and 2D bar-code markers encode an ID in a grid of black and white cells.

Key points.

Template markers have a thick black border with a custom pattern inside, matched by correlation with each stored template.
Template matching is slow when the library is large, and confusion between similar patterns can occur.
Bar-code markers (ARToolKitPlus, ARTag) store a binary ID in the cells, decoded directly without templates.
Bar-code markers scale to many IDs and can include error checking.

Imperceptible markers: image markers, infrared markers

Definition. Imperceptible markers are hidden from human eyes but detectable by the system, so they do not spoil the scene.

Key points.

Image markers use natural images or pictures as markers, detected by feature points such as SIFT or SURF.
Infrared markers reflect or emit IR light that an IR camera sees but people do not.
IR markers work in the dark and do not disturb the visual design.
They need special cameras or hardware, and natural-image tracking is costlier to compute.

Miniature markers

Definition. Miniature markers are very small fiducials that are unobtrusive but still trackable.

Key points.

Small markers suit small objects, handhelds and crowded scenes.
Because they cover few pixels, detection range is short and pose accuracy is lower.
Higher camera resolution and simple patterns help.
Several small markers can be combined to improve stability.

Discussion on marker use

Definition. Choosing a marker means trading off robustness, size, appearance, cost and speed for the application.

Key points.

Markers give cheap, fast and accurate tracking, but need preparation of the environment.
Detection fails with occlusion, poor lighting, blur and steep viewing angles.
Visible markers can look unattractive, which motivates imperceptible and natural-feature tracking.
Pick bar-code markers for many IDs, template markers for custom looks, and IR or image markers for discreet use.

General marker detection application

Definition. A general marker detection application is a pipeline that detects markers of any type and returns their ID and pose to the AR renderer.

Key points.

The pipeline is capture, preprocess, detect, identify, estimate pose, then render.
Detected markers are identified as template or bar-code types by decoding or matching.
The result is a list of marker IDs with pose matrices for the renderer.
Reliability improves by tracking between frames and filtering out unstable detections.

Last-minute revision

AR overlays virtual objects on the real world, aligned in 3D and interactive in real time.
Registration aligns virtual objects with real positions, and tracking keeps them aligned.
A fiducial marker is a known pattern used to find pose.
Detection order: grayscale, threshold, contour, quadrilateral, identify.
Pose is $R$ and $t$ in a $4\times4$ matrix with 6 DOF.
Template markers use correlation matching, and bar-code markers decode a binary ID.
Imperceptible markers: image (feature points) and infrared (IR camera).
Miniature markers are unobtrusive but have short range.
Markers are cheap and fast but fail under occlusion and bad lighting.

Memory hooks

Loop for every frame: Capture, Detect, Pose, Render (CDPR).
Template equals match, bar-code equals decode.
Imperceptible means seen by camera, not by eye.
Small marker, short range.

Coverage checklist

Terminology: covered, no past questions.
Simple augmented reality: covered, no past questions.
Augmented reality as an emerging technology: covered, no past questions.
Augmented reality applications: covered, no past questions.
Marker detection: covered, no past questions.
Marker pose: covered, no past questions.
Marker types and identification: Template markers, 2D bar-code markers: covered, no past questions.
Imperceptible markers: Image markers, Infrared markers: covered, no past questions.
Miniature markers: covered, no past questions.
Discussion on marker use: covered, no past questions.
General marker detection application: covered, no past questions.

Account

Augmented & Virtual Reality (AD-803 (B)) - Unit 4 Short Notes

How unit 4 is examined

Terminology

Simple augmented reality

Augmented reality as an emerging technology

Augmented reality applications

Marker detection

Marker pose

Marker types and identification: template markers, 2D bar-code markers

Imperceptible markers: image markers, infrared markers

Miniature markers

Discussion on marker use

General marker detection application

Last-minute revision

Memory hooks

Coverage checklist