Introduction to Linear Algebra (CY-401) Unit 3 Notes

Unit 3: Advanced Linear Algebra - Exam-Focused Notes

I. Vector Spaces and Subspace Theory

Subspaces

A subspace $W$ of a vector space $V$ over a field $F$ is a subset that is itself a vector space under the operations of $V$. Closure Properties: $W$ must satisfy:

$\mathbf{0} \in W$
If $\mathbf{u}, \mathbf{v} \in W$, then $\mathbf{u} + \mathbf{v} \in W$
If $\mathbf{u} \in W$ and $\alpha \in F$, then $\alpha \mathbf{u} \in W$

Sum and Intersection

Sum: $$\displaystyle U + W = \{ \mathbf{u} + \mathbf{w} \mid \mathbf{u} \in U, \mathbf{w} \in W \} $$
Intersection: $$\displaystyle U \cap W = \{ \mathbf{v} \mid \mathbf{v} \in U \text{ and } \mathbf{v} \in W \} $$

Both are subspaces.

Direct Sum

$$\displaystyle V = U \oplus W $$ iff:

$$\displaystyle V = U + W $$
$$\displaystyle U \cap W = \{\mathbf{0}\} $$ Uniqueness: If $$\displaystyle V = U_1 \oplus \cdots \oplus U_k $$, every $\mathbf{v} \in V$ has a unique representation $$\displaystyle \mathbf{v} = \mathbf{u}_1 + \cdots + \mathbf{u}_k $$ with $$\displaystyle \mathbf{u}_i \in U_i $$.

[!TIP] Common Mistake

Do not assume $$\displaystyle U \cap W = \{\mathbf{0}\} $$ automatically implies $U+W$ is direct; you must also have $$\displaystyle U+W = V $$ for an internal direct sum decomposition.

Quotient Spaces

For subspace $W \subseteq V$, define equivalence relation: $$\displaystyle \mathbf{x} \sim \mathbf{y} \iff \mathbf{x} - \mathbf{y} \in W $$. Quotient space $$\displaystyle V/W = \{ \mathbf{x} + W \mid \mathbf{x} \in V \} $$ with operations:

$$\displaystyle (\mathbf{x}+W) + (\mathbf{y}+W) = (\mathbf{x}+\mathbf{y})+W $$
$$\displaystyle \alpha(\mathbf{x}+W) = \alpha\mathbf{x}+W $$ Natural projection $\eta: V \to V/W$, $$\displaystyle \eta(\mathbf{x}) = \mathbf{x}+W $$ is linear, surjective, $$\displaystyle \ker \eta = W $$.

Dimension Formula:

$$ \dim(V/W) = \dim V - \dim W $$

Proof sketch: Extend a basis of $W$ to a basis of $V$; the cosets of the added vectors form a basis for $V/W$.

[!TIP] Exam Application

Past papers ask: "Show that quotient space of finitely generated space is finitely generated." Use: if $V$ spanned by $$\displaystyle \{\mathbf{v}_1,\dots,\mathbf{v}_n\} $$, then $V/W$ is spanned by $$\displaystyle \{\mathbf{v}_1+W,\dots,\mathbf{v}_n+W\} $$.

Finitely Generated (Spanning)

$V$ is finitely generated if there exists a finite set $S$ with $$\displaystyle \operatorname{span}(S) = V $$. Equivalent to $V$ having a finite basis.

Topological Property: No Proper Open Subspace

In any inner product space (with metric induced by norm), the only subspace that is both open and closed (clopen) is the whole space and $\{\mathbf{0}\}$. A proper subspace has empty interior. Proof sketch: If $W$ is a proper subspace, pick $\mathbf{v} \notin W$. Project $\mathbf{v}$ onto $W$: $$\displaystyle \mathbf{v} = \mathbf{w} + \mathbf{u} $$ with $\mathbf{u} \perp W$, $\mathbf{u} \neq \mathbf{0}$. Then $$\displaystyle B(\mathbf{v}, \|\mathbf{u}\|/2) \cap W = \emptyset $$, so $W$ has no interior points.

II. Linear Transformations and Operators

Definition & Matrix Representation

$T: V \to W$ is linear if $$\displaystyle T(\alpha\mathbf{u}+\beta\mathbf{v}) = \alpha T(\mathbf{u}) + \beta T(\mathbf{v}) $$ for all $\mathbf{u},\mathbf{v} \in V$, $\alpha,\beta \in F$.

Given bases $$\displaystyle \mathcal{B}_V $$, $$\displaystyle \mathcal{B}_W $$, the matrix representation $$\displaystyle [T]_{\mathcal{B}_V}^{\mathcal{B}_W} $$ has columns as $$\displaystyle T(\mathbf{v}_j) $$ coordinates in $$\displaystyle \mathcal{B}_W $$.

Range (Image) and Null Space (Kernel)

$$\displaystyle \operatorname{Im}(T) = \{ T(\mathbf{v}) \mid \mathbf{v} \in V \} $$
$$\displaystyle \ker(T) = \{ \mathbf{v} \in V \mid T(\mathbf{v}) = \mathbf{0} \} $$

Both are subspaces.

Rank-Nullity Theorem

$$ \dim(\operatorname{Im} T) + \dim(\ker T) = \dim V $$

Proof: Extend a basis of $\ker T$ to a basis of $V$; the images of the added vectors form a basis for $\operatorname{Im} T$.

[!TIP] Core Theorem

This is frequently asked. Remember: $$\displaystyle \operatorname{rank}(T) = \dim(\operatorname{Im} T) $$, $$\displaystyle \operatorname{nullity}(T) = \dim(\ker T) $$.

Composition & Non-commutativity

Composition: $$\displaystyle (S \circ T)(\mathbf{v}) = S(T(\mathbf{v})) $$.

Matrix representation: $$\displaystyle [S \circ T] = [S][T] $$ relative to appropriate bases. Non-commutativity example: On $$\displaystyle \mathbb{R}^2 $$, let

$$ T = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix},\quad U = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}. $$

Then $$\displaystyle TU = 0 $$ but $$\displaystyle UT = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \neq 0 $$.

Linear Transformations on Quotient Spaces

Given $T: V \to V$ with $W$ $T$-invariant ($T(W) \subseteq W$), define $\overline{T}: V/W \to V/W$ by $$\displaystyle \overline{T}(\mathbf{v}+W) = T(\mathbf{v})+W $$. Well-defined and linear.

The natural projection $\eta: V \to V/W$ satisfies $$\displaystyle \eta \circ T = \overline{T} \circ \eta $$.

Invariant Subspaces

$W \subseteq V$ is $T$-invariant if $T(W) \subseteq W$.

Eigenspaces $$\displaystyle E_\lambda = \ker(T-\lambda I) $$ are invariant.
Example: For $$\displaystyle T = \begin{bmatrix} 1 & 0 \\ 0 & 2 \end{bmatrix} $$, both coordinate axes are invariant.

Nilpotent Operators

$T$ is nilpotent if $$\displaystyle T^k = 0 $$ for some $k \in \mathbb{N}$. The smallest such $k$ is the index of nilpotency. Examples:

Differentiation $$\displaystyle D: P_n \to P_n $$, $$\displaystyle D(p)=p' $$: $$\displaystyle D^{n+1}=0 $$.
Commutator: If $A$ nilpotent, then $$\displaystyle T(B)=[A,B]=AB-BA $$ is nilpotent on $$\displaystyle M_n(F) $$.

[!TIP] Past Paper Connection

Questions often ask: "If $A$ nilpotent, prove $$\displaystyle T(B)=[A,B] $$ is nilpotent." Use: $$\displaystyle T^k(B) = \sum_{i=0}^k (-1)^i \binom{k}{i} A^{k-i} B A^i $$. Since $$\displaystyle A^k=0 $$, each term has $A$ to power $\ge k-i$ or $i$, so for $k$ large enough, all terms vanish.

III. Dual Spaces and Annihilators

Dual Space

$$\displaystyle V^* = \mathcal{L}(V, F) $$ is the dual space: all linear functionals $f: V \to F$.

If $$\displaystyle \mathcal{B} = \{\mathbf{v}_1,\dots,\mathbf{v}_n\} $$ basis of $V$, the dual basis $$\displaystyle \mathcal{B}^* = \{f_1,\dots,f_n\} \subseteq V^* $$ defined by $$\displaystyle f_i(\mathbf{v}_j) = \delta_{ij} $$.

Annihilator

For $W \subseteq V$, the annihilator is:

$$ W^0 = \{ f \in V^* \mid f(\mathbf{w}) = 0 \text{ for all } \mathbf{w} \in W \} $$

$$\displaystyle W^0 $$ is a subspace of $$\displaystyle V^* $$.

Dimension Formula

$$ \dim W + \dim W^0 = \dim V $$

Proof: Let $$\displaystyle \dim V = n $$, $$\displaystyle \dim W = k $$. Extend basis $$\displaystyle \{\mathbf{w}_1,\dots,\mathbf{w}_k\} $$ of $W$ to basis $$\displaystyle \{\mathbf{w}_1,\dots,\mathbf{w}_k, \mathbf{v}_{k+1},\dots,\mathbf{v}_n\} $$ of $V$. Then $$\displaystyle \{f_{k+1},\dots,f_n\} $$ (dual basis) spans $$\displaystyle W^0 $$.

Double Dual

$$\displaystyle V^{**} = (V^*)^* $$ is the double dual. There is a natural isomorphism $$\displaystyle \phi: V \to V^{**} $$ given by $$\displaystyle \phi(\mathbf{v})(f) = f(\mathbf{v}) $$. If $V$ finite-dimensional, $\phi$ is an isomorphism.

[!TIP] Key Insight

Annihilators help convert subspace problems in $V$ to subspace problems in $$\displaystyle V^* $$. Past papers ask: "Show $$\displaystyle U=V \iff U^0=\{0\} $$." Proof: If $U \neq V$, pick $\mathbf{v} \notin U$, extend to basis, define $$\displaystyle f \in V^* $$ with $f(\mathbf{v}) \neq 0$, $$\displaystyle f|_U=0 $$ so $$\displaystyle f \in U^0 $$.

IV. Polynomials and Matrix Functions

Characteristic Polynomial

For $$\displaystyle A \in M_n(F) $$, the characteristic polynomial is:

$$ p_A(t) = \det(tI - A) $$

Eigenvalues are roots of $$\displaystyle p_A(t)=0 $$.

For triangular $A$, eigenvalues are diagonal entries (since $\det(tI-A)$ is product of $$\displaystyle (t-a_{ii}) $$).

Minimal Polynomial

The minimal polynomial $$\displaystyle m_A(t) $$ is the monic polynomial of least degree such that $$\displaystyle m_A(A)=0 $$. Properties:

$$\displaystyle m_A(t) $$ exists and is unique.
$$\displaystyle m_A(t) $$ divides any polynomial $f(t)$ with $$\displaystyle f(A)=0 $$.
$$\displaystyle m_A(t) $$ divides $$\displaystyle p_A(t) $$ (Cayley-Hamilton implies $$\displaystyle p_A(A)=0 $$).
$$\displaystyle m_A(t) $$ and $$\displaystyle p_A(t) $$ have the same distinct roots.

Cayley-Hamilton Theorem

Every square matrix satisfies its own characteristic polynomial:

$$ p_A(A) = 0 $$

Proof sketch: Consider $$\displaystyle p_A(t) = \det(tI-A) $$. For $n \times n$, $$\displaystyle p_A(A) $$ is not simply $$\displaystyle \det(AI-A)=0 $$; use adjugate matrix: $$\displaystyle (tI-A)\operatorname{adj}(tI-A) = p_A(t)I $$. Substitute $$\displaystyle t=A $$: $$\displaystyle 0 \cdot \operatorname{adj}(0) = p_A(A)I $$, but careful: treat as polynomial identity in $t$, then substitute $A$; since matrices commute with polynomials in themselves, $$\displaystyle p_A(A)=0 $$.

Applications:

Compute $$\displaystyle A^{-1} $$: if $$\displaystyle p_A(t) = t^n + c_{n-1}t^{n-1} + \cdots + c_0 $$, then $$\displaystyle A^{n} + c_{n-1}A^{n-1} + \cdots + c_0 I = 0 $$. If $$\displaystyle c_0 \neq 0 $$, $$\displaystyle A^{-1} = -\frac{1}{c_0}(A^{n-1} + c_{n-1}A^{n-2} + \cdots + c_1 I) $$.
Compute powers $$\displaystyle A^k $$ for $k \ge n$ by reducing using $$\displaystyle p_A(A)=0 $$.

Companion Matrix

For monic polynomial $$\displaystyle p(t) = t^n + a_{n-1}t^{n-1} + \cdots + a_0 $$, the companion matrix is:

$$ C(p) = \begin{bmatrix} 0 & 0 & \cdots & 0 & -a_0 \\ 1 & 0 & \cdots & 0 & -a_1 \\ 0 & 1 & \cdots & 0 & -a_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 & -a_{n-1} \end{bmatrix} $$

Its characteristic polynomial is $p(t)$, and if $p(t)$ is the minimal polynomial of $A$, then $A$ is similar to $C(p)$.

[!TIP] Past Paper Example

"Find char and min poly for $$\displaystyle B=\begin{bmatrix}2&1&0\\0&2&0\\0&0&3\end{bmatrix} $$."

Char poly: $$\displaystyle (t-2)^2(t-3) $$.

Min poly: $(t-2)(t-3)$ since $$\displaystyle (B-2I)(B-3I)=0 $$ but neither factor alone zero.

V. Eigenvalues, Eigenvectors, and Diagonalization

Definitions

$\lambda \in F$ is an eigenvalue of $T$ (or $A$) if $\exists \mathbf{v} \neq \mathbf{0}$ with $$\displaystyle T(\mathbf{v}) = \lambda \mathbf{v} $$. Such $\mathbf{v}$ is an eigenvector. Eigenspace: $$\displaystyle E_\lambda = \ker(T - \lambda I) $$.

Multiplicities

Algebraic multiplicity (AM): multiplicity of $\lambda$ as root of $$\displaystyle p_A(t) $$.
Geometric multiplicity (GM): $$\displaystyle \dim E_\lambda = \dim \ker(A-\lambda I) $$.

Always $1 \le \text{GM} \le \text{AM}$.

Diagonalization Criteria

$A$ is diagonalizable iff $V$ has a basis of eigenvectors of $A$. Sufficient: $A$ has $n$ distinct eigenvalues $\implies$ diagonalizable (since GM=1=AM for each). Necessary and sufficient: For each eigenvalue $\lambda$, $$\displaystyle \text{GM}(\lambda) = \text{AM}(\lambda) $$.

[!TIP] Quick Check

If sum of GM over distinct eigenvalues equals $n$, then $A$ diagonalizable.

Spectral Theorems

Real Symmetric Matrices ($$\displaystyle A = A^T $$):

All eigenvalues are real.
Eigenvectors corresponding to distinct eigenvalues are orthogonal.
$A$ is orthogonally diagonalizable: $\exists$ orthogonal $Q$ ($$\displaystyle Q^TQ=I $$) such that $$\displaystyle Q^T A Q = D $$ diagonal.

Complex Hermitian Matrices ($$\displaystyle A = A^* $$, $$\displaystyle A^* = \overline{A}^T $$):

All eigenvalues are real.
Eigenvectors for distinct eigenvalues are orthogonal.
$A$ is unitarily diagonalizable: $\exists$ unitary $U$ ($$\displaystyle U^*U=I $$) such that $$\displaystyle U^* A U = D $$.

Eigenvalues of $$\displaystyle A^*A $$:

For any $$\displaystyle A \in M_n(\mathbb{C}) $$, $$\displaystyle A^*A $$ is Hermitian, so eigenvalues are real and $\ge 0$.

Moreover, $$\displaystyle A^*A $$ is unitarily diagonalizable.

[!TIP] Proof Sketch for $$\displaystyle A^*A $$ eigenvalues real/nonnegative:

If $$\displaystyle A^*A\mathbf{v} = \lambda \mathbf{v} $$, $\mathbf{v} \neq 0$, then $$\displaystyle \langle A^*A\mathbf{v}, \mathbf{v} \rangle = \langle A\mathbf{v}, A\mathbf{v} \rangle = \|A\mathbf{v}\|^2 \ge 0 $$. LHS $$\displaystyle = \lambda \langle \mathbf{v}, \mathbf{v} \rangle = \lambda \|\mathbf{v}\|^2 $$, so $$\displaystyle \lambda = \frac{\|A\mathbf{v}\|^2}{\|\mathbf{v}\|^2} \ge 0 $$.

VI. Canonical Forms

Rational Canonical Form (Frobenius Normal Form)

Based on invariant factors of $A$.

For each distinct eigenvalue $\lambda$, consider the primary component $$\displaystyle V_\lambda = \ker(A-\lambda I)^m $$ for large $m$.

Within $$\displaystyle V_\lambda $$, decompose into cyclic subspaces generated by vectors $$\displaystyle \mathbf{v}, (A-\lambda I)\mathbf{v}, \dots, (A-\lambda I)^{k-1}\mathbf{v} $$.

Matrix of $A$ on such cyclic subspace (basis as above) is a companion matrix of $$\displaystyle (t-\lambda)^k $$.

RCF is block diagonal with these companion matrices, unique up to ordering.

Jordan Canonical Form

For $A$ over $\mathbb{C}$ (or algebraically closed field), JCF exists. Jordan block $$\displaystyle J_k(\lambda) $$: $k \times k$ matrix with $\lambda$ on diagonal, 1 on superdiagonal, 0 elsewhere.

$$ J_k(\lambda) = \begin{bmatrix} \lambda & 1 & & \\ & \lambda & \ddots & \\ & & \ddots & 1 \\ & & & \lambda \end{bmatrix} $$

Jordan matrix $J$ is block diagonal with Jordan blocks. $A$ is similar to $J$: $$\displaystyle P^{-1}AP = J $$.

Diagonalizable iff all Jordan blocks are $1 \times 1$.

Computing Jordan Blocks

For eigenvalue $\lambda$, let $$\displaystyle m_k = \dim \ker(A-\lambda I)^k $$.

Number of Jordan blocks for $\lambda$ = $$\displaystyle m_1 $$.
Size of largest block = smallest $k$ with $$\displaystyle m_k = \text{AM}(\lambda) $$.
Number of blocks of size $\ge k$ = $$\displaystyle m_k - m_{k-1} $$.

[!TIP] Example from Past Paper

Matrix $$\displaystyle A = \begin{bmatrix}6&1&0\\0&6&1\\0&0&6\end{bmatrix} $$: eigenvalue 6 only.

$$\displaystyle A-6I = \begin{bmatrix}0&1&0\\0&0&1\\0&0&0\end{bmatrix} $$, $\ker(A-6I)$ spanned by $$\displaystyle e_1 $$ → $$\displaystyle m_1=1 $$.

$$\displaystyle (A-6I)^2 = \begin{bmatrix}0&0&1\\0&0&0\\0&0&0\end{bmatrix} $$, $$\displaystyle \ker(A-6I)^2 $$ spanned by $$\displaystyle e_1,e_2 $$ → $$\displaystyle m_2=2 $$.

$$\displaystyle (A-6I)^3=0 $$, $$\displaystyle \ker(A-6I)^3 = \mathbb{C}^3 $$ → $$\displaystyle m_3=3 $$.

Blocks: one block of size 3 (since $$\displaystyle m_1=1 $$, $$\displaystyle m_2-m_1=1 $$, $$\displaystyle m_3-m_2=1 $$). So JCF is $$\displaystyle J_3(6) $$.

Primary Decomposition Theorem

Let $$\displaystyle p_A(t) = \prod_{i=1}^k p_i(t)^{e_i} $$ be primary factorization (distinct irreducible $$\displaystyle p_i $$).

Then:

$$ V = V_1 \oplus \cdots \oplus V_k $$

where $$\displaystyle V_i = \ker p_i(A)^{e_i} $$ are $A$-invariant, and $$\displaystyle A|_{V_i} $$ has minimal polynomial $$\displaystyle p_i(t)^{e_i} $$.

This decomposition is unique.

Corollary: $A$ diagonalizable iff each $$\displaystyle p_i(t) $$ is linear (i.e., all eigenvalues in $F$) and each $$\displaystyle e_i=1 $$.

VII. Inner Product Spaces

Definition

An inner product on $V$ over $F$ ($$\displaystyle F=\mathbb{R} $$ or $\mathbb{C}$) is a map $\langle \cdot, \cdot \rangle: V \times V \to F$ such that for all $\mathbf{u},\mathbf{v},\mathbf{w} \in V$, $\alpha,\beta \in F$:

Linearity in first argument: $$\displaystyle \langle \alpha\mathbf{u}+\beta\mathbf{v}, \mathbf{w} \rangle = \alpha\langle\mathbf{u},\mathbf{w}\rangle + \beta\langle\mathbf{v},\mathbf{w}\rangle $$
Conjugate symmetry: $$\displaystyle \langle \mathbf{u}, \mathbf{v} \rangle = \overline{\langle \mathbf{v}, \mathbf{u} \rangle} $$ (over $\mathbb{R}$, symmetry)
Positive-definiteness: $\langle \mathbf{v}, \mathbf{v} \rangle \ge 0$, with equality iff $$\displaystyle \mathbf{v}=\mathbf{0} $$.

Examples:

$$\displaystyle \mathbb{R}^n $$: $$\displaystyle \langle \mathbf{x}, \mathbf{y} \rangle = \mathbf{x}^T\mathbf{y} $$
$C[a,b]$: $$\displaystyle \langle f,g \rangle = \int_a^b f(x)\overline{g(x)} dx $$
$$\displaystyle M_n(\mathbb{R}) $$: $$\displaystyle \langle A,B \rangle = \operatorname{tr}(A^TB) $$

Norm & Inequalities

Norm: $$\displaystyle \|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle} $$.

Cauchy-Schwarz: $$\displaystyle |\langle \mathbf{u}, \mathbf{v} \rangle| \le \|\mathbf{u}\| \|\mathbf{v}\| $$, equality iff $\mathbf{u},\mathbf{v}$ linearly dependent.
Triangle inequality: $\|\mathbf{u}+\mathbf{v}\| \le \|\mathbf{u}\|+\|\mathbf{v}\|$.
Parallelogram law: $$\displaystyle \|\mathbf{u}+\mathbf{v}\|^2 + \|\mathbf{u}-\mathbf{v}\|^2 = 2\|\mathbf{u}\|^2 + 2\|\mathbf{v}\|^2 $$.

Orthogonality & Projection

$\mathbf{u} \perp \mathbf{v}$ iff $$\displaystyle \langle \mathbf{u}, \mathbf{v} \rangle = 0 $$. Orthogonal complement: $$\displaystyle W^\perp = \{ \mathbf{v} \in V \mid \langle \mathbf{v}, \mathbf{w} \rangle = 0 \ \forall \mathbf{w} \in W \} $$. Projection theorem: For finite-dimensional $V$ and subspace $W$, every $\mathbf{v} \in V$ decomposes uniquely as $$\displaystyle \mathbf{v} = \mathbf{w} + \mathbf{w}^\perp $$ with $\mathbf{w} \in W$, $$\displaystyle \mathbf{w}^\perp \in W^\perp $$. The map $\mathbf{v} \mapsto \mathbf{w}$ is the orthogonal projection onto $W$.

Gram-Schmidt Orthogonalization

Given linearly independent $$\displaystyle \{\mathbf{v}_1,\dots,\mathbf{v}_n\} $$, construct orthogonal basis $$\displaystyle \{\mathbf{u}_1,\dots,\mathbf{u}_n\} $$:

$$ \begin{aligned} \mathbf{u}_1 &= \mathbf{v}_1 \\ \mathbf{u}_2 &= \mathbf{v}_2 - \frac{\langle \mathbf{v}_2, \mathbf{u}_1 \rangle}{\langle \mathbf{u}_1, \mathbf{u}_1 \rangle} \mathbf{u}_1 \\ \mathbf{u}_3 &= \mathbf{v}_3 - \frac{\langle \mathbf{v}_3, \mathbf{u}_1 \rangle}{\langle \mathbf{u}_1, \mathbf{u}_1 \rangle} \mathbf{u}_1 - \frac{\langle \mathbf{v}_3, \mathbf{u}_2 \rangle}{\langle \mathbf{u}_2, \mathbf{u}_2 \rangle} \mathbf{u}_2 \\ &\vdots \end{aligned} $$

Normalize to get orthonormal basis: $$\displaystyle \mathbf{e}_i = \mathbf{u}_i / \|\mathbf{u}_i\| $$.

[!TIP] Past Paper Requirement

"State and prove Gram-Schmidt." Provide algorithm and proof that resulting vectors are orthogonal and span same space.

Adjoint of a Linear Operator

$T: V \to V$ (inner product space). The adjoint $$\displaystyle T^* $$ satisfies:

$$ \langle T(\mathbf{u}), \mathbf{v} \rangle = \langle \mathbf{u}, T^*(\mathbf{v}) \rangle \quad \forall \mathbf{u},\mathbf{v} \in V. $$

If orthonormal basis $\mathcal{B}$, then $$\displaystyle [T^*]_{\mathcal{B}} = ([T]_{\mathcal{B}})^* $$ (conjugate transpose).

Self-Adjoint (Hermitian) & Unitary Operators

Self-adjoint: $$\displaystyle T = T^* $$. Matrix: $$\displaystyle A = A^* $$ (Hermitian). Eigenvalues real, eigenvectors orthogonal.
Unitary (or orthogonal in real case): $$\displaystyle T^*T = TT^* = I $$. Preserves inner product: $$\displaystyle \langle T\mathbf{u}, T\mathbf{v} \rangle = \langle \mathbf{u}, \mathbf{v} \rangle $$. Eigenvalues have $$\displaystyle |\lambda|=1 $$.
Normal: $$\displaystyle T T^* = T^* T $$. Spectral theorem: $T$ normal $\iff$ $V$ has orthonormal basis of eigenvectors of $T$ $\iff$ $T$ unitarily diagonalizable.

Projections

$P$ is a projection if $$\displaystyle P^2 = P $$.

Orthogonal projection iff $P$ is self-adjoint ($$\displaystyle P=P^* $$).
$P$ orthogonal projection onto $W$ along $$\displaystyle W^\perp $$.

VIII. Bilinear Forms

Definition & Matrix Representation

A bilinear form on $V$ over $F$ is a map $f: V \times V \to F$ linear in each argument.

Given basis $$\displaystyle \mathcal{B} = \{\mathbf{v}_1,\dots,\mathbf{v}_n\} $$, the matrix of $f$ is $$\displaystyle A = (a_{ij}) $$ where $$\displaystyle a_{ij} = f(\mathbf{v}_j, \mathbf{v}_i) $$ (note order!). Then for $$\displaystyle \mathbf{x} = \sum x_i \mathbf{v}_i $$, $$\displaystyle \mathbf{y} = \sum y_j \mathbf{v}_j $$:

$$ f(\mathbf{x}, \mathbf{y}) = \mathbf{x}^T A \mathbf{y} \quad \text{(if coordinates as column vectors)} $$

Change of Basis

If $P$ is change-of-basis matrix (new basis columns in old basis), then new matrix $$\displaystyle A' = P^T A P $$. Congruence ($$\displaystyle A' = P^T A P $$) vs similarity ($$\displaystyle A' = P^{-1} A P $$).

Rank & Non-degenerate

Rank of $f$ = rank of its matrix (independent of basis). $f$ is non-degenerate iff its matrix is invertible (or: if $$\displaystyle f(\mathbf{x},\mathbf{y})=0 $$ $\forall \mathbf{y}$ implies $$\displaystyle \mathbf{x}=0 $$).

Symmetric Bilinear Forms

$$\displaystyle f(\mathbf{x},\mathbf{y}) = f(\mathbf{y},\mathbf{x}) $$ $\iff$ $$\displaystyle A = A^T $$.

Over $F$ with $\operatorname{char} \neq 2$, can diagonalize: $\exists$ basis where $A$ is diagonal. Sylvester's Law of Inertia (over $\mathbb{R}$): For real symmetric $A$, numbers of positive, negative, zero eigenvalues (inertia) are invariant under congruence. Signature $(p,q)$ (positive/negative counts) is an invariant.

Skew-Symmetric Bilinear Forms

$$\displaystyle f(\mathbf{x},\mathbf{y}) = -f(\mathbf{y},\mathbf{x}) $$ $\iff$ $$\displaystyle A = -A^T $$.

Over $\mathbb{R}$ (or any field not of characteristic 2), can put into canonical form:

$$ \begin{bmatrix} 0 & 1 & & & \\ -1 & 0 & & & \\ & & \ddots & & \\ & & & 0 & 1 \\ & & & -1 & 0 \end{bmatrix} $$

with $$\displaystyle \frac{1}{2}\operatorname{rank}(f) $$ blocks of $\begin{bmatrix}0&1\\-1&0\end{bmatrix}$.

Dimension of space of skew-symmetric forms on $$\displaystyle \mathbb{R}^n $$ is $$\displaystyle \frac{n(n-1)}{2} $$.

Rank 1 Bilinear Forms

$f$ has rank 1 iff $$\displaystyle f(\mathbf{x},\mathbf{y}) = \phi(\mathbf{x})\psi(\mathbf{y}) $$ for some linear functionals $$\displaystyle \phi, \psi \in V^* $$, not both zero. Proof: Rank 1 matrix $$\displaystyle A = \mathbf{u} \mathbf{v}^T $$ for vectors $\mathbf{u},\mathbf{v}$; then $$\displaystyle f(\mathbf{x},\mathbf{y}) = \mathbf{x}^T (\mathbf{u} \mathbf{v}^T) \mathbf{y} = (\mathbf{x}^T\mathbf{u})(\mathbf{v}^T\mathbf{y}) = \phi(\mathbf{x})\psi(\mathbf{y}) $$ with $$\displaystyle \phi(\mathbf{x})=\mathbf{x}^T\mathbf{u} $$, $$\displaystyle \psi(\mathbf{y})=\mathbf{v}^T\mathbf{y} $$.

Group-Invariant Bilinear Forms

A bilinear form $f$ on $V$ is $G$-invariant if $$\displaystyle f(g\mathbf{x}, g\mathbf{y}) = f(\mathbf{x},\mathbf{y}) $$ for all $g \in G \subseteq GL(V)$. Example: On $$\displaystyle \mathbb{R}^n $$, $O(n)$-invariant forms are scalar multiples of the standard dot product.

On $$\displaystyle M_n(\mathbb{R}) $$, $O(n)$-invariant form: $$\displaystyle B(A,B) = \operatorname{tr}(A^T B) $$. Verification: For $Q \in O(n)$, $$\displaystyle B(QAQ^T, QBQ^T) = \operatorname{tr}((QAQ^T)^T (QBQ^T)) = \operatorname{tr}(A^T Q^T Q B Q^T Q) = \operatorname{tr}(A^T B) = B(A,B) $$.

[!TIP] Past Paper Task

"Find all bilinear forms on $n \times 1$ matrices invariant under $O(n,\mathbb{R})$."

Answer: $$\displaystyle f(\mathbf{x},\mathbf{y}) = c \mathbf{x}^T \mathbf{y} $$ for some $c \in \mathbb{R}$.

IX. Special Topics and Applications

Nilpotent Operators (Recap)

Differentiation on $$\displaystyle P_n $$: $$\displaystyle D^{n+1}=0 $$.
Commutator: If $A$ nilpotent, $$\displaystyle T(B)=[A,B] $$ nilpotent.
Similarity invariants for nilpotent operators: Jordan blocks for eigenvalue 0. Two nilpotent operators similar iff they have same Jordan form (same block sizes).

Topological: No Proper Open Subspace

In any inner product space (with norm topology), the only subspaces that are open are $V$ and $\{\mathbf{0}\}$. Proof: If $W$ is a proper subspace, pick $\mathbf{v} \notin W$. Let $\mathbf{u}$ be orthogonal projection of $\mathbf{v}$ onto $W$; then $\mathbf{v}-\mathbf{u} \perp W$, $$\displaystyle \|\mathbf{v}-\mathbf{u}\|>0 $$. For any $\mathbf{w} \in W$, $$\displaystyle \|\mathbf{v}-\mathbf{w}\|^2 = \|\mathbf{v}-\mathbf{u}\|^2 + \|\mathbf{u}-\mathbf{w}\|^2 \ge \|\mathbf{v}-\mathbf{u}\|^2 $$. So $$\displaystyle B(\mathbf{v}, \|\mathbf{v}-\mathbf{u}\|/2) \cap W = \emptyset $$. Hence $W$ has empty interior, not open.

Direct Sum Decompositions

Internal: $$\displaystyle V = W_1 \oplus \cdots \oplus W_k $$ means $$\displaystyle V = \sum W_i $$ and $$\displaystyle W_i \cap \sum_{j \neq i} W_j = \{0\} $$.
External: $$\displaystyle W_1 \oplus \cdots \oplus W_k = \{ (\mathbf{w}_1,\dots,\mathbf{w}_k) \mid \mathbf{w}_i \in W_i \} $$ with componentwise operations.
Used in primary decomposition and Jordan form construction.

Annihilator Applications

$$\displaystyle \dim W + \dim W^0 = \dim V $$.
$$\displaystyle (U+W)^0 = U^0 \cap W^0 $$.
$$\displaystyle (U \cap W)^0 = U^0 + W^0 $$ (under finite-dimensionality).
$$\displaystyle W = (W^0)^0 $$ if $W$ finite-dimensional.

Summary of Key Theorems (Boxed)

Rank-Nullity Theorem:

$$ \boxed{\dim(\operatorname{Im} T) + \dim(\ker T) = \dim V} $$

Cayley-Hamilton Theorem:

$$ \boxed{p_A(A) = 0} $$

Spectral Theorem (Real Symmetric):

$$ \boxed{A = A^T \implies \exists \text{ orthogonal } Q \text{ s.t. } Q^T A Q = D \text{ diagonal}} $$

Spectral Theorem (Complex Hermitian):

$$ \boxed{A = A^* \implies \exists \text{ unitary } U \text{ s.t. } U^* A U = D} $$

Primary Decomposition Theorem:

$$ \boxed{V = \bigoplus_{\lambda} E_\lambda \text{ (generalized eigenspaces) if minimal polynomial splits}} $$

Dimension Formula for Quotient:

$$ \boxed{\dim(V/W) = \dim V - \dim W} $$

Annihilator Dimension:

$$ \boxed{\dim W + \dim W^0 = \dim V} $$

Diagonalization Criterion:

$$ \boxed{A \text{ diagonalizable } \iff \forall \lambda,\ \text{GM}(\lambda) = \text{AM}(\lambda)} $$

[!TIP] Final Exam Strategy

For proofs, write clear steps: state what you’re proving, use definitions, apply theorems (like rank-nullity), conclude.

For computations (Jordan form, diagonalization), systematically find eigenvalues, then eigenvectors/nullities, then construct basis.

For short notes, define, state key properties/theorems, give one example.

Always check: for symmetric/Hermitian matrices, eigenvalues real; for unitary, eigenvalues on unit circle; for normal, unitarily diagonalizable.