Introduction to Linear Algebra (CY-401) Unit 2 Notes

I. Vector Spaces and Subspaces

Vector Space: A set $V$ over a field $F$ with operations $+$ and $\cdot$ satisfying:

$(V,+)$ is an abelian group.
$$\displaystyle a(\mathbf{u}+\mathbf{v}) = a\mathbf{u} + a\mathbf{v} $$, $$\displaystyle (a+b)\mathbf{v} = a\mathbf{v} + b\mathbf{v} $$, $$\displaystyle a(b\mathbf{v}) = (ab)\mathbf{v} $$, $$\displaystyle 1\mathbf{v} = \mathbf{v} $$ for $a,b \in F$, $\mathbf{u},\mathbf{v} \in V$.

Subspace: $W \subseteq V$ is a subspace iff:

$\mathbf{0} \in W$,
$$\displaystyle \mathbf{u},\mathbf{v} \in W \implies \mathbf{u}+\mathbf{v} \in W $$,
$a \in F, \mathbf{u} \in W \implies a\mathbf{u} \in W$.

[!TIP] To prove a subset is a subspace, verify closure under addition and scalar multiplication (zero vector follows).

Sum & Intersection: $$\displaystyle U_1+U_2 = \{u_1+u_2 \mid u_i \in U_i\} $$, $$\displaystyle U_1 \cap U_2 $$ is always a subspace.

Direct Sum (Internal): $$\displaystyle V = U_1 \oplus \cdots \oplus U_n $$ iff every $\mathbf{v} \in V$ can be written uniquely as $$\displaystyle \mathbf{v} = u_1 + \cdots + u_n $$ with $$\displaystyle u_i \in U_i $$. Equivalent to $$\displaystyle V = U_1 + \cdots + U_n $$ and $$\displaystyle U_i \cap \sum_{j \neq i} U_j = \{0\} $$ for all $i$.

Dimension Formula (for two subspaces):

$$ \dim(U_1 + U_2) = \dim U_1 + \dim U_2 - \dim(U_1 \cap U_2) $$

Proof sketch: Extend a basis of $$\displaystyle U_1 \cap U_2 $$ to bases of $$\displaystyle U_1 $$ and $$\displaystyle U_2 $$, then union gives basis of $$\displaystyle U_1+U_2 $$.

II. Quotient Spaces and Direct Sums (External)

Quotient Space $V/W$:

Cosets: $$\displaystyle x+W = \{x+w \mid w \in W\} $$.
Operations: $$\displaystyle (x+W)+(y+W) = (x+y)+W $$, $$\displaystyle a(x+W) = ax+W $$. Well-defined because $W$ is subspace.
Natural Projection: $\eta: V \to V/W$, $$\displaystyle \eta(x)=x+W $$.

Linearity: $$\displaystyle \eta(x+y)=x+y+W = (x+W)+(y+W) = \eta(x)+\eta(y) $$, $$\displaystyle \eta(ax)=ax+W = a(x+W)=a\eta(x) $$.
Dimension Theorem: $$\displaystyle \boxed{\dim(V/W) = \dim V - \dim W} $$.

Proof: If $$\displaystyle \{u_1,\dots,u_k\} $$ basis for $W$, extend to basis $$\displaystyle \{u_1,\dots,u_k, v_1,\dots,v_m\} $$ for $V$. Then $$\displaystyle \{v_1+W, \dots, v_m+W\} $$ basis for $V/W$.
If $V$ finitely generated, then $V/W$ is finitely generated.

Direct Sum (External): $$\displaystyle V = V_1 \oplus V_2 $$ iff $$\displaystyle V = V_1+V_2 $$ and $$\displaystyle V_1 \cap V_2 = \{0\} $$.

Complementary Subspaces: For any subspace $$\displaystyle V_0 \subseteq V $$, there exists $$\displaystyle V_1 $$ such that $$\displaystyle V = V_0 \oplus V_1 $$.

Proof: Extend basis of $$\displaystyle V_0 $$ to basis of $V$; let $$\displaystyle V_1 $$ be span of added vectors.

III. Dual Spaces and Annihilators

Dual Space: $$\displaystyle V^* = \{ f: V \to F \mid f \text{ linear} \} $$. It is a vector space over $F$ with $$\displaystyle (f+g)(v)=f(v)+g(v) $$, $$\displaystyle (af)(v)=a f(v) $$.

Annihilator of $W \subseteq V$:

$$ W^0 = \{ f \in V^* \mid f(w)=0 \; \forall w \in W \}. $$

$$\displaystyle W^0 $$ is a subspace of $$\displaystyle V^* $$.
Dimension Formula: $$\displaystyle \boxed{\dim W + \dim W^0 = \dim V} $$.

Proof: Let $$\displaystyle \{w_1,\dots,w_k\} $$ basis for $W$, extend to basis $$\displaystyle \{w_1,\dots,w_k, v_1,\dots,v_m\} $$ for $V$. Define dual basis $$\displaystyle \{f_1,\dots,f_{k+m}\} $$ for $$\displaystyle V^* $$. Then $$\displaystyle W^0 = \operatorname{span}\{f_{k+1},\dots,f_{k+m}\} $$, so $$\displaystyle \dim W^0 = m = \dim V - \dim W $$.
For finite-dimensional $V$: $$\displaystyle U = V \iff U^0 = \{0\} $$.

Double Dual: Map $$\displaystyle \psi: V \to V^{**} $$ defined by $$\displaystyle \psi(v)(f) = f(v) $$ is an isomorphism.

IV. Linear Transformations

Definition: $T: V \to W$ is linear if $$\displaystyle T(av+bw) = aT(v)+bT(w) $$ for all $a,b \in F$, $v,w \in V$.

Matrix Representation: Given bases $$\displaystyle \mathcal{B}=\{v_1,\dots,v_n\} $$ for $V$, $$\displaystyle \mathcal{B}'=\{w_1,\dots,w_m\} $$ for $W$, the matrix $$\displaystyle [T]_{\mathcal{B},\mathcal{B}'} $$ has columns $$\displaystyle [T(v_j)]_{\mathcal{B}'} $$.

Change of Basis: If $P$ is change-of-basis matrix from $\mathcal{B}$ to $\mathcal{B}'$, then $$\displaystyle [T]_{\mathcal{B}'} = P^{-1} [T]_{\mathcal{B}} P $$.

Rank-Nullity Theorem:

$$ \dim(\operatorname{Im} T) + \dim(\ker T) = \dim V. $$

Proof: Let $$\displaystyle \{u_1,\dots,u_k\} $$ basis for $\ker T$, extend to basis $$\displaystyle \{u_1,\dots,u_k, v_1,\dots,v_m\} $$ for $V$. Then $$\displaystyle \{T(v_1),\dots,T(v_m)\} $$ is basis for $\operatorname{Im} T$.

Algebra of Operators: $\mathcal{L}(V)$ is a ring under addition and composition.

[!TIP] $$\displaystyle TU=0 $$ does not imply $$\displaystyle UT=0 $$. Example on $$\displaystyle \mathbb{R}^2 $$:

$$\displaystyle T(x,y)=(y,0) $$, $$\displaystyle U(x,y)=(0,x) $$. Then $$\displaystyle TU(x,y)=T(0,x)=(x,0) $$? Wait, compute: $$\displaystyle U(x,y)=(0,x) $$, then $$\displaystyle T(0,x)=(x,0) $$, so $TU \neq 0$. Actually, $$\displaystyle TU(x,y)=T(0,x)=(x,0) $$, not zero. To get $$\displaystyle TU=0 $$, adjust: let $$\displaystyle T(x,y)=(y,0) $$, $$\displaystyle U(x,y)=(0,0) $$? That gives both zero. Need $$\displaystyle TU=0 $$ but $UT \neq 0$. Let $$\displaystyle T(x,y)=(0,x) $$, $$\displaystyle U(x,y)=(y,0) $$. Then $$\displaystyle TU(x,y)=T(y,0)=(0,y) $$, not zero. Try: $$\displaystyle T(x,y)=(y,0) $$, $$\displaystyle U(x,y)=(0,y) $$. Then $$\displaystyle TU(x,y)=T(0,y)=(y,0) \neq 0 $$. I recall standard example: on $$\displaystyle \mathbb{R}^2 $$, let $$\displaystyle T(e_1)=0 $$, $$\displaystyle T(e_2)=e_1 $$; $$\displaystyle U(e_1)=e_2 $$, $$\displaystyle U(e_2)=0 $$. Then $$\displaystyle TU(e_1)=T(e_2)=e_1 \neq 0 $$, $$\displaystyle UT(e_1)=U(0)=0 $$, so $$\displaystyle UT=0 $$ but $TU \neq 0$. That's the opposite. We want $$\displaystyle TU=0 $$, $UT \neq 0$. So swap: let $$\displaystyle T(e_1)=e_2 $$, $$\displaystyle T(e_2)=0 $$; $$\displaystyle U(e_1)=0 $$, $$\displaystyle U(e_2)=e_1 $$. Then $$\displaystyle TU(e_1)=T(0)=0 $$, $$\displaystyle TU(e_2)=T(e_1)=e_2 \neq 0 $$. Not zero. Let $$\displaystyle T(e_1)=0 $$, $$\displaystyle T(e_2)=e_1 $$; $$\displaystyle U(e_1)=e_2 $$, $$\displaystyle U(e_2)=0 $$. Then $$\displaystyle TU(e_1)=T(e_2)=e_1 \neq 0 $$, $$\displaystyle UT(e_1)=U(0)=0 $$. So $$\displaystyle UT=0 $$, $TU \neq 0$. For $$\displaystyle TU=0 $$, $UT \neq 0$, take $$\displaystyle T(e_1)=e_2 $$, $$\displaystyle T(e_2)=0 $$; $$\displaystyle U(e_1)=0 $$, $$\displaystyle U(e_2)=e_1 $$. Then $$\displaystyle TU(e_1)=T(0)=0 $$, $$\displaystyle TU(e_2)=T(e_1)=e_2 \neq 0 $$. Not zero. Actually, to have $$\displaystyle TU=0 $$, we need $\operatorname{Im} U \subseteq \ker T$. So let $\ker T$ be span of $$\displaystyle e_1 $$, $\operatorname{Im} U \subseteq \ker T$. Let $$\displaystyle U(e_1)=0 $$, $$\displaystyle U(e_2)=e_1 $$. Then $$\displaystyle \operatorname{Im} U = \operatorname{span}\{e_1\} \subseteq \ker T $$ if $$\displaystyle T(e_1)=0 $$. Let $$\displaystyle T(e_1)=0 $$, $$\displaystyle T(e_2)=e_2 $$ (so $$\displaystyle \ker T = \operatorname{span}\{e_1\} $$). Then $$\displaystyle TU(e_1)=T(0)=0 $$, $$\displaystyle TU(e_2)=T(e_1)=0 $$, so $$\displaystyle TU=0 $$. But $$\displaystyle UT(e_1)=U(0)=0 $$, $$\displaystyle UT(e_2)=U(e_2)=e_1 \neq 0 $$. So $UT \neq 0$. Yes, that works:

$$\displaystyle T(x,y) = (0, y) $$, $$\displaystyle U(x,y) = (0, x) $$. Then $$\displaystyle TU(x,y)=T(0,x)=(0,x) \neq 0 $$? Wait, $$\displaystyle T(0,x)=(0,x) $$, so not zero. My basis definition: $$\displaystyle T(e_1)=0 $$, $$\displaystyle T(e_2)=e_2 $$ means $$\displaystyle T(x,y)=(0,y) $$. $$\displaystyle U(e_1)=0 $$, $$\displaystyle U(e_2)=e_1 $$ means $$\displaystyle U(x,y)=(y,0) $$. Then $$\displaystyle TU(x,y)=T(y,0)=(0,0) $$? $$\displaystyle T(y,0)=(0,0) $$ because first component 0, second component 0? $$\displaystyle T(y,0)=(0,0) $$ since $$\displaystyle T(a,b)=(0,b) $$, so $$\displaystyle T(y,0)=(0,0) $$. Yes, $$\displaystyle TU=0 $$. $$\displaystyle UT(x,y)=U(0,y)=(y,0) \neq 0 $$ if $y \neq 0$. Perfect.

Invariant Subspace: $W \subseteq V$ is $T$-invariant if $T(W) \subseteq W$.

Examples: $\{0\}$, $V$, eigenspaces, $\ker T$, $\operatorname{Im} T$.
Restriction: $$\displaystyle T|_W: W \to W $$ is linear. Minimal polynomial of $$\displaystyle T|_W $$ divides minimal polynomial of $T$.

Nilpotent Operator: $T$ is nilpotent if $$\displaystyle T^k = 0 $$ for some $k \geq 1$.

Example: Differentiation operator $D$ on $$\displaystyle P_n $$ (polynomials of degree $\leq n$): $$\displaystyle D^{n+1}=0 $$.
If $A$ is nilpotent, then $$\displaystyle T(B)=AB-BA $$ is nilpotent on the space of matrices.

Proof: Since $$\displaystyle A^k=0 $$, one can show $$\displaystyle T^k(B) $$ is a linear combination of terms like $$\displaystyle A^i B A^{k-i} $$, all zero if $i \geq k$ or $k-i \geq k$, so eventually zero.

V. Eigenvalues, Eigenvectors, and Diagonalization

Eigenvalue/Eigenvector: $\lambda \in F$ is eigenvalue if $\exists \mathbf{v} \neq 0$ such that $$\displaystyle T(\mathbf{v}) = \lambda \mathbf{v} $$. $\mathbf{v}$ is eigenvector.

Characteristic equation: $$\displaystyle \det(T - \lambda I)=0 $$ or $$\displaystyle \det(A - \lambda I)=0 $$.
Characteristic Polynomial: $$\displaystyle p_A(\lambda) = \det(\lambda I - A) $$ (or $\det(A - \lambda I)$ up to sign).

Algebraic Multiplicity (AM): multiplicity of $\lambda$ as root of $$\displaystyle p_A $$. Geometric Multiplicity (GM): $\dim \ker(A - \lambda I)$ (dimension of eigenspace).

Always $1 \leq \text{GM} \leq \text{AM}$.

Diagonalization:

$A$ is diagonalizable iff $V$ has basis of eigenvectors iff sum of GM over all $\lambda$ equals $n$ iff minimal polynomial splits into distinct linear factors.
If $A$ has $n$ distinct eigenvalues, then $A$ is diagonalizable.
Diagonalizing matrix $P$: columns are linearly independent eigenvectors. Then $$\displaystyle P^{-1}AP = D $$ diagonal with eigenvalues.
Eigenspaces for distinct eigenvalues are linearly independent, hence their direct sum is a subspace.

Cayley-Hamilton Theorem:

Every square matrix $A$ satisfies its own characteristic polynomial: $$\displaystyle p_A(A) = 0 $$.

Proof sketch (for diagonalizable case): If $$\displaystyle A = PDP^{-1} $$, then $$\displaystyle p_A(A) = P p_A(D) P^{-1} = 0 $$ since $$\displaystyle p_A(\lambda_i)=0 $$. General case: use rational canonical form or primary decomposition.

Applications:

Compute $$\displaystyle A^k $$: express $k$ in terms of powers less than $n$.
Find $$\displaystyle A^{-1} $$: from $$\displaystyle p_A(A)=0 $$, solve for $$\displaystyle A^{-1} $$ if $A$ invertible (constant term nonzero).
Verify $$\displaystyle f(A)=0 $$ for polynomial $f$.

Minimal Polynomial $$\displaystyle m_A(t) $$:

Monic polynomial of least degree such that $$\displaystyle m_A(A)=0 $$.
Divides $$\displaystyle p_A(t) $$ and has same roots.
$A$ diagonalizable iff $$\displaystyle m_A(t) $$ has no repeated roots.

VI. Jordan Canonical Form

Jordan Block $J(\lambda, k)$:

$$ \begin{bmatrix} \lambda & 1 & & \\ & \lambda & \ddots & \\ & & \ddots & 1 \\ & & & \lambda \end{bmatrix} $$

Size $k \times k$, $\lambda$ on diagonal, 1’s on superdiagonal.

Jordan Canonical Form (JCF):

Over $\mathbb{C}$ (or algebraically closed field), every matrix $A$ is similar to a block diagonal matrix with Jordan blocks.
Uniqueness: up to ordering of blocks.
For each eigenvalue $\lambda$, blocks correspond to cycles of generalized eigenvectors.

Computing JCF:

Find eigenvalues from $$\displaystyle p_A $$.
For each $\lambda$, compute nullities $$\displaystyle n_k = \dim \ker(A-\lambda I)^k $$.
Number of Jordan blocks for $\lambda$ = $$\displaystyle n_1 $$.
Sizes: let $$\displaystyle d_k = n_k - n_{k-1} $$ (with $$\displaystyle n_0=0 $$). Then $$\displaystyle d_k $$ = number of blocks of size $\geq k$. The block sizes are determined by the sequence $$\displaystyle d_k $$.

Jordan Basis: For a block of size $k$, need a chain:

$$ \mathbf{v}, (A-\lambda I)\mathbf{v}, \dots, (A-\lambda I)^{k-1}\mathbf{v}, \quad \text{with } (A-\lambda I)^k \mathbf{v}=0 \text{ but } (A-\lambda I)^{k-1}\mathbf{v} \neq 0. $$

Primary Decomposition Theorem:

Let $$\displaystyle m_A(t) = p_1(t)^{e_1} \cdots p_r(t)^{e_r} $$ with distinct irreducible $$\displaystyle p_i $$. Then

$$ V = \ker p_1(T)^{e_1} \oplus \cdots \oplus \ker p_r(T)^{e_r}. $$

Each $$\displaystyle \ker p_i(T)^{e_i} $$ is $T$-invariant, and $T$ restricted to it has minimal polynomial $$\displaystyle p_i^{e_i} $$.
Application: Given $$\displaystyle p_A $$ and $$\displaystyle m_A $$, determine possible Jordan forms. For example, if $$\displaystyle p_A=(t-1)^2(t-2)^2 $$, $$\displaystyle m_A=(t-1)(t-2) $$, then for $$\displaystyle \lambda=1 $$: one block of size 1? Actually, $$\displaystyle m_A $$ has $$\displaystyle (t-1)^1 $$, so largest block for $$\displaystyle \lambda=1 $$ is size 1, and since AM=2, there are two blocks of size 1. Similarly for $$\displaystyle \lambda=2 $$. So JCF is $\operatorname{diag}(1,1,2,2)$.

VII. Inner Product Spaces

Inner Product on $V$ over $F$ ($$\displaystyle F=\mathbb{R} $$ or $\mathbb{C}$):

$$\displaystyle (x,y) = \overline{(y,x)} $$ (conjugate symmetry).
$$\displaystyle (ax+by, z) = a(x,z) + b(y,z) $$ (linear in first argument).
$(x,x) \geq 0$ with equality iff $$\displaystyle x=0 $$.

Norm: $$\displaystyle \|x\| = \sqrt{(x,x)} $$.

Key Inequalities:

Cauchy-Schwarz: $|(x,y)| \leq \|x\| \|y\|$.
Triangle Inequality: $\|x+y\| \leq \|x\| + \|y\|$.

Orthogonality: $x \perp y$ iff $$\displaystyle (x,y)=0 $$.

Properties:

Pythagorean Theorem: If $x \perp y$, then $$\displaystyle \|x+y\|^2 = \|x\|^2 + \|y\|^2 $$.
Parallelogram Law: $$\displaystyle \|x+y\|^2 + \|x-y\|^2 = 2\|x\|^2 + 2\|y\|^2 $$.
Projection Theorem: For subspace $W$, any $x \in V$ decomposes uniquely as $$\displaystyle x = w + w^\perp $$ with $w \in W$, $$\displaystyle w^\perp \in W^\perp $$. Moreover, $w$ is the unique vector in $W$ minimizing $\|x-w\|$.

Examples:

$$\displaystyle \mathbb{R}^n $$ with dot product: $$\displaystyle (x,y) = \sum x_i y_i $$.
$C[a,b]$ with $$\displaystyle (f,g) = \int_a^b f(x)\overline{g(x)} dx $$.
Weighted: on $$\displaystyle \mathbb{R}^2 $$, $$\displaystyle (x,y) = 3x_1y_1 + 2x_2y_2 $$.

[!TIP] To show a given function is an inner product, verify the three axioms.

No Proper Open Subspaces: In any inner product space (with norm topology), any proper subspace has empty interior, hence cannot be open.

Proof: If $W$ proper subspace, take $v \notin W$. For any $w \in W$, the line $\{w + t v \mid t \in F\}$ is not contained in $W$, so no ball around $w$ is contained in $W$.

VIII. Bilinear Forms

Bilinear Form: $f: V \times V \to F$ such that $$\displaystyle f(av_1+bv_2, w) = a f(v_1,w) + b f(v_2,w) $$ and $$\displaystyle f(v, aw_1+bw_2) = a f(v,w_1) + b f(v,w_2) $$.

Matrix Representation: Fix basis $$\displaystyle \{e_1,\dots,e_n\} $$, then $$\displaystyle f(x,y) = x^T A y $$ where $$\displaystyle A_{ij} = f(e_i, e_j) $$. $A$ is unique.

Rank: $$\displaystyle \operatorname{rank}(f) = \operatorname{rank}(A) $$.

Symmetric Bilinear Form: $$\displaystyle f(x,y)=f(y,x) $$ iff $A$ symmetric.

Diagonalization: Over $\mathbb{R}$, there exists an orthogonal basis in which $A$ is diagonal (Sylvester’s theorem).
Sylvester’s Law of Inertia: For real symmetric $A$, the numbers of positive, negative, and zero eigenvalues (inertia) are invariant under congruence $$\displaystyle P^T A P $$. Signature $(p,q)$ is an invariant.

Skew-Symmetric Bilinear Form: $$\displaystyle f(x,y) = -f(y,x) $$ iff $$\displaystyle A^T = -A $$.

For odd $n$, $$\displaystyle \det A = 0 $$ (since $$\displaystyle \det A = \det A^T = \det(-A) = (-1)^n \det A = -\det A $$).
Dimension of space of skew-symmetric forms on $$\displaystyle \mathbb{R}^n $$: $$\displaystyle \frac{n(n-1)}{2} $$.
Example on $$\displaystyle \mathbb{R}^3 $$: General form:

$$ f(x,y) = a(x_1y_2 - x_2y_1) + b(x_1y_3 - x_3y_1) + c(x_2y_3 - x_3y_2). $$

Group-Invariant Bilinear Forms:

$f$ is $G$-invariant if $$\displaystyle f(Tx, Ty) = f(x,y) $$ for all $T \in G$, $x,y \in V$.
Example: On $O(n,\mathbb{R})$, $$\displaystyle B(A,B) = \operatorname{tr}(A^T B) $$ is invariant because $$\displaystyle \operatorname{tr}((PA)^T (PB)) = \operatorname{tr}(A^T P^T P B) = \operatorname{tr}(A^T B) $$ for $P \in O(n)$.
Classification: Bilinear forms on $$\displaystyle F^{n \times 1} $$ invariant under $O(n,F)$ ($$\displaystyle F=\mathbb{R} $$ or $\mathbb{C}$) are precisely scalar multiples of the standard dot product.

Proof: Invariance implies $$\displaystyle f(x,y) = f(Px, Py) = x^T P^T A P y $$ for all $P \in O(n)$. This forces $$\displaystyle A = c I $$.

Rank-1 Bilinear Forms:

$f$ has rank 1 iff $$\displaystyle f(x,y) = g(x) h(y) $$ for some linear functionals $$\displaystyle g,h \in V^* $$.
Proof: If $$\displaystyle f(x,y)=g(x)h(y) $$, then matrix is $$\displaystyle u v^T $$ where $u$ represents $g$, $v$ represents $h$, so rank 1. Conversely, if $$\displaystyle \operatorname{rank}(A)=1 $$, then $$\displaystyle A = u v^T $$ for vectors $u,v$, so $$\displaystyle f(x,y)= (x^T u)(v^T y) = g(x)h(y) $$.

IX. Adjoint Operators and Special Classes

Adjoint of $T: V \to V$ (inner product space):

$$\displaystyle T^* $$ satisfies $$\displaystyle (Tx, y) = (x, T^* y) $$ for all $x,y \in V$.
Matrix Representation: If $A$ is matrix of $T$ w.r.t. orthonormal basis, then matrix of $$\displaystyle T^* $$ is $$\displaystyle A^* $$ (conjugate transpose).
Proof: $$\displaystyle (Tx, y) = x^* A^* y = (x, A^* y) $$.

Self-Adjoint (Hermitian):

$$\displaystyle T = T^* $$; matrix is Hermitian ($$\displaystyle A^* = A $$).
Theorem: All eigenvalues of a Hermitian matrix are real.

Proof: If $$\displaystyle Ax = \lambda x $$, $x \neq 0$, then $$\displaystyle x^* A x = \lambda x^* x $$. But $$\displaystyle x^* A x = (Ax)^* x = \overline{\lambda} x^* x $$, so $$\displaystyle \lambda = \overline{\lambda} $$.
Eigenvectors corresponding to distinct eigenvalues are orthogonal.

Unitary Operator:

$$\displaystyle T^* T = TT^* = I $$.
Preserves inner product: $$\displaystyle (Tx, Ty) = (x,y) $$.
Matrix $A$ is unitary: $$\displaystyle A^* A = I $$.

Normal Operator:

$$\displaystyle T T^* = T^* T $$.
Spectral Theorem: $T$ is normal iff it is unitarily diagonalizable (over $\mathbb{C}$).
Theorem: $$\displaystyle A^* A $$ has real non-negative eigenvalues and is unitarily diagonalizable.

Examples:

Check if given matrix is Hermitian: $$\displaystyle A^* = A $$?
Unitary: $$\displaystyle A^* A = I $$?

X. Canonical Forms (Rational & Jordan) & Primary Decomposition

Rational Canonical Form:

Based on companion matrices of invariant factors.
Every matrix $A$ over $F$ is similar to a unique block-diagonal matrix where each block is a companion matrix of a monic polynomial $$\displaystyle p_i(t) $$, with $$\displaystyle p_1 | p_2 | \cdots | p_k $$ (invariant factors).
Uniqueness: the invariant factors are uniquely determined by $A$.

Jordan Canonical Form (see Section VI): Special case when characteristic polynomial splits into linear factors. Blocks are Jordan blocks.

Primary Decomposition:

Let $$\displaystyle m_A(t) = \prod_{i=1}^r p_i(t)^{e_i} $$ with distinct irreducible $$\displaystyle p_i $$.
Then $$\displaystyle V = \bigoplus_{i=1}^r W_i $$ where $$\displaystyle W_i = \ker p_i(T)^{e_i} $$.
Each $$\displaystyle W_i $$ is $T$-invariant, and $$\displaystyle T|_{W_i} $$ has minimal polynomial $$\displaystyle p_i^{e_i} $$.
Application: Determine possible Jordan forms given $$\displaystyle p_A $$ and $$\displaystyle m_A $$. For each eigenvalue $\lambda$, the largest Jordan block size equals the exponent of $(t-\lambda)$ in $$\displaystyle m_A $$, and the number of blocks equals $\dim \ker(A-\lambda I)$.

Triangular Form (Schur’s Theorem):

If all eigenvalues of $T$ lie in $F$, then there exists a basis of $V$ such that $[T]$ is upper triangular.
Over $\mathbb{C}$, always possible.

XI. Frequently Asked "Write Short Notes" Topics

Cayley-Hamilton Theorem: Every matrix satisfies its characteristic polynomial. Proof via rational canonical form or diagonalization. Applications: computing $$\displaystyle A^{-1} $$, $$\displaystyle A^k $$, verifying polynomials.

Primary Decomposition Theorem: Decomposes $V$ into $T$-invariant subspaces corresponding to primary factors of $$\displaystyle m_T $$. Used to reduce problems to primary components.

Invariant Subspaces: $W$ such that $T(W) \subseteq W$. Include eigenspaces, kernels, images. Restriction operator and divisibility of minimal polynomials.

Dual Space $$\displaystyle V^* $$: Space of linear functionals. Basis dual to given basis. Double dual isomorphism.

Bilinear Forms: Generalizations of inner products. Classification via matrix rank, symmetry. Symmetric forms diagonalizable over $\mathbb{R}$; skew-symmetric have even rank.

Inner Product Spaces: Vector spaces with inner product. Norm, orthogonality, projection theorem. Examples: $$\displaystyle \mathbb{R}^n $$, $C[a,b]$.

Jordan Block: Elementary block in JCF. Size determines chain length of generalized eigenvectors.

Annihilator $$\displaystyle W^0 $$: Subspace of $$\displaystyle V^* $$ vanishing on $W$. Dimension formula $$\displaystyle \dim W + \dim W^0 = \dim V $$.

Direct Sum Decompositions: Internal ($$\displaystyle V = \bigoplus U_i $$) vs external. Uniqueness of decomposition when sum is direct.

Unitary and Normal Operators: Unitary preserves inner product; normal commutes with adjoint. Normal operators unitarily diagonalizable.

Symmetric Bilinear Forms: $$\displaystyle f(x,y)=f(y,x) $$. Diagonalizable over $\mathbb{R}$; signature invariant.

Group Preserving Bilinear Forms: $$\displaystyle f(Tx,Ty)=f(x,y) $$. For $O(n)$, only scalar multiples of dot product on column vectors.

XII. Problem-Solving Techniques & Common Proofs

Proving Linearity: Show $$\displaystyle T(av+bw) = aT(v)+bT(w) $$ using definition.

Eigenvalues/Eigenvectors: Solve $$\displaystyle \det(A-\lambda I)=0 $$, then $$\displaystyle (A-\lambda I)\mathbf{v}=0 $$.

Diagonalization: Find $n$ linearly independent eigenvectors. Construct $P$ and $$\displaystyle D=P^{-1}AP $$.

Cayley-Hamilton Applications: From $$\displaystyle p_A(A)=0 $$, express $$\displaystyle A^n $$ in lower powers to compute $$\displaystyle A^k $$ or $$\displaystyle A^{-1} $$.

Jordan Form Computation:

Eigenvalues from $$\displaystyle p_A $$.
For each $\lambda$, compute $$\displaystyle \dim \ker(A-\lambda I)^k $$ for $$\displaystyle k=1,2,\dots $$ until nullity stabilizes.
Determine block sizes from differences $$\displaystyle d_k = n_k - n_{k-1} $$.

Gram-Schmidt Process: Orthogonalize basis $$\displaystyle \{v_1,\dots,v_n\} $$:

$$ u_1 = v_1, \quad u_k = v_k - \sum_{j=1}^{k-1} \frac{(v_k, u_j)}{(u_j, u_j)} u_j. $$

Then normalize: $$\displaystyle e_j = u_j / \|u_j\| $$.

Bilinear Form Invariance: For group $G$, check $$\displaystyle f(Tx,Ty)=f(x,y) $$ for generators of $G$. For $O(n)$, use $$\displaystyle P^T A P = A $$ for all $P \in O(n)$ implies $$\displaystyle A = cI $$.

Using Theorems:

Rank-nullity to find dimensions.
Primary decomposition to split into invariant subspaces.
Triangularization to prove properties about eigenvalues.

[!TIP] Always verify dimension formulas and divisibility conditions (e.g., minimal polynomial divides characteristic polynomial).

Account

Introduction to Linear Algebra (CY-401) - Unit 2 Short Notes