Introduction to Linear Algebra (CY-401) - Unit 5 Short Notes

UNIT 5: Advanced Linear Algebra – Short Notes

Based on RGPV CY-401 Past Papers (2022–2025)

I. Quotient Spaces and Direct Sums

Quotient Space \( V/W \):

Let \( W \) be a subspace of \( V \). The set of all cosets \( v + W = \{v + w \mid w \in W\} \) forms a vector space \( V/W \) with operations:

\[ (u+W) + (v+W) = (u+v)+W, \quad \alpha(v+W) = (\alpha v)+W. \]

The natural map \( \eta: V \to V/W \), \( \eta(v) = v+W \), is linear and surjective with \( \ker \eta = W \).

Key Theorem: If \( V \) is finite-dimensional, then

\[ > \dim(V/W) = \dim V - \dim W. > \]

Direct Sum (Internal):

Subspaces \( U_1, \dots, U_k \subseteq V \) form a direct sum, denoted \( V = U_1 \oplus \cdots \oplus U_k \), if every \( v \in V \) can be written uniquely as \( v = u_1 + \cdots + u_k \) with \( u_i \in U_i \).
Equivalent conditions:

• \( V = U_1 + \cdots + U_k \) and \( U_i \cap \sum_{j \neq i} U_j = \{0\} \) for all \( i \).

• For two subspaces: \( V = U_1 \oplus U_2 \iff V = U_1 + U_2 \) and \( U_1 \cap U_2 = \{0\} \).

Existence of Complementary Subspace:

For any subspace \( W \subseteq V \), there exists \( W' \) such that \( V = W \oplus W' \).

Exam Tip: To show a sum is not direct, find a non-zero vector expressible in two different ways or show \( U_i \cap \sum_{j \neq i} U_j \neq \{0\} \).

II. Dual Spaces and Annihilators

Dual Space \( V^* \):

The space of all linear functionals \( f: V \to F \) (where \( F \) is the field).

• \( V^* \) is a vector space with \( (f+g)(v)=f(v)+g(v) \), \( (\alpha f)(v)=\alpha f(v) \).

• If \( \mathcal{B} = \{e_1,\dots,e_n\} \) is a basis for \( V \), the dual basis \( \mathcal{B}^* = \{f_1,\dots,f_n\} \subseteq V^* \) satisfies \( f_i(e_j) = \delta_{ij} \).

\[ \boxed{\dim V^* = \dim V} \]

Annihilator \( W^0 \):

For \( W \subseteq V \),

\[ W^0 = \{ f \in V^* \mid f(w)=0 \text{ for all } w \in W \}. \]

• \( W^0 \) is a subspace of \( V^* \).

\[ \boxed{\dim W + \dim W^0 = \dim V} \]

Isomorphism:

\[ (V/W)^* \cong W^0 \]

via \( f + W \mapsto f|_W \) (well-defined isomorphism).

Special Cases:

• \( \{0\}^0 = V^* \), \( V^0 = \{0\} \).

III. Linear Transformations and Adjoints

Adjoint \( T^* \):

In an inner product space \( (V, (\cdot,\cdot)) \), the adjoint \( T^*: V \to V \) satisfies

\[ (T\alpha \mid \beta) = (\alpha \mid T^*\beta) \quad \forall \alpha,\beta \in V. \]

• Uniqueness: \( T^* \) exists and is linear.

• Matrix representation: If \( A \) is matrix of \( T \) w.r.t. orthonormal basis, then \( A^* \) (conjugate transpose) is matrix of \( T^* \).

Special Operators (in inner product spaces):

Exam Tip: To check if \( T \) is self-adjoint/unitary, compute \( (T\alpha,\beta) \) and \( (\alpha,T\beta) \) for arbitrary vectors or use matrix \( A \) and verify \( A = A^* \) or \( A^*A = I \).

IV. Invariant Subspaces

Definition:

Subspace \( W \subseteq V \) is \( T \)-invariant if \( T(W) \subseteq W \).

Common Invariant Subspaces:

• Eigenspaces \( E_\lambda = \ker(T-\lambda I) \).

• \( \ker T \), \( \operatorname{im} T \).

• \( V \) and \( \{0\} \).

Finding Invariant Subspaces (e.g., for \( 2\times2 \) matrices):

• If \( T \) has an eigenvalue \( \lambda \), then \( E_\lambda \) is invariant.

• If \( T \) is not diagonalizable, the span of a generalized eigenvector chain is invariant.

Role:

Invariant subspaces allow decomposition of \( V \) into \( T \)-invariant direct sums, leading to block triangular matrix representations.

V. Eigenvalues, Eigenvectors, and Diagonalization

Eigenvalue/Eigenvector:

\( \lambda \in F \) is eigenvalue of \( T \) (or \( A \)) if \( \exists \) non-zero \( v \) with \( T(v)=\lambda v \) or \( Av = \lambda v \).
Characteristic equation:

\[ \boxed{\det(A - \lambda I) = 0} \]

Solutions are eigenvalues.

Multiplicities:

• Algebraic multiplicity (AM): multiplicity of \( \lambda \) as root of char. polynomial.

• Geometric multiplicity (GM): \( \dim \ker(A-\lambda I) = \dim E_\lambda \).

Always \( 1 \leq \text{GM} \leq \text{AM} \).

Diagonalization Criteria:

1. \( T \) is diagonalizable \( \iff \) \( V \) has a basis of eigenvectors.

2. Equivalent: minimal polynomial of \( T \) splits into distinct linear factors.

3. If \( n \) distinct eigenvalues \( \Rightarrow \) diagonalizable.

Process:

1. Find eigenvalues \( \lambda_i \).

2. For each \( \lambda_i \), find basis of \( E_{\lambda_i} \).

3. If total basis vectors \( = n \), then \( P = [\text{basis vectors}] \) gives \( P^{-1}AP = \operatorname{diag}(\lambda_1,\dots,\lambda_n) \).

Triangularization:

If all eigenvalues of \( A \) lie in \( F \), then \( \exists \) basis where \( A \) is upper triangular.

VI. Characteristic and Minimal Polynomials

Characteristic Polynomial:

\[ p_A(t) = \det(tI - A). \]

• Monic of degree \( n \).

• Roots are eigenvalues (with AM).

Minimal Polynomial \( m_A(t) \):

• Monic polynomial of least degree such that \( m_A(A)=0 \).

• Divides any annihilating polynomial.

• Divides \( p_A(t) \), and both have same distinct roots.

Relationship:

• \( m_A(t) = \prod_{\lambda} (t-\lambda)^{k_\lambda} \), where \( k_\lambda \) = size of largest Jordan block for \( \lambda \).

• \( p_A(t) = \prod_{\lambda} (t-\lambda)^{n_\lambda} \), where \( n_\lambda \) = AM of \( \lambda \).

Cayley-Hamilton Theorem:

\[ \boxed{p_A(A) = 0} \]

Every matrix satisfies its own characteristic equation.

Exam Tip: To find \( m_A \), check divisors of \( p_A \) starting from smallest degree until \( m(A)=0 \). Use Jordan form: largest block size for \( \lambda \) gives exponent in \( m_A \).

VII. Cayley-Hamilton Theorem and Primary Decomposition

Cayley-Hamilton (Proof Idea):

• For diagonalizable \( A \): \( p_A(A)=0 \) since \( p_A(\lambda_i)=0 \) and \( A \) acts like \( \lambda_i \) on eigenvectors.

• General case: Use primary decomposition or consider \( A \) as linear transformation on \( V \), and \( m_A(t) \) divides \( p_A(t) \), but \( p_A(A)=0 \) by definition of \( m_A \)? Actually, CH is proved independently (e.g., via companion matrices or polynomial identities).

Applications:

• Compute \( A^{-1} \) (if \( A \) invertible): from \( p_A(A)=0 \), solve for \( A^{-1} \).

• Compute high powers \( A^k \) by reducing modulo \( m_A(t) \).

Primary Decomposition Theorem:

Let \( T: V \to V \), with minimal polynomial

\[ m_T(t) = \prod_{i=1}^k p_i(t)^{e_i}, \quad p_i \text{ distinct irreducible}. \]

Then

\[ \boxed{V = W_1 \oplus \cdots \oplus W_k} \]

where \( W_i = \ker p_i(T)^{e_i} \) are \( T \)-invariant, and \( m_{T|_{W_i}}(t) = p_i(t)^{e_i} \).

For eigenvalues (over algebraically closed field like \( \mathbb{C} \)):

If \( \lambda_1,\dots,\lambda_k \) distinct eigenvalues, then

\[ V = E_{\lambda_1} \oplus \cdots \oplus E_{\lambda_k} \iff T \text{ diagonalizable}. \]

More generally, \( V = \bigoplus_{\lambda} V_\lambda \) where \( V_\lambda = \ker (T-\lambda I)^{n_\lambda} \) (generalized eigenspace).

VIII. Jordan Canonical Form

Jordan Block \( J_k(\lambda) \):

\( k\times k \) matrix with \( \lambda \) on diagonal, 1’s on superdiagonal, 0 elsewhere:

\[ J_k(\lambda) = \begin{bmatrix} \lambda & 1 & & \\ & \lambda & \ddots & \\ & & \ddots & 1 \\ & & & \lambda \end{bmatrix} \]

Jordan Canonical Form (JCF) \( J \):

Block diagonal matrix with Jordan blocks \( J_{k_i}(\lambda_i) \).

• Existence & Uniqueness over \( \mathbb{C} \) (up to block order).

• \( J \) is similar to \( A \): \( P^{-1}AP = J \).

Relationships:

• Characteristic polynomial: \( p_A(t) = \prod (t-\lambda_i)^{k_i} \) (sum of block sizes for each \( \lambda \)).

• Minimal polynomial: \( m_A(t) = \prod (t-\lambda_i)^{s_i} \), where \( s_i = \max\{ \text{block sizes for } \lambda_i \} \).

• Geometric multiplicity of \( \lambda \): number of Jordan blocks for \( \lambda \).

Constructing Jordan Basis:

1. Find eigenvalues \( \lambda \).

2. For each \( \lambda \), compute generalized eigenspace \( V_\lambda = \ker(A-\lambda I)^n \).

3. For each \( \lambda \), find chains of generalized eigenvectors:

   • Start with \( v \) such that \( (A-\lambda I)^m v = 0 \) but \( (A-\lambda I)^{m-1}v \neq 0 \) (length \( m \) chain).

   • Chain: \( v, (A-\lambda I)v, \dots, (A-\lambda I)^{m-1}v \).

4. Jordan basis = union of all chains.

Exam Tip: For a given matrix, compute \( \ker(A-\lambda I)^k \) for increasing \( k \) to find chain lengths. Number of blocks = \( \dim \ker(A-\lambda I) \). Largest block size = smallest \( k \) with \( \ker(A-\lambda I)^k = V_\lambda \).

IX. Rational Canonical Form

Invariant Factors:

For linear operator \( T \) on \( V \) (finite-dimensional), there exist monic polynomials

\[ d_1(t) \mid d_2(t) \mid \cdots \mid d_k(t) \]

such that

\[ V = \ker d_1(T) \oplus \cdots \oplus \ker d_k(T) \]

and \( m_T(t) = d_k(t) \), \( p_T(t) = d_1(t)\cdots d_k(t) \). The \( d_i \) are invariant factors.

Rational Canonical Form (Frobenius Normal Form):

Matrix block diagonal with companion matrices of invariant factors \( d_i(t) \).

• Existence and uniqueness over any field \( F \).

• Preferred when eigenvalues not in \( F \) (Jordan may not exist).

Companion Matrix of \( d(t)=t^m + a_{m-1}t^{m-1}+\cdots+a_0 \):

\[ C(d) = \begin{bmatrix} 0 & 0 & \cdots & 0 & -a_0 \\ 1 & 0 & \cdots & 0 & -a_1 \\ 0 & 1 & \cdots & 0 & -a_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 & -a_{m-1} \end{bmatrix} \]

Elementary Divisors:

If \( p(t) \) irreducible, powers \( p(t)^{e} \) from factorization of invariant factors. Jordan form uses linear \( (t-\lambda) \); rational uses any irreducible.

X. Inner Product Spaces

Inner Product \( (\cdot,\cdot) \) on \( V \) (real or complex):

1. \( (u,v) = \overline{(v,u)} \) (conjugate symmetry).

2. \( (\alpha u + \beta v, w) = \alpha(u,w) + \beta(v,w) \) (linear in first arg; conjugate linear in second for complex).

3. \( (v,v) \geq 0 \), with equality \( \iff v=0 \).

Induced Norm:

\[ \|v\| = \sqrt{(v,v)}. \]

Key Inequalities:

• Cauchy-Schwarz: \( |(u,v)| \leq \|u\|\|v\| \).

• Triangle: \( \|u+v\| \leq \|u\| + \|v\| \).

Orthogonality: \( u \perp v \iff (u,v)=0 \).
Orthogonal Complement: \( W^\perp = \{ v \in V \mid (v,w)=0 \ \forall w\in W \} \).

\[ \boxed{V = W \oplus W^\perp} \]

(Projection theorem: \( \forall v \in V \), \( v = w + w^\perp \) uniquely with \( w\in W, w^\perp\in W^\perp \)).

Gram-Schmidt Orthogonalization:

From linearly independent \( \{v_1,\dots,v_n\} \), construct orthogonal basis \( \{u_1,\dots,u_n\} \):

\[ \begin{aligned} u_1 &= v_1 \\ u_2 &= v_2 - \frac{(v_2,u_1)}{(u_1,u_1)}u_1 \\ u_3 &= v_3 - \frac{(v_3,u_1)}{(u_1,u_1)}u_1 - \frac{(v_3,u_2)}{(u_2,u_2)}u_2 \\ &\vdots \end{aligned} \]

Normalize to get orthonormal basis \( e_i = u_i/\|u_i\| \).

Exam Tip: To show a given \( (\cdot,\cdot) \) is an inner product, verify all three properties. For \( \mathbb{R}^n \), standard dot product; for \( \mathbb{C}^n \), \( (z,w)=\sum z_i \overline{w_i} \).

No Proper Open Subspace:

In an inner product space, any proper subspace has empty interior (since you can always find a small vector orthogonal to it).

XI. Bilinear Forms

Definition:

\( B: V \times V \to F \) is bilinear if linear in each argument separately.

Matrix representation w.r.t. basis \( \mathcal{B} \):

\[ B(\alpha,\beta) = [\alpha]_{\mathcal{B}}^T A [\beta]_{\mathcal{B}}, \]

where \( A_{ij} = B(e_i,e_j) \). Rank of \( B \) = rank of \( A \).

Symmetric Bilinear Forms (\( B(\alpha,\beta)=B(\beta,\alpha) \)):

• Over \( F \) with \( \operatorname{char} \neq 2 \), \( \exists \) basis where \( A \) is diagonal.

• Sylvester’s Law of Inertia (over \( \mathbb{R} \)):

  In any diagonal representation \( \operatorname{diag}(1,\dots,1,-1,\dots,-1,0,\dots,0) \), the numbers \( p \) (positive), \( n \) (negative), \( z \) (zero) are invariants.

  Signature = \( p-n \), index = \( \min(p,n) \).

Skew-Symmetric Bilinear Forms (\( B(\alpha,\beta) = -B(\beta,\alpha) \)):

• \( B(v,v)=0 \) for all \( v \).

• Over \( F \) with \( \operatorname{char} \neq 2 \), \( \exists \) basis where \( A \) is block diagonal with blocks

  \[ \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix} \]

  and possibly zero rows/columns.

• Dimension of space of skew-symmetric forms on \( \mathbb{R}^n \): \( \frac{n(n-1)}{2} \).

• Example on \( \mathbb{R}^3 \): \( B((x_1,y_1,z_1),(x_2,y_2,z_2)) = x_1y_2 - y_1x_2 + y_1z_2 - z_1y_2 + z_1x_2 - x_1z_2 \).

Group-Invariant Bilinear Forms:

\( B \) is \( G \)-invariant if \( B(T\alpha, T\beta) = B(\alpha,\beta) \) for all \( T \in G \subseteq GL(V) \).

Example: On \( M_n(\mathbb{R}) \), \( B(A,B)=\operatorname{tr}(A^TB) \) is \( O(n) \)-invariant.

Rank 1 Bilinear Forms:

\( B \) has rank 1 \( \iff \) \( B(\alpha,\beta) = \varphi(\alpha)\psi(\beta) \) for some linear functionals \( \varphi,\psi \in V^* \), not both zero.

XII. Special Matrix Types and Additional Theorems

Hermitian Matrices (\( A^* = A \)):

• All eigenvalues real.

• Unitarily diagonalizable: \( \exists \) unitary \( U \) with \( U^*AU \) diagonal.

• Eigenvectors for distinct eigenvalues are orthogonal.

Unitary Matrices (\( A^*A = I \)):

• \( A^{-1} = A^* \).

• Eigenvalues lie on unit circle: \( |\lambda|=1 \).

• Columns (and rows) form orthonormal basis of \( \mathbb{C}^n \).

Normal Matrices (\( AA^* = A^*A \)):

• Unitarily diagonalizable.

• Includes Hermitian, unitary, skew-Hermitian, etc.

Matrices \( A^*A \):

• Hermitian and positive semi-definite: \( (A^*A x, x) = \|Ax\|^2 \geq 0 \).

• Eigenvalues \( \geq 0 \).

• Unitarily similar to diagonal with non-negative entries.

Key Theorems:

1. Rank-Nullity Theorem:

   For \( T: V \to W \),

   \[ \boxed{\dim \ker T + \dim \operatorname{im} T = \dim V} \]

2. Linear Functionals on \( \mathbb{R}^n \):

   Every \( f \in (\mathbb{R}^n)^* \) has form \( f(x) = a \cdot x \) for unique \( a \in \mathbb{R}^n \).

3. Nilpotent Operators:

   \( T \) nilpotent \( \iff \) all eigenvalues 0 \( \iff \) \( T^k=0 \) for some \( k \).

   Two nilpotent operators similar \( \iff \) same invariant factors/JCF.

4. Triangularization:

   If all eigenvalues of \( T \) lie in \( F \), then \( \exists \) basis of \( V \) with \( [T] \) upper triangular.

Final Exam Strategy:

• For proof questions (e.g., Cayley-Hamilton, primary decomposition), state theorem clearly, outline key steps (use dimension formulas, invariant subspaces).

• For computations (Jordan form, diagonalization), always: (1) char poly → eigenvalues, (2) eigenvectors/gen. eigenvectors, (3) determine block structure from dimensions of kernels.

• For bilinear/inner product, verify properties systematically.

• Remember: Direct sum means unique representation; invariant subspace closed under \( T \); minimal polynomial gives size of largest Jordan block.