Matrices

Characteristic Roots and Vectors

In linear algebra, the characteristic roots and vectors of a square matrix are fundamental concepts that reveal crucial information about the matrix's properties and transformations. These concepts are particularly important in understanding eigenvalues and eigenvectors, which have wide-ranging applications in physics, engineering, computer science, and economics.

Eigenvalues (Characteristic Roots)

For a given square matrix $A$ of size $n \times n$, a scalar $\lambda$ is called an eigenvalue (or characteristic root) of $A$ if there exists a non-zero vector $x$ such that $Ax = \lambda x$.

The equation $Ax = \lambda x$ can be rewritten as $Ax - \lambda x = 0$, which is equivalent to $(A - \lambda I)x = 0$, where $I$ is the identity matrix of the same size as $A$.

For this equation to have a non-trivial solution for $x$ (i.e., $x \neq 0$), the matrix $(A - \lambda I)$ must be singular. A singular matrix has a determinant of zero. Therefore, the eigenvalues $\lambda$ are the solutions to the characteristic equation: $$ \det(A - \lambda I) = 0 $$

The determinant $\det(A - \lambda I)$ is a polynomial in $\lambda$ of degree $n$, known as the characteristic polynomial of $A$. The roots of this polynomial are the eigenvalues of the matrix $A$.

Eigenvectors (Characteristic Vectors)

The non-zero vector $x$ that satisfies the equation $Ax = \lambda x$ for a specific eigenvalue $\lambda$ is called an eigenvector (or characteristic vector) corresponding to that eigenvalue.

For each distinct eigenvalue $\lambda$, there is a set of corresponding eigenvectors. If $x$ is an eigenvector for $\lambda$, then any non-zero scalar multiple $cx$ (where $c \neq 0$) is also an eigenvector for $\lambda$, because $A(cx) = c(Ax) = c(\lambda x) = \lambda(cx)$.

The set of all eigenvectors corresponding to an eigenvalue $\lambda$, along with the zero vector, forms a subspace called the eigenspace of $\lambda$.

Procedure to Find Eigenvalues and Eigenvectors

1. Form the characteristic equation: $\det(A - \lambda I) = 0$.
2. Solve the characteristic equation: Find the roots $\lambda_1, \lambda_2, \dots, \lambda_n$. These are the eigenvalues of matrix $A$.
3. Find the eigenvectors for each eigenvalue: For each eigenvalue $\lambda_i$, solve the system of linear equations $(A - \lambda_i I)x = 0$ for the vector $x$. The non-zero solutions are the eigenvectors corresponding to $\lambda_i$.

Example:

Let $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$. Find its eigenvalues and eigenvectors.

1. Form $(A - \lambda I)$: $$ A - \lambda I = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 4-\lambda & 1 \\ 2 & 3-\lambda \end{pmatrix} $$
2. Solve $\det(A - \lambda I) = 0$: $$ \det \begin{pmatrix} 4-\lambda & 1 \\ 2 & 3-\lambda \end{pmatrix} = (4-\lambda)(3-\lambda) - (1)(2) = 12 - 4\lambda - 3\lambda + \lambda^2 - 2 = \lambda^2 - 7\lambda + 10 = 0 $$ Factoring the quadratic equation: $(\lambda - 5)(\lambda - 2) = 0$. The eigenvalues are $\lambda_1 = 5$ and $\lambda_2 = 2$.
3. Find eigenvectors:
   • For $\lambda_1 = 5$: Solve $(A - 5I)x = 0$. $$ \begin{pmatrix} 4-5 & 1 \\ 2 & 3-5 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $$ This gives the equation $-x_1 + x_2 = 0$, or $x_1 = x_2$. Let $x_2 = k$ (where $k \neq 0$). Then $x_1 = k$. The eigenvectors are of the form $\begin{pmatrix} k \\ k \end{pmatrix} = k \begin{pmatrix} 1 \\ 1 \end{pmatrix}$. A representative eigenvector is $\begin{pmatrix} 1 \\ 1 \end{pmatrix}$.
   • For $\lambda_2 = 2$: Solve $(A - 2I)x = 0$. $$ \begin{pmatrix} 4-2 & 1 \\ 2 & 3-2 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $$ This gives the equation $2x_1 + x_2 = 0$, or $x_2 = -2x_1$. Let $x_1 = k$ (where $k \neq 0$). Then $x_2 = -2k$. The eigenvectors are of the form $\begin{pmatrix} k \\ -2k \end{pmatrix} = k \begin{pmatrix} 1 \\ -2 \end{pmatrix}$. A representative eigenvector is $\begin{pmatrix} 1 \\ -2 \end{pmatrix}$.

Cayley–Hamilton Theorem

The Cayley–Hamilton theorem is a profound result in linear algebra that provides a direct relationship between a square matrix and its characteristic polynomial. It states that every square matrix satisfies its own characteristic equation. This theorem has significant implications for matrix computations and theoretical analyses.

Statement of the Theorem

Let $A$ be an $n \times n$ square matrix. Let $p(\lambda) = \det(A - \lambda I)$ be its characteristic polynomial. The Cayley–Hamilton theorem states that $p(A) = 0$, where $0$ is the $n \times n$ zero matrix.

In other words, if you substitute the matrix $A$ for the variable $\lambda$ in its characteristic polynomial, the resulting matrix will be the zero matrix.

Proof Outline (Conceptual)

A formal proof involves the concept of the adjugate (or classical adjoint) of a matrix. The adjugate of $A$, denoted as $\text{adj}(A)$, is the transpose of the cofactor matrix of $A$. A key property is that $A \cdot \text{adj}(A) = \det(A) \cdot I$.

Consider the matrix $(A - \lambda I)$. Its determinant is the characteristic polynomial $p(\lambda)$. The adjugate matrix, $\text{adj}(A - \lambda I)$, is a matrix whose entries are polynomials in $\lambda$ of degree at most $n-1$.

We can write $\text{adj}(A - \lambda I) = B_{n-1}\lambda^{n-1} + B_{n-2}\lambda^{n-2} + \dots + B_1\lambda + B_0$, where the $B_i$ are matrices with constant entries.

The identity $(A - \lambda I) \text{adj}(A - \lambda I) = \det(A - \lambda I) I$ holds. Substituting the polynomial form for the adjugate: $$ (A - \lambda I)(B_{n-1}\lambda^{n-1} + \dots + B_0) = p(\lambda) I $$ Expanding the left side and comparing coefficients of powers of $\lambda$ on both sides leads to a system of equations involving $A$ and the matrices $B_i$. It can be shown that substituting $A$ for $\lambda$ in the characteristic polynomial equation results in the zero matrix.

Applications and Significance

1. Finding the Inverse of a Matrix: If $A$ is an invertible matrix, its characteristic polynomial is $p(\lambda) = (-1)^n \lambda^n + c_{n-1}\lambda^{n-1} + \dots + c_1\lambda + c_0$. Since $p(A) = 0$, we have: $$ (-1)^n A^n + c_{n-1}A^{n-1} + \dots + c_1A + c_0I = 0 $$ If $A$ is invertible, then $\det(A) \neq 0$, which means $c_0 = p(0) = \det(A) \neq 0$. We can rearrange the equation: $$ c_0I = -((-1)^n A^n + c_{n-1}A^{n-1} + \dots + c_1A) $$ $$ I = -\frac{1}{c_0} ((-1)^n A^{n-1} + c_{n-1}A^{n-2} + \dots + c_1I) A $$ Thus, the inverse of $A$ is: $$ A^{-1} = -\frac{1}{c_0} ((-1)^n A^{n-1} + c_{n-1}A^{n-2} + \dots + c_1I) $$ This provides a way to compute the inverse using powers of $A$ and its coefficients, without explicitly using methods like Gaussian elimination or adjugate matrix calculation.
2. Calculating Higher Powers of a Matrix: The theorem allows us to express higher powers of $A$ (e.g., $A^k$ for $k > n$) as linear combinations of lower powers ($I, A, A^2, \dots, A^{n-1}$). From $p(A)=0$, we have $A^n = -\frac{1}{c_n}(c_{n-1}A^{n-1} + \dots + c_0I)$, assuming $c_n = (-1)^n$. Multiplying by $A$ again gives $A^{n+1}$ in terms of lower powers, and so on.
3. Theoretical Importance: The theorem establishes a fundamental link between the algebraic structure of a matrix (its powers) and its spectral properties (related to its characteristic polynomial and eigenvalues).

Example:

Verify the Cayley-Hamilton theorem for the matrix $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$.

1. We already found the characteristic polynomial: $p(\lambda) = \lambda^2 - 7\lambda + 10$.
2. According to the Cayley-Hamilton theorem, $p(A)$ should be the zero matrix: $A^2 - 7A + 10I = 0$.
3. Calculate $A^2$: $$ A^2 = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} = \begin{pmatrix} (4)(4)+(1)(2) & (4)(1)+(1)(3) \\ (2)(4)+(3)(2) & (2)(1)+(3)(3) \end{pmatrix} = \begin{pmatrix} 16+2 & 4+3 \\ 8+6 & 2+9 \end{pmatrix} = \begin{pmatrix} 18 & 7 \\ 14 & 11 \end{pmatrix} $$
4. Calculate $7A$: $$ 7A = 7 \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} = \begin{pmatrix} 28 & 7 \\ 14 & 21 \end{pmatrix} $$
5. Calculate $10I$: $$ 10I = 10 \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 10 & 0 \\ 0 & 10 \end{pmatrix} $$
6. Substitute into the equation $A^2 - 7A + 10I$: $$ \begin{pmatrix} 18 & 7 \\ 14 & 11 \end{pmatrix} - \begin{pmatrix} 28 & 7 \\ 14 & 21 \end{pmatrix} + \begin{pmatrix} 10 & 0 \\ 0 & 10 \end{pmatrix} = \begin{pmatrix} 18-28+10 & 7-7+0 \\ 14-14+0 & 11-21+10 \end{pmatrix} = \begin{pmatrix} 0 & 0 \\ 0 & 0 \end{pmatrix} $$ The result is the zero matrix, verifying the Cayley-Hamilton theorem for this matrix.

Canonical Forms Under Equivalence

In linear algebra, the concept of canonical forms is crucial for simplifying and classifying matrices. A canonical form is a standard, simplified representation of an object (like a matrix) that is unique for all objects within a certain equivalence class. Equivalence here refers to a specific type of transformation.

Equivalence of Matrices

Two $m \times n$ matrices $A$ and $B$ are said to be equivalent if there exist an $m \times m$ invertible matrix $P$ and an $n \times n$ invertible matrix $Q$ such that $B = PAQ$.

This definition is particularly relevant when considering linear transformations between vector spaces. If $A$ represents a linear transformation $T: V \to W$ with respect to some bases, and $P$ and $Q$ represent changes of bases in $V$ and $W$ respectively, then $B$ represents the same transformation $T$ with respect to the new bases. The equivalence relation preserves the rank of the matrix.

Canonical Form Under Equivalence

The canonical form of a matrix under equivalence is a particularly simple matrix that is equivalent to the original matrix. For any $m \times n$ matrix $A$ of rank $r$, there exist invertible matrices $P$ and $Q$ such that: $$ PAQ = \begin{pmatrix} I_r & 0 \\ 0 & 0 \end{pmatrix} $$ where $I_r$ is the $r \times r$ identity matrix, and the other blocks are zero matrices of appropriate dimensions ($r \times (n-r)$, $(m-r) \times r$, and $(m-r) \times (n-r)$).

This form is unique for a given matrix $A$ and is known as the **canonical form under equivalence** or the **normal form**. It reveals that any matrix of rank $r$ can be represented by a diagonal matrix with $r$ ones on the diagonal and zeros elsewhere, through appropriate choices of bases.

Procedure to Find the Canonical Form

1. Determine the rank $r$ of the matrix $A$. This can be done by reducing the matrix to row echelon form or by finding the largest non-zero minor.
2. The canonical form will be a matrix of the same dimensions as $A$, with $r$ ones on the main diagonal and zeros everywhere else.

Applications

The concept of canonical forms under equivalence is fundamental in:

• Understanding linear transformations: It shows that any linear transformation between finite-dimensional vector spaces can be represented by a very simple matrix (a diagonal matrix with 1s and 0s) by choosing appropriate bases.
• Classification of matrices: Matrices that have the same canonical form under equivalence are equivalent.
• Invariant properties: The rank of a matrix is an invariant under equivalence transformations.

Example:

Find the canonical form under equivalence for the matrix $A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{pmatrix}$.

1. First, find the rank of $A$. We can perform row operations to simplify it: $$ A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \end{pmatrix} $$ Replace $R_2$ with $R_2 - 2R_1$: $$ \begin{pmatrix} 1 & 2 & 3 \\ 2 - 2(1) & 4 - 2(2) & 6 - 2(3) \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \end{pmatrix} $$ The row echelon form has one non-zero row. Therefore, the rank of $A$ is $r=1$.
2. The matrix $A$ is a $2 \times 3$ matrix. Its canonical form under equivalence will have the same dimensions ($2 \times 3$) and its rank is $r=1$.
3. The canonical form is given by $\begin{pmatrix} I_r & 0 \\ 0 & 0 \end{pmatrix}$, where $I_r$ is the $r \times r$ identity matrix. Here $r=1$, so $I_1 = (1)$. The dimensions of the blocks are:
   • $I_r$: $1 \times 1$
   • $0$ (top right): $1 \times (3-1) = 1 \times 2$
   • $0$ (bottom left): $(2-1) \times 1 = 1 \times 1$
   • $0$ (bottom right): $(2-1) \times (3-1) = 1 \times 2$
   So, the canonical form is: $$ \begin{pmatrix} I_1 & 0_{1 \times 2} \\ 0_{1 \times 1} & 0_{1 \times 2} \end{pmatrix} = \begin{pmatrix} (1) & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix} $$ This is the $2 \times 3$ matrix $\begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}$.

This means there exist invertible matrices $P$ (size $2 \times 2$) and $Q$ (size $3 \times 3$) such that $PAQ = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}$.

Canonical Form Under Congruence

It is important to distinguish this from other types of canonical forms. For instance, under congruence, two $n \times n$ matrices $A$ and $B$ are congruent if there exists an invertible matrix $P$ such that $B = P^T A P$. For symmetric matrices, the canonical form under congruence is a diagonal matrix with a certain number of $+1$s, $-1$s, and $0$s on the diagonal (Sylvester's Law of Inertia). This is different from equivalence.

Canonical Form Under Similarity

Under similarity, two $n \times n$ matrices $A$ and $B$ are similar if there exists an invertible matrix $P$ such that $B = P^{-1}AP$. Similar matrices represent the same linear transformation with respect to different bases. The canonical forms under similarity are the Jordan Normal Form and the Rational Canonical Form. These are more complex and are used when $P$ and $Q$ in the equivalence definition are required to be the same matrix ($Q=P^{-1}$).

The canonical form under equivalence is the simplest, focusing solely on the rank as the invariant.

Summary and Key Takeaways

This unit delves into fundamental aspects of matrices that are crucial for understanding linear transformations and matrix properties. We've covered characteristic roots and vectors, the Cayley-Hamilton theorem, and canonical forms under equivalence.

Characteristic Roots and Vectors (Eigenvalues and Eigenvectors)

• Definition: $\lambda$ is an eigenvalue (characteristic root) and $x$ is the corresponding eigenvector if $Ax = \lambda x$ for a non-zero vector $x$.
• Characteristic Equation: $\det(A - \lambda I) = 0$. The roots are the eigenvalues.
• Finding Eigenvectors: For each eigenvalue $\lambda$, solve $(A - \lambda I)x = 0$.
• Significance: They describe directions that are only scaled by the linear transformation represented by $A$.

Cayley–Hamilton Theorem

• Statement: Every square matrix $A$ satisfies its own characteristic equation $p(\lambda)$. That is, $p(A) = 0$.
• Applications:
  • Finding the inverse of a matrix ($A^{-1}$).
  • Calculating higher powers of a matrix ($A^k$).
• Mnemonic: "A matrix is its own characteristic polynomial's root."

Canonical Forms Under Equivalence

• Definition: Matrices $A$ and $B$ are equivalent if $B = PAQ$ for invertible $P, Q$.
• Canonical Form: Any $m \times n$ matrix $A$ of rank $r$ is equivalent to a unique matrix $\begin{pmatrix} I_r & 0 \\ 0 & 0 \end{pmatrix}$, where $I_r$ is the $r \times r$ identity matrix.
• Invariant: The rank $r$ is the only invariant under equivalence transformations.
• Significance: Simplifies representation of linear transformations and classifies matrices based on rank.

Distinction from Other Canonical Forms

• Equivalence: $B = PAQ$. Canonical form depends on rank.
• Similarity: $B = P^{-1}AP$. Canonical form is Jordan Normal Form or Rational Canonical Form. Preserves eigenvalues.
• Congruence: $B = P^T A P$. Canonical form for symmetric matrices depends on signs of eigenvalues (Sylvester's Law).

Understanding these concepts provides a strong foundation for advanced topics in linear algebra and its applications. Mastering the procedures for finding eigenvalues, verifying the Cayley-Hamilton theorem, and determining the canonical form under equivalence will be crucial for exam success.