Operators and Matrices

Basic Matrix Operations

Matrices are fundamental tools in mathematics, especially in linear algebra and functional analysis. They are rectangular arrays of numbers, symbols, or expressions arranged in rows and columns. We often denote a matrix by a capital letter, like A, and its elements by $a_{ij}$, where 'i' represents the row number and 'j' represents the column number.

Consider a matrix A with 'm' rows and 'n' columns. This is called an m x n matrix.

$$ A = \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{pmatrix} $$

Matrix Addition and Subtraction

Two matrices can be added or subtracted only if they have the same dimensions (i.e., the same number of rows and columns). The resulting matrix has the same dimensions, and each element is the sum or difference of the corresponding elements of the original matrices.

If A and B are m x n matrices, then their sum $C = A + B$ is an m x n matrix where $c_{ij} = a_{ij} + b_{ij}$. Similarly, their difference $D = A - B$ is an m x n matrix where $d_{ij} = a_{ij} - b_{ij}$.

Example: Let $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ and $B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}$. Then $A + B = \begin{pmatrix} 1+5 & 2+6 \\ 3+7 & 4+8 \end{pmatrix} = \begin{pmatrix} 6 & 8 \\ 10 & 12 \end{pmatrix}$. And $A - B = \begin{pmatrix} 1-5 & 2-6 \\ 3-7 & 4-8 \end{pmatrix} = \begin{pmatrix} -4 & -4 \\ -4 & -4 \end{pmatrix}$.

Scalar Multiplication

Scalar multiplication involves multiplying every element of a matrix by a scalar (a single number). If A is an m x n matrix and 'c' is a scalar, then the scalar multiple cA is an m x n matrix where each element is $c \cdot a_{ij}$.

Example: Let $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ and $c = 3$. Then $3A = 3 \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} = \begin{pmatrix} 3 \cdot 1 & 3 \cdot 2 \\ 3 \cdot 3 & 3 \cdot 4 \end{pmatrix} = \begin{pmatrix} 3 & 6 \\ 9 & 12 \end{pmatrix}$.

Matrix Multiplication

Matrix multiplication is more complex. For the product of two matrices A and B (in that order, AB) to be defined, the number of columns in matrix A must be equal to the number of rows in matrix B. If A is an m x n matrix and B is an n x p matrix, their product AB is an m x p matrix.

The element in the i-th row and j-th column of the product matrix AB, denoted as $c_{ij}$, is calculated by taking the dot product of the i-th row of A and the j-th column of B. $$ c_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj} $$

Example: Let $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ (2x2) and $B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}$ (2x2). The product AB is a 2x2 matrix. $c_{11} = (1 \cdot 5) + (2 \cdot 7) = 5 + 14 = 19$ $c_{12} = (1 \cdot 6) + (2 \cdot 8) = 6 + 16 = 22$ $c_{21} = (3 \cdot 5) + (4 \cdot 7) = 15 + 28 = 43$ $c_{22} = (3 \cdot 6) + (4 \cdot 8) = 18 + 32 = 50$ So, $AB = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}$.

Note that matrix multiplication is generally not commutative, meaning $AB \neq BA$ in most cases.

Transpose of a Matrix

The transpose of a matrix A, denoted as $A^T$, is obtained by interchanging its rows and columns. If A is an m x n matrix, then $A^T$ is an n x m matrix. The element $a_{ij}$ in A becomes the element $a_{ji}$ in $A^T$.

Example: If $A = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix}$, then $A^T = \begin{pmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{pmatrix}$.

Properties of transpose: $(A+B)^T = A^T + B^T$, $(cA)^T = cA^T$, $(AB)^T = B^T A^T$.

Identity Matrix

An identity matrix, denoted by I, is a square matrix (number of rows equals number of columns) that has 1s on the main diagonal and 0s everywhere else. It acts as the multiplicative identity for matrices, meaning $AI = IA = A$ for any compatible matrix A.

For a 3x3 identity matrix: $I_3 = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$.

Determinant of a Matrix

The determinant is a scalar value that can be computed from the elements of a square matrix. It provides important information about the matrix, such as whether it is invertible. The determinant is denoted as det(A) or |A|.

Determinant for 2x2 Matrices

For a 2x2 matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$, the determinant is calculated as: $$ |A| = ad - bc $$

Example: If $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$, then $|A| = (1 \cdot 4) - (2 \cdot 3) = 4 - 6 = -2$.

Determinant for 3x3 Matrices

For a 3x3 matrix $A = \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix}$, the determinant can be calculated using the cofactor expansion method. A common way is to expand along the first row: $$ |A| = a \begin{vmatrix} e & f \\ h & i \end{vmatrix} - b \begin{vmatrix} d & f \\ g & i \end{vmatrix} + c \begin{vmatrix} d & e \\ g & h \end{vmatrix} $$ Where $\begin{vmatrix} e & f \\ h & i \end{vmatrix} = ei - fh$, and so on for the other 2x2 determinants. $$ |A| = a(ei - fh) - b(di - fg) + c(dh - eg) $$

Example: Let $A = \begin{pmatrix} 1 & 2 & 3 \\ 0 & 1 & 4 \\ 2 & 0 & 5 \end{pmatrix}$. $|A| = 1 \begin{vmatrix} 1 & 4 \\ 0 & 5 \end{vmatrix} - 2 \begin{vmatrix} 0 & 4 \\ 2 & 5 \end{vmatrix} + 3 \begin{vmatrix} 0 & 1 \\ 2 & 0 \end{vmatrix}$ $|A| = 1((1 \cdot 5) - (4 \cdot 0)) - 2((0 \cdot 5) - (4 \cdot 2)) + 3((0 \cdot 0) - (1 \cdot 2))$ $|A| = 1(5 - 0) - 2(0 - 8) + 3(0 - 2)$ $|A| = 1(5) - 2(-8) + 3(-2)$ $|A| = 5 + 16 - 6 = 15$.

Determinant for n x n Matrices (General Case)

For larger square matrices, the determinant is calculated using cofactor expansion along any row or column. The general formula for cofactor expansion along the i-th row is: $$ |A| = \sum_{j=1}^{n} a_{ij} C_{ij} $$ Where $C_{ij} = (-1)^{i+j} M_{ij}$ is the cofactor of the element $a_{ij}$, and $M_{ij}$ is the determinant of the submatrix obtained by deleting the i-th row and j-th column of A (this submatrix is called the minor).

Expansion along the j-th column is: $$ |A| = \sum_{i=1}^{n} a_{ij} C_{ij} $$

Key Properties of Determinants:

• If a matrix has a row or column of all zeros, its determinant is 0.
• If two rows or columns of a matrix are identical, its determinant is 0.
• If a matrix is triangular (upper or lower), its determinant is the product of its diagonal entries.
• $|AB| = |A| |B|$.
• $|A^T| = |A|$.
• A square matrix A is invertible if and only if $|A| \neq 0$.

Spectrum of an Operator

In functional analysis, an operator is a mapping between vector spaces. For linear operators on Hilbert spaces (which are vector spaces with an inner product that is complete), we can study their properties, including their spectrum. The spectrum of an operator is a generalization of the set of eigenvalues of a matrix.

Let H be a Hilbert space and T: H → H be a bounded linear operator. The spectrum of T, denoted by $\sigma(T)$, is the set of all complex numbers $\lambda$ for which the operator $(T - \lambda I)$ is not invertible. Here, I is the identity operator.

The operator $(T - \lambda I)$ fails to be invertible for three possible reasons:

• $(T - \lambda I)$ is not injective (its null space is non-trivial). This means there exists a non-zero vector x such that $(T - \lambda I)x = 0$, or $Tx = \lambda x$. In this case, $\lambda$ is an eigenvalue of T, and x is its corresponding eigenvector.
• $(T - \lambda I)$ is not surjective (its range is not the entire space H). This means there is no vector y in H such that $(T - \lambda I)x = y$ for all y in H.
• $(T - \lambda I)$ is not bounded below (its inverse, if it exists on its range, is not a bounded operator).

The set of all such $\lambda$ constitutes the spectrum $\sigma(T)$.

Types of Spectra

The spectrum is often decomposed into different parts:

• Point Spectrum ($ \sigma_p(T) $): The set of eigenvalues of T. These are the values $\lambda$ for which $(T - \lambda I)$ is not injective.
• Continuous Spectrum ($ \sigma_c(T) $): The set of values $\lambda$ for which $(T - \lambda I)$ is injective, its range is dense in H, but $(T - \lambda I)^{-1}$ is not a bounded operator.
• Residual Spectrum ($ \sigma_r(T) $): The set of values $\lambda$ for which $(T - \lambda I)$ is injective and its range is not dense in H.

The entire spectrum is the union of these three sets: $\sigma(T) = \sigma_p(T) \cup \sigma_c(T) \cup \sigma_r(T)$.

Spectrum of a Matrix (Finite-Dimensional Case)

When dealing with matrices, which represent operators on finite-dimensional vector spaces (like $ \mathbb{C}^n $ or $ \mathbb{R}^n $), the concepts simplify. For an n x n matrix A, the operator is $T(x) = Ax$.

In finite-dimensional spaces, every linear operator is bounded. Also, for a linear operator on a finite-dimensional space, if $(A - \lambda I)$ is injective, then its range is the entire space, and its inverse is automatically bounded. Therefore, the continuous and residual spectra are empty.

For a matrix A acting on $ \mathbb{C}^n $, the spectrum $ \sigma(A) $ is precisely the set of its eigenvalues. $$ \sigma(A) = \{ \lambda \in \mathbb{C} \mid \det(A - \lambda I) = 0 \} $$ The equation $ \det(A - \lambda I) = 0 $ is called the characteristic equation of the matrix A. The roots of this polynomial equation are the eigenvalues.

Example: Let $ A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} $. We need to find $ \lambda $ such that $ \det(A - \lambda I) = 0 $. $ A - \lambda I = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix} $ $ \det(A - \lambda I) = (2-\lambda)(2-\lambda) - (1)(1) = (2-\lambda)^2 - 1 $ Set the determinant to zero: $ (2-\lambda)^2 - 1 = 0 $ $ (2-\lambda)^2 = 1 $ $ 2-\lambda = \pm 1 $ Case 1: $ 2-\lambda = 1 \implies \lambda = 1 $ Case 2: $ 2-\lambda = -1 \implies \lambda = 3 $ So, the spectrum of matrix A is $ \sigma(A) = \{1, 3\} $. These are the eigenvalues.

Spectral Theorem in Finite-Dimensional Hilbert Spaces

The Spectral Theorem is one of the most important results in linear algebra and functional analysis. It provides a deep understanding of the structure of certain types of linear operators, particularly normal operators, by relating them to diagonal matrices. In finite-dimensional Hilbert spaces, the theorem has a particularly elegant and powerful form.

Normal Operators

A bounded linear operator T on a Hilbert space H is called normal if it commutes with its adjoint operator $T^*$, i.e., $TT^* = T^*T$. The adjoint $T^*$ is a unique operator such that $ \langle Tx, y \rangle = \langle x, T^*y \rangle $ for all vectors x, y in H.

For matrices, the adjoint of a matrix A is its conjugate transpose, denoted by $A^\dagger$ or $A^*$. A matrix A is normal if $AA^* = A^*A$.

Examples of normal operators include:

• Self-adjoint (Hermitian) operators: $T = T^*$. Matrices satisfying $A = A^\dagger$.
• Unitary operators: $T^*T = TT^* = I$. Matrices satisfying $A^*A = AA^* = I$.
• Skew-self-adjoint operators: $T = -T^*$. Matrices satisfying $A = -A^\dagger$.

The Spectral Theorem for Normal Operators (Finite-Dimensional Case)

Let H be a finite-dimensional Hilbert space (e.g., $ \mathbb{C}^n $ with the standard inner product) and let T: H → H be a normal operator. Then there exists an orthonormal basis $\{v_1, v_2, \dots, v_n\}$ for H such that T is represented by a diagonal matrix with respect to this basis.

Specifically, if $ \lambda_1, \lambda_2, \dots, \lambda_n $ are the eigenvalues of T (repeated according to their algebraic multiplicity), then T can be written as: $$ T = \lambda_1 P_1 + \lambda_2 P_2 + \dots + \lambda_n P_n $$ where $P_i$ are orthogonal projection operators onto the eigenspaces corresponding to $\lambda_i$. These projections satisfy:

• $P_i P_j = 0$ for $i \neq j$
• $P_i^2 = P_i$ (idempotent)
• $P_i^* = P_i$ (self-adjoint)
• $ \sum_{i=1}^{n} P_i = I $ (the identity operator)

The eigenvalues $ \lambda_i $ are the diagonal entries of the matrix representation of T in the basis of eigenvectors.

In matrix form, if A is a normal matrix, then there exists a unitary matrix U (whose columns form an orthonormal basis of eigenvectors) such that: $$ A = U D U^* $$ where D is a diagonal matrix whose diagonal entries are the eigenvalues of A. $U^*$ is the conjugate transpose of U. This means A is unitarily diagonalizable.

Example: Consider the real symmetric (and thus normal) matrix $ A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} $. We found its eigenvalues to be $ \lambda_1 = 1 $ and $ \lambda_2 = 3 $. Let's find the corresponding eigenvectors: For $ \lambda_1 = 1 $: $ (A - 1I)x = 0 \implies \begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $ This gives $ x_1 + x_2 = 0 $, so $ x_2 = -x_1 $. An eigenvector is $ \begin{pmatrix} 1 \\ -1 \end{pmatrix} $. For $ \lambda_2 = 3 $: $ (A - 3I)x = 0 \implies \begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} $ This gives $ -x_1 + x_2 = 0 $, so $ x_2 = x_1 $. An eigenvector is $ \begin{pmatrix} 1 \\ 1 \end{pmatrix} $. The eigenvectors are $ v_1 = \begin{pmatrix} 1 \\ -1 \end{pmatrix} $ and $ v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix} $. To form an orthonormal basis, we normalize them: $ ||v_1|| = \sqrt{1^2 + (-1)^2} = \sqrt{2} $. Normalized $ u_1 = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ -1 \end{pmatrix} $. $ ||v_2|| = \sqrt{1^2 + 1^2} = \sqrt{2} $. Normalized $ u_2 = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 \\ 1 \end{pmatrix} $. The unitary matrix U has these normalized eigenvectors as columns: $ U = \begin{pmatrix} 1/\sqrt{2} & 1/\sqrt{2} \\ -1/\sqrt{2} & 1/\sqrt{2} \end{pmatrix} $. The diagonal matrix D with eigenvalues is $ D = \begin{pmatrix} 1 & 0 \\ 0 & 3 \end{pmatrix} $ (corresponding to the order of eigenvectors). The adjoint $ U^* = U^T = \begin{pmatrix} 1/\sqrt{2} & -1/\sqrt{2} \\ 1/\sqrt{2} & 1/\sqrt{2} \end{pmatrix} $ (since it's real). Let's check: $ UDU^* = \begin{pmatrix} 1/\sqrt{2} & 1/\sqrt{2} \\ -1/\sqrt{2} & 1/\sqrt{2} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & 3 \end{pmatrix} \begin{pmatrix} 1/\sqrt{2} & -1/\sqrt{2} \\ 1/\sqrt{2} & 1/\sqrt{2} \end{pmatrix} $ $ = \begin{pmatrix} 1/\sqrt{2} & 3/\sqrt{2} \\ -1/\sqrt{2} & 3/\sqrt{2} \end{pmatrix} \begin{pmatrix} 1/\sqrt{2} & -1/\sqrt{2} \\ 1/\sqrt{2} & 1/\sqrt{2} \end{pmatrix} $ $ = \begin{pmatrix} (1/2) + (3/2) & (-1/2) + (3/2) \\ (-1/2) + (3/2) & (1/2) + (3/2) \end{pmatrix} = \begin{pmatrix} 4/2 & 2/2 \\ 2/2 & 4/2 \end{pmatrix} = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} = A $. This confirms the Spectral Theorem for this matrix.

Spectral Theorem for Self-Adjoint Operators

A special and very important case of the Spectral Theorem is for self-adjoint operators (Hermitian matrices). For a self-adjoint operator T on a finite-dimensional Hilbert space, all its eigenvalues are real. The theorem states that T is unitarily diagonalizable, and the unitary matrix U can be chosen to be real orthogonal if T is real.

$$ T = \sum_{i=1}^{n} \lambda_i P_i $$ where $ \lambda_i \in \mathbb{R} $ are the eigenvalues and $ P_i $ are orthogonal projections.

In matrix form, for a Hermitian matrix A, there exists a unitary matrix U such that $ A = UDU^* $, where D is a diagonal matrix with real eigenvalues on the diagonal.

Spectrum and Diagonalization

The spectrum of an operator (or matrix) is intrinsically linked to its diagonalizability.

• A matrix A is diagonalizable if and only if for each eigenvalue $ \lambda $, the geometric multiplicity (dimension of the eigenspace) equals the algebraic multiplicity (multiplicity of $ \lambda $ as a root of the characteristic polynomial).
• If a matrix is normal, it is always diagonalizable by a unitary transformation.
• If a matrix is self-adjoint (Hermitian), it is always diagonalizable by a unitary transformation, and its eigenvalues are real.

Understanding the spectrum helps in analyzing the behavior of linear systems, solving differential equations, and in numerous applications in physics and engineering, such as quantum mechanics where operators representing observables are self-adjoint.