Vector Spaces, Dual Spaces, and Inner Product Spaces

1. Vector Spaces

A vector space is a fundamental concept in linear algebra. It's a collection of objects called vectors, which can be added together and multiplied by scalars (numbers). These operations follow specific rules, making the space behave in a predictable way. Think of vectors as arrows that can be stretched, shrunk, or added to each other.

More formally, a vector space V over a field F (usually the real numbers ℝ or complex numbers ℂ) is a non-empty set V equipped with two operations:

1. Vector Addition: A binary operation '+' : V × V → V
2. Scalar Multiplication: A binary operation '·' : F × V → V

These operations must satisfy the following ten axioms:

Axioms of Vector Addition:

• A1: Closure under Addition: For any u, v ∈ V, u + v ∈ V.
• A2: Associativity of Addition: For any u, v, w ∈ V, (u + v) + w = u + (v + w).
• A3: Commutativity of Addition: For any u, v ∈ V, u + v = v + u.
• A4: Existence of Zero Vector: There exists an element 0 ∈ V such that for any u ∈ V, u + 0 = u.
• A5: Existence of Additive Inverse: For every u ∈ V, there exists an element -u ∈ V such that u + (-u) = 0.

Axioms of Scalar Multiplication:

• M1: Closure under Scalar Multiplication: For any c ∈ F and v ∈ V, c · v ∈ V.
• M2: Distributivity of Scalar Multiplication over Vector Addition: For any c ∈ F and u, v ∈ V, c · (u + v) = c · u + c · v.
• M3: Distributivity of Scalar Multiplication over Field Addition: For any c, d ∈ F and v ∈ V, (c + d) · v = c · v + d · v.
• M4: Associativity of Scalar Multiplication: For any c, d ∈ F and v ∈ V, (c · d) · v = c · (d · v).
• M5: Multiplicative Identity: For any v ∈ V, 1 · v = v, where 1 is the multiplicative identity in F.

The set F is called the field of scalars. If F = ℝ, V is a real vector space. If F = ℂ, V is a complex vector space.

Examples of Vector Spaces:

• ℝⁿ: The set of all n-tuples of real numbers (x₁, x₂, ..., x<0xE2><0x82><0x99>) with component-wise addition and scalar multiplication. For example, ℝ² is the familiar 2D plane.
• P<0xE2><0x82><0x99>(x): The set of all polynomials of degree at most n with real coefficients.
• M_m×n(ℝ): The set of all m × n matrices with real entries.
• C[a, b]: The set of all continuous real-valued functions on the interval [a, b].

Subspaces:

A non-empty subset W of a vector space V is called a subspace of V if W itself forms a vector space under the same addition and scalar multiplication operations as V. To check if a subset W is a subspace, we only need to verify three conditions:

1. Non-empty: W must contain the zero vector of V. (0 ∈ W)
2. Closure under Addition: For any u, v ∈ W, u + v ∈ W.
3. Closure under Scalar Multiplication: For any c ∈ F and v ∈ W, c · v ∈ W.

Example: The set of all vectors in ℝ³ of the form (x, y, 0) forms a subspace of ℝ³ (it's the xy-plane).

Linear Combinations and Span:

Given vectors v₁, v₂, ..., v<0xE2><0x82><0x99> in a vector space V, a linear combination of these vectors is an expression of the form c₁v₁ + c₂v₂ + ... + c<0xE2><0x82><0x99>v<0xE2><0x82><0x99>, where c₁, c₂, ..., c<0xE2><0x82><0x99> are scalars from the field F.

The span of a set of vectors {v₁, v₂, ..., v<0xE2><0x82><0x99>}, denoted span({v₁, v₂, ..., v<0xE2><0x82><0x99>}), is the set of all possible linear combinations of these vectors. The span of a set of vectors is always a subspace.

Linear Independence and Dependence:

A set of vectors {v₁, v₂, ..., v<0xE2><0x82><0x99>} in a vector space V is said to be linearly independent if the only solution to the equation c₁v₁ + c₂v₂ + ... + c<0xE2><0x82><0x99>v<0xE2><0x82><0x99> = 0 is c₁ = c₂ = ... = c<0xE2><0x82><0x99> = 0.

If there exists a non-trivial solution (at least one cᵢ is non-zero), then the set of vectors is said to be linearly dependent. This means at least one vector in the set can be expressed as a linear combination of the others.

Basis and Dimension:

A basis for a vector space V is a set of vectors B = {v₁, v₂, ..., v<0xE2><0x82><0x99>} such that:

1. B is linearly independent.
2. B spans V (span(B) = V).

Every non-zero vector space has a basis. The basis is not unique, but the number of vectors in any basis for a given vector space is the same.

The dimension of a vector space V, denoted dim(V), is the number of vectors in any basis for V.

Examples:

• The standard basis for ℝⁿ is {(1, 0, ..., 0), (0, 1, ..., 0), ..., (0, 0, ..., 1)}. So, dim(ℝⁿ) = n.
• The standard basis for P<0xE2><0x82><0x99>(x) is {1, x, x², ..., xⁿ}. So, dim(P<0xE2><0x82><0x99>(x)) = n + 1.

Linear Transformations:

A function T: V → W between two vector spaces V and W is called a linear transformation (or linear map) if for all u, v ∈ V and all scalars c ∈ F:

1. T(u + v) = T(u) + T(v)
2. T(c · u) = c · T(u)

These two properties ensure that the structure of the vector spaces is preserved by the transformation.

The set of all linear transformations from V to W forms a vector space.

Kernel and Image (Range):

For a linear transformation T: V → W:

• The kernel of T, denoted ker(T) or N(T), is the set of all vectors in V that T maps to the zero vector in W: ker(T) = {v ∈ V | T(v) = 0<0xE1><0xB5><0xA2>}. The kernel is a subspace of V.
• The image (or range) of T, denoted Im(T) or R(T), is the set of all vectors in W that are images of vectors in V: Im(T) = {w ∈ W | w = T(v) for some v ∈ V}. The image is a subspace of W.

The Rank-Nullity Theorem states that for a linear transformation T: V → W, where V is finite-dimensional: dim(V) = dim(ker(T)) + dim(Im(T)) This theorem is crucial for understanding the relationship between the "null space" and the "range space" of a linear map.

2. Dual Spaces

A dual space is a space of linear functions (functionals) defined on another vector space.

Let V be a vector space over a field F. The dual space of V, denoted V* or V', is the set of all linear functionals from V to F. A linear functional is a linear map f: V → F.

The operations of vector addition and scalar multiplication are defined on V* as follows:

• For any f, g ∈ V* and v ∈ V, (f + g)(v) = f(v) + g(v).
• For any c ∈ F and f ∈ V*, (c · f)(v) = c · f(v).

With these operations, V* itself forms a vector space over F.

Relationship between Dimension of V and V*:

If V is a finite-dimensional vector space over F with dim(V) = n, then its dual space V* is also finite-dimensional and dim(V*) = dim(V) = n.

Canonical Isomorphism:

For a finite-dimensional vector space V, there is a natural (canonical) isomorphism between V and its bidual space V** (the dual space of V*). This means V is naturally isomorphic to V**. This is not generally true for infinite-dimensional spaces.

Dual Basis:

If B = {v₁, v₂, ..., v<0xE2><0x82><0x99>} is a basis for a finite-dimensional vector space V, then there exists a unique basis for V*, called the dual basis, denoted B* = {f¹, f², ..., fⁿ}, such that:
fⁱ(v_j) = δ_ij
where δ_ij is the Kronecker delta, which is 1 if i = j and 0 if i ≠ j.

This means that the functional fⁱ maps the basis vector v_i to 1 and all other basis vectors v_j (where j ≠ i) to 0.

Any linear functional f ∈ V* can be uniquely expressed as a linear combination of the dual basis vectors: f = a₁f¹ + a₂f² + ... + a<0xE2><0x82><0x99>fⁿ where a_i = f(v_i).

Dual of Subspaces and Quotient Spaces:

There are relationships between the dual of a subspace and the quotient space, and between the dual of a quotient space and a subspace.

• If W is a subspace of V, then V*/W° is isomorphic to W*. Here W° is the annihilator of W.
• If N is a subspace of V, then (V/N)* is isomorphic to N°. Here N° is the annihilator of N, which is the set of all functionals in V* that vanish on N.

3. Inner Product Spaces

An inner product space is a vector space equipped with an additional operation called an inner product. This operation allows us to define concepts like length (norm) and angle between vectors, which are familiar from Euclidean geometry.

Let V be a vector space over the field F (where F is ℝ or ℂ). An inner product on V is a function ⟨·, ·⟩ : V × V → F that satisfies the following properties for all u, v, w ∈ V and c ∈ F:

Properties of Inner Product:

• 1. Linearity in the first argument: ⟨cu + v, w⟩ = c⟨u, w⟩ + ⟨v, w⟩.
• 2. Conjugate symmetry: ⟨u, v⟩ = ⟨v, u⟩ (where the bar denotes complex conjugation). If V is a real vector space, this simplifies to symmetry: ⟨u, v⟩ = ⟨v, u⟩.
• 3. Positive-definiteness: ⟨v, v⟩ ≥ 0, and ⟨v, v⟩ = 0 if and only if v = 0.

A vector space V with an inner product is called an inner product space.

Norm (Length):

The inner product defines a norm (or length) on V, denoted ||v||, by: ||v|| = √⟨v, v⟩ This norm satisfies the properties of a norm:

• ||v|| ≥ 0, and ||v|| = 0 iff v = 0.
• ||cv|| = |c| ||v|| for any scalar c.
• ||u + v|| ≤ ||u|| + ||v|| (Triangle Inequality).

Angle and Orthogonality:

In real inner product spaces, the angle θ between two non-zero vectors u and v can be defined using the Cauchy-Schwarz inequality: |⟨u, v⟩| ≤ ||u|| ||v|| Equality holds if and only if u and v are linearly dependent. The cosine of the angle is given by: cos(θ) = ⟨u, v⟩ / (||u|| ||v||)

Two vectors u and v are said to be orthogonal if their inner product is zero: ⟨u, v⟩ = 0. If two vectors are orthogonal, they are geometrically perpendicular.

Orthonormal Basis:

An orthonormal basis for an inner product space V is a basis B = {e₁, e₂, ..., e<0xE2><0x82><0x99>} such that:

• The basis vectors are mutually orthogonal: ⟨e_i, e_j⟩ = 0 for i ≠ j.
• Each basis vector has unit length (is normalized): ||e_i|| = 1, which means ⟨e_i, e_i⟩ = 1.

So, ⟨e_i, e_j⟩ = δ_ij (Kronecker delta).

Every finite-dimensional inner product space has an orthonormal basis. The Gram-Schmidt process is a constructive algorithm to obtain an orthonormal basis from any given basis.

If an orthonormal basis is used, the representation of vectors and the computation of inner products become much simpler. For an orthonormal basis {e₁, ..., e<0xE2><0x82><0x99>}:

• Any vector v ∈ V can be written as v = ⟨v, e₁⟩e₁ + ⟨v, e₂⟩e₂ + ... + ⟨v, e<0xE2><0x82><0x99>⟩e<0xE2><0x82><0x99>.
• The inner product of two vectors u and v is ⟨u, v⟩ = Σ_i=1ⁿ ⟨u, e_i⟩⟨v, e_i⟩.
• The norm squared is ||v||² = Σ_i=1ⁿ |⟨v, e_i⟩|². (This is Bessel's Inequality, which becomes Parseval's Identity for a complete orthonormal set).

Examples of Inner Product Spaces:

• ℝⁿ with the dot product: For u = (u₁, ..., u<0xE2><0x82><0x99>) and v = (v₁, ..., v<0xE2><0x82><0x99>), the dot product is ⟨u, v⟩ = u ⋅ v = Σ_i=1ⁿ u_iv_i. The induced norm is the Euclidean norm ||v|| = √(v₁² + ... + v<0xE2><0x82><0x99>²).
• ℂⁿ with the standard inner product: For u = (u₁, ..., u<0xE2><0x82><0x99>) and v = (v₁, ..., v<0xE2><0x82><0x99>), ⟨u, v⟩ = Σ_i=1ⁿ u_i<0xE1><0xB5><0xAB>v_i. Note the complex conjugate on the first component.
• C[a, b] (space of continuous functions) with the integral inner product: For f, g ∈ C[a, b], ⟨f, g⟩ = ∫_a^b f(x)g(x) dx (for real-valued functions). The induced norm is ||f|| = √(∫_a^b [f(x)]² dx).

Adjoint Operators:

In the context of inner product spaces, we can define the adjoint of a linear operator. Let T: V → W be a linear operator between two inner product spaces V and W. The adjoint operator T*: W → V is the unique linear operator such that for all u ∈ V and w ∈ W: ⟨T(u), w⟩<0xE1><0xB5><0xA2> = ⟨u, T*(w)⟩<0xE1><0xB5><0xA3> where the subscripts indicate the inner product space in which the inner product is taken.

If V and W are finite-dimensional and we choose orthonormal bases, the matrix representation of T* is the conjugate transpose (or transpose for real spaces) of the matrix representation of T.

Operators with special properties related to their adjoints include:

• Self-adjoint (Hermitian) operators: T* = T.
• Unitary (Orthogonal) operators: T*T = I (identity operator). These preserve inner products and norms.
• Normal operators: T*T = TT*.

The Spectral Theorem states that for a self-adjoint (or normal) operator on a finite-dimensional inner product space, there exists an orthonormal basis of eigenvectors.