Eigenvectors and Eigenvalues
1 Eigenvalues and Eigenvectors
Eigenvalues and eigenvectors are special scalars and vectors (numbers and directions) that describe how a linear transformation (matrix) acts on a [[vector space]].
An eigenvector is a non-zero vector whose direction does not change when a matrix transformation is applied to it; it only gets stretched, compressed or reflected.
An eigenvalue is the factor by which an eigenvector is stretched or compressed during a matrix transformation.
2 Eigenvalue Equation
For a square matrix \(\mathbf{A} \in \mathbb{R}^{n \times n}\), a non-zero vector \(\mathbf{q}\) and scalar \(\lambda\) satisfying:
\[ \mathbf{A}\mathbf{q} = \lambda \mathbf{q} \]
are called an eigenvector and its associated eigenvalue, respectively. The eigenvalue equation says that \(\mathbf{A}\) stretches (or reflects) \(\mathbf{q}\) by factor \(\lambda\) without changing its direction.
Eigenvalues are found by solving the characteristic equation
\[ \det(\mathbf{A} - \lambda \mathbf{I}) = 0. \]
3 Eigendecomposition
A symmetric matrix \(\mathbf{A} \in \mathbb{R}^{n \times n}\) always has \(n\) real eigenvalues \(\lambda_1, \ldots, \lambda_n\) and \(n\) orthonormal eigenvectors \(\mathbf{q}_1, \ldots, \mathbf{q}_n\) (by the spectral theorem). We can write:
\[ \mathbf{A} = \mathbf{Q} \boldsymbol{\Lambda} \mathbf{Q}^\top, \]
where \(\mathbf{Q} = [\mathbf{q}_1 \mid \cdots \mid \mathbf{q}_n]\) is orthogonal and \(\boldsymbol{\Lambda} = \mathrm{diag}(\lambda_1, \ldots, \lambda_n)\).
4 Trace and Determinant via Eigenvalues
Two convenient identities link eigenvalues to matrix invariants:
\[ \mathrm{tr}(\mathbf{A}) = \sum_{i=1}^n \lambda_i, \qquad \det(\mathbf{A}) = \prod_{i=1}^n \lambda_i. \]
These are useful for understanding properties of the hat matrix later.