x x
A = λ
Chap. 7. Linear Algebra: Matrix Eigenvalue Problems
x = 0: (no practical interest)
x ≠ 0: eigenvectors of A; exist only for certain values of λ (eigenvalues or characteristic roots)
Multiplication of A = same effect as the multiplication of x by a scalar λ
Important to determine the stability of chemical & biological processes
- Eigenvalue: special set of scalars associated with a linear systems of equations.
Each eigenvalue is paired with a corresponding eigenvectors.
7.1. Eigenvalues, Eigenvectors
- Eigenvalue problems:square matrix
eigenvectors
unknown scalar
( A I ) x 0
or x
x
A = λ − λ =
unknown vector
eigenvectors Set of eigenvalues: spectrum of A
How to Find Eigenvalues and Eigenvectors Ex. 1.)
−
= −
2 2
2 A 5
2 2
1
1 2
1
x x
2 x
2
x x
2 x
5
λ
=
−
λ
= +
− ( A − λ I ) x = 0
In homogeneous linear system, nontrivial solutions exist when det (A-λI)=0.
0 6 2 7
2
2 ) 5
I A
det(
) (
D = λ
2+ λ + =
λ
−
− λ
−
= − λ
−
= λ
Characteristic equation of A:
Characteristic determinant
Characteristic polynomial
Eigenvalues: λ1=-1 and λ2= -6
Eigenvectors: for λ1=-1, for λ2=-6,
= 2
x
11
= −
1 x
22
obtained from Gauss elimination
General Case
Theorem 1:
Eigenvalues of a square matrix A roots of the characteristic equation of A.
nxn matrix has at least one eigenvalue, and at most n numerically different eigenvalues.
Theorem 2:
If x is an eigenvector of a matrix A, corresponding to an eigenvalue λ, so is kx with any k≠0.
Ex. 2) multiple eigenvalue
- Algebraic multiplicity of λ: order Mλ of an eigenvalue λ
Geometric multiplicity of λ: number of mλ of linear independent eigenvectors corresponding to λ. (=dimension of eigenspace of λ) In general, mλ ≤Mλ
Defect of λ: Δλ=Mλ-mλ
n n
nn 2
2 n 1
1 n
1 n
n 1 2
12 1
11
x x
a x
a x
a
x x
a x
a x
a
λ
= +
+ +
λ
= +
+ +
( A − λ I ) x = 0 , D ( λ ) = det ( A − λ I ) = 0
Ex 3) algebraic & geometric multiplicity, positive defect
Ex. 4) complex eigenvalues and eigenvectors
7.2. Some Applications of Eigenvalue Problems
Ex. 1) Stretching of an elastic membrane.Find the principal directions: direction of position vector x of P
= (same or opposite) direction of the position vector y of Q
=
=
= +
2 1 2
2 1 2 2
1
x
x 5 3
3 5 y
y y , 1 x
x
= −
=
=
=
=
=
1 for 1
x , 2
1 for 1
x , 8 x
x A y
2 2
2
1 1
1
λ λ
λ λ
λ
Eigenvalue represents speed of response Eigenvector ~ direction
Ex. 4) Vibrating system of two masses on two springs
Solution vector:
solve eigenvalues and eigenvectors
Examples for stability analysis of linear ODE systems using eigenmodes Stability criterion: signs of real part of eigenvalues of the matrix
A determine the stability of the linear system.
Re(λ) < 0: stable Re(λ) > 0: unstable Ex. 1) Node-sink
2 1
'' 2
2 1
'' 1
y 2 y
2 y
y 2 y
5 y
−
=
+
−
=
e
wtx y =
) t 6 sin b
t 6 cos a
( x ) t sin b t cos a
( x y
) w (
x x
A
2 2
2 1
1 1
2
+ +
+
=
=
= λ λ
x dt A
x x = d =
2 2
2 1
1
x 2 x
x x
5 . 0 x
−
=
+
−
=
2 5 . 0
2 1
−
=
−
= λ
λ
stableEx. 2) Saddle
Ex. 3) Unstable focus
Ex. 4) Center
2 1
2
2 1
1
x x
2 x
x x
2 x
−
=
+
=
=
=
= −
−
=
4896 .
0
8719 .
for 0 x
, 5616 .
2
9628 .
0
2703 .
for 0 x
, 5616 .
1
2 2 2
1 1
1
λ λ
λ
λ
unstable2 1
2
2 1
1
x x
2 x
x 2 x
x
+
−
=
+
=
i 2 1
i 2 1
2 1
−
= +
= λ
λ
unstablePhase plane ?
Phase plane ?
2 1
2
2 1
1
x x
4 x
x x
x
+
=
−
−
=
i 7321 .
1 0
i 7321 .
1 0
2 1
−
= +
=
λ
λ
7.3. Symmetric, Skew-Symmetric, and Orthogonal Matrices
- Three classes of real square matrices(1) Symmetric:
(2) Skew-symmetric:
(3) Orthogonal:
Theorem 1:
(a) The eigenvalues of a symmetric matrix are real.
(b) The eigenvalues of a skew-symmetric matrix are pure imaginary or zero.
−
−
−
−
=
−
=
0 20
12
20 0
9
12 9
0 ,
a a
, A
A
T kj jk
−
−
−
=
=
4 2
5
2 0
1
5 1
3 ,
a a
, A
A
T kj jk
−
−
= −
3 2 3
2 3 1
3 1 3 2 3 2
3 2 3 1 3 2
, A
AT 1
( )
( A A ) skew symmetric
2 S 1
symmetric A
2 A R 1
, S R A
T T
−
−
=
+
= +
=
Zero-diagonal terms
) real , A A ( k k
A k
k A : Conjugate k
k
A =λ =λ =λ =
Transpose, and then multiply k: kTATk = kTλk kTλk = kTλ k λ kTk =λkTk
Ex. 3)
Orthogonal Transformations and Matrices
- Orthogonal transformation in the 2D plane and 3D space: rotation Theorem 2: (Invariance of inner product)
An orthogonal transformation preserves the value of the inner product of vectors.
the length or norm of a vector in Rn given by
Theorem 3: (Orthonormality of column and row vectors)
A real square matrix is orthogonal iff its column (or row) vectors, a1,… , an form an orthonormal system
i 25 ,
0 0
20 12
20 0
9
12 9
0
±
= λ
−
−
−
) matrix orthogonal
: A ( x A y =
8 , 5 2
3 3
5 λ =
θ θ
θ
−
= θ
=
2 1 2
1
x x cos
sin
sin cos
y y y
) Ex
) vectors column
: b , a ( b a b
a ⋅ =
Ta a a
a
a = ⋅ =
T
=
= ≠
=
⋅ 1 if j k
k j if a 0
a a
a
j k jT kb a b a b ) A A ( a
b A A a ) b A ( ) a A ( v u v u
) orthogonal :
A ( b A v , a A u
T 1
T
T T T
T
⋅
=
=
=
=
=
=
⋅
=
=
−
(
1 n)
T n T 1 T
1 a a
a a , A A I A
A
=
− =
Theorem 4: The determinant of an orthogonal matrix has the value of +1 or –1.
Theorem 5: Eigenvalues of an orthogonal matrix A are real or complex conjugates in pairs and have absolute value 1.
7.4. Complex Matrices: Hermitian, Skew-Hermitian, Unitary
- Conjugate matrix:- Three classes of complex square matrices:
(1) Hermitian:
(2) Skew-Hermitian:
(3) Unitary:
kj T
jk
, A a
a
A = =
+
−
= −
−
−
−
= +
i 2 6 i
5
7 i
4 A 3
i 2 6 7
i 5 i
4
A 3 T
+
= −
= 1 3 i 7
i 3 1 , 4
a a
, A
A
T kj jk
− +
−
− +
=
−
= 2 i i
i 2 i
, 3 a a
, A
A
T kj jk
= −
2i 3 1
2 1
2 3 i 1 2 1 ,
A AT 1
Diagonal-terms: real
Diagonal-terms:
pure imag. or 0
T 2 T
1
) det( A A ) det A det A (det A ) A
A det(
I det
1 = =
−= = =
jj jj a a =
jj
jj a
a = −
- Generalization of section 7.3
Hermitian matrix: real symmetric
Skew-Hermitian matrix: real skew-symmetric Unitary matrix: real orthogonal
Eigenvalues
Theorem 1:
(a) Eigenvalues of Hermitian (symmetric) matrix real
(b) Skew-Hermitian (skew-symmetric) matrix pure imag. or zero (c) Unitary (orthogonal) matrix absolute value 1
Forms
: a form in the components x1,… , xn of x, A coefficient matrix
A A
A
T=
T=
A A
A
T=
T= −
1 T T
A A
A = =
−x A x
T[ ]
11 1 1 12 1 2 21 2 1 22 2 22 1
22 21
12 11
2 1
T
a x x a x x a x x a x x
x x a
a
a x a
x x
A
x = + + +
=
Re λ Im λ Skew-Hermitian
Hermitian Unitary
Proof of Theorem 1:
(a) Eigenvalues of Hermitian (symmetric) matrix real
(b) Eigenvalues of Skew-Hermitian (skew-symmetric) matrix pure imag. or zero
(c) Eigenvalues of Unitary (orthogonal) matrix absolute value 1
(
x Ax)
x A x x Ax(
x Ax)
x A x
) A A
, A A
use
! ( real
? real x
x x A x
x x x
x x A x x
x A
T T
T T T
T T
T T T
T
T T
T
=
=
=
=
=
=
=
=
=
=
= λ
λ λ
λ
(
x Ax)
x A
xT = − T
( ) ( ) ( ) ( )
Ax Ax( )
x( )
x x x x x x xx x
A : transpose conjugate
x x
A
T 2 T
T T T
T T
=
=
=
=
=
=
λ λ
λ λ
λ
λ λ
n n nn 1
n 1 n
n 2 n 2 1
2 21
n 1 n 1 1
1 11 n
1 j
n
1 k
k j jk T
x x a x
x a
x x a x
x a
x x a x
x a x
x a x
A x
+ +
+ +
+ +
+
+ +
=
=
= =
- For general n,
- For real A, x,
Quadratic form
- Hermitian A: Hermitian form, Skew-Hermitian A: Skew-Hermitian form
Theorem 1: For every choice of the vector x, the value of a Hermitian form is real, and the value of a skew-Hermitian form is pure imaginary or 0.
2 n nn 2
n 2 n 1
n 1 n
n 2 n 2 2
2 22 1
2 21
n 1 n 1 2
1 12 2
1 11 n
1 j
n
1 k
k j jk T
x a x
x a x
x a
x x a x
a x
x a
x x a x
x a x
a x
x a x
A x
+ +
+ +
+ +
+
+ +
+
=
=
= =
Properties of Unitary Matrices. Complex Vector Space Cn. - Complex vector space: Cn
Inner product:
length or norm:
Theorem 2: A unitary transformation, y=Ax (A: unitary matrix) preserves the value of the inner product and norm.
- Unitary system: complex analog of an orthonormal system of real vectors
Theorem 3: A square matrix is unitary iff its column vectors form a unitary system.
Theorem 4: The determinant of a unitary matrix has absolute value 1.
b a b
a⋅ = T
2 n 2
1
T
a a a
a a
a
a = ⋅ = = + +
=
= ≠
=
⋅ 1 if j k
k j if a 0
a a
a
j k jT kb a b a b A A a ) b A ( ) a A ( v u v
u⋅ = T = T = T T = T = ⋅
2
T 1 T
A det A
det A det
A det A det A
det A det )
A A det(
) A A det(
I det 1
=
=
=
=
=
=
=
−7.5. Similarity of Matrices, Basis of Eigenvectors, Diagonalization
-Eigenvectors of n x n matrix A forming a basis for Rn or Cn ~ used for diagonalizing A
Similarity of Matrices
- n x n matirx is similar to an n x n matrix A if for nonsingular n x n P Similarity transformation:
Theorem 1: has the same eigenvalues as A if is similar to A.
is an eigenvector of corresponding to the same eigenvalue, if x is an eigenvector of A.
Properties of Eigenvectors
Theorem 2: λ1, λ2, …, λn: distinct eigenvalues of an n x n matrix.
Corresponding eigenvectors x1, x2, …, xn a linearly independent set.
( ) ( )
P x P xAˆ x
P P A P x I A P x A P
x P x
A P x
x A
1 1
1 1
1 1
1 1
−
−
−
−
−
−
−
−
=
=
=
=
=
=
λ λ
λ
P A P Aˆ = −1 Aˆ
A from Aˆ
Aˆ Aˆ
x P
y = −1 Aˆ
Theorem 3: n x n matrix A has n distinct eigenvalues A has a basis of eigenvector for Cn (or Rn).
Ex. 1)
Theorem 4: A Hermitian, skew-Hermitian, or unitary matrix has a basis of eigenvectors for Cn that is a unitary system.
A symmetric matrix has an orthonomal basis of eigenvectors for Rn.
Ex. 3) From Ex. 1, orthonormal basis of eigenvectors
- Basis of eigenvectors of a matrix A: useful in transformation and diagonalization
Complicated calculation of A on x sum of simple evaluation on the eigenvectors of A.
−
=
1 , 1 1 rs 1 eigenvecto of
basis 5 a
3 3 A 5
−
2 / 1
2 / , 1 2 / 1
2 / 1
( )
n n n 1
1 1
n n 2
2 1
1
n 1
n n 2 2
1 1
x c
x c
x c x
c x
c A y
) basis :
x ..., , x ( x c x
c x
c x , x A y
λ λ
+ +=
+ +
+
=
+ +
+
=
=
(
1 2)
1T 2 1 22 T 2 1 2
T 1 1
2 T 2 1 2 2
T 1 2
T 1 T
1
2 T 1 1 1 2
T 1 2
T T 1 1
T 1 T
T 1 2
2 T 1 2 1 2 2
2 ) 2 ( 1 1
1 ) 1 (
, x x 0
x x x
x
x x x
x x
A x : left the on x Multiply )
2 (
x x x
x x
A x x
A x : right the
on x multiply then
, Transpose )
1 (
0 x
x x
x show
; x x
A ,
x x
A
λ λ λ
λ λ
λ
λ λ
λ λ
λ λ
λ
≠
−
=
=
=
=
=
=
→
=
=
=
⋅
=
=
Diagonalization
Theorem 5: If an n x n matrix A has a basis of eigenvectors, then
is diagonal, with the eigenvalues of A on the main diagonal.
(X: matrix with eigenvectors as column vectors)
n x n matrix A is diagonalizable iff A has n linearly independent eigenvectors.
Sufficient condition for diagonalization: If an n x n matrix A has n distinct eigenvalues, it is diagonalizable.
Ex. 4)
Ex. 5) Diagonalization X A X D = −1
...) , 3 , 2 m ( X A X
Dm = −1 m =
=
= −
−
= −
1 0
0 X 6
A X 1 ; 1
1 X 4
1 ; , 1 1 rs 4 eigenvecto 2 ;
1 4
A 5 1
[ ] [ ] [ ]
X A X X A A X X A X X A X D
D X A X
D X x
x x
A x
A x
x A X A
2 1 1
1 1
2
1
n n 1
1 n
1 n
1
−
−
−
−
−
=
=
=
=
=
=
=
=
λ
λ
x1,…,xn: basis of eigenvectors of A for Cn (or Rn) corresponding to λ1, …, λn
Transformation of Forms to Principal Axes Quadratic form:
If A is real symmetric, A has an orthogonal basis of n eigenvectors
X is orthogonal.
Theorem 6: The substitution x=Xy, transforms a quadratic form
to the principal axes form
where λ1,…, λn eigenvalues of the symmetric Matrix A, and X is
orthogonal matrix with corresponding eigenvectors as column vectors.
Ex. 6) Conic sections.
x A x Q = T
1
T X
X = − A = XDX−1 = XDXT
(
y xX XxDXX xx, yx DXyy)
y y yQ
1 T
2 n n 2
n 1 2 1 1 T
T T
=
=
=
+ + +
=
=
=
−
λ λ
λ
= ==
= n
1 j
n
1 k
k j jk
TAx a x x
x Q
2 n n 2
n 1 2 1 1
TDy y y y
y
Q = =λ +λ ++λ
Example) Solution of linear 1st-order Eqn.:
Ex.)
) 0 ( y X e X ) t ( z X ) t ( y
) 0 ( z e z z
D z
) 0 ( z
) 0 ( z e
0
0 e
) t ( z
) t ( z z
z 0
0 z
z
z D z z
X A z X
y X z z
X y : Define
y dt A
y y d
1 t D t D
2 1 t
t
2 1 2
1 2
1 2
1
1
2 1
−
−
=
=
=
=
=
=
=
→
=
=
→
=
=
=
λ λ
λ λ
2 2
2 1
1
y 2 y
y y
5 . 0 y
−
=
+
−
=
) 0 ( 8321 y
. 0 0
5547 .
0 1
e 0
0 e
8321 .
0 0
5547 .
0 ) 1
t ( y
2 8321 for
. 0
5547 .
y 0 , 5 . 0 0 for
y 1
1 t
2 t
5 . 0
2 2 1 1
−
−
−
