**Eigenvalue **

**Decomposition**

**Wanho Choi**
**(wanochoi.com)**

**Eigenvalues & Eigenvectors**

**Av**

_{i}### =

### λ

_{i}**v**

_{i}**A : n by n matrix**

**A : n by n matrix**

**v**

_{i}*: the i-th eigenvector*

### λ

_{i}*: the i-th eigenvalue*

### λ

_{i}**, v**

_{i}### (

### )

*: the i-th eigen-pair*

### (

*i : 1, 2, 3, ! , n*

### )

**Geometrical Meaning**

**Av**

_{i}### =

### λ

_{i}**v**

_{i}**The transformed vector (Ax) is a scalar multiple of the vector (x).**

**Ax is parallel to x.**

*eigen* *= characteristic*

*eigenvector : strectching direction*
*eigenvalue: stretching factor*

https://www.youtube.com/watch?v=8UX82qVJzYI

**When A is a symmetric matrix, …**

• Two different **eigenvectors** are **orthogonal** each
other.
**v*** _{i}*T

**Av**

_{j}**= v**

*T*

_{i}### ( )

**Av**

_{j}**= v**

*T*

_{i}### ( )

λ

_{j}**v**

_{j}**= v**

*Tλ*

_{i}

_{j}**v**

*= λ*

_{j}

_{j}**v**

*T*

_{i}**v**

_{j}**v**

*T*

_{i}**Av**

_{j}**= v**

*T*

_{i}**A**T

**v**

_{j}**= v**

### (

*T*

_{i}**A**T

### )

**v**

_{j}**= Av**

### ( )

*T*

_{i}**v**

*=*

_{j}### ( )

λ

_{i}**v**

*T*

_{i}**v**

*= λ*

_{j}

_{i}**v**

*T*

_{i}**v**

*⇒ λ*

_{j}

_{j}**v**

*T*

_{i}**v**

*= λ*

_{j}

_{i}**v**

*T*

_{i}**v**

*⇔*

_{j}### (

λ*− λ*

_{i}

_{j}### )

**v**

*T*

_{i}**v**

_{j}**= 0**

**∴v**

*T*

_{i}**v**

_{j}**= 0 ∵**

### (

λ*≠ λ*

_{i}

_{j}### )

**Av**

*= λ*

_{i}

_{i}**v**

_{i}**, Av**

*= λ*

_{j}

_{j}**v**

_{j}### (

λ*≠ λ*

_{i}

_{j}, i*≠ j*

### )

**A**T

**= A**

**How to calculate eigenvalues**

**Av**

_{i}### =

### λ

_{i}**v**

_{i}**⇔ A −**

### (

### λ

_{i}**I**

### )

**v**

_{i}**= 0**

**⇔ det A −**

### (

### λ

_{i}**I**

### )

### = 0

### λ

_{i}*: the i-th roots of the n-th order polynomial*

**Invertible Matrix Theorem**

• **The matrix A is invertible**

if and only if **0 is not an eigenvalue of A.**

**Av*** _{i}* = λ

_{i}**v**

_{i}**If A**-1

**is exist, then v**

_{i}**= A**-1λ

_{i}**v**

*.*

_{i}**∴A**-1

**v**

*= 1 λ*

_{i}

_{i}**v**

*i*

**⇔ A**-1 = 1 λ1 0 ! 0 0 1 λ

_{2}! 0 " " # " 0 0 0 1 λ

*⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥*

_{n}**Some Useful Properties**

•
•
•
•
**det(A)**= λ

_{1}× λ

_{2}×!× λ

_{n}*= λ*

**trace(A)**_{1}+ λ

_{2}+!+ λ

_{n}* rank(A)* = the number of non-zero eigenvalues

**Eigenvalue(Spectral) Decomposition**

**AV**=

**− a**1 −

**− a**2 − !

**− a**

*n*− ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ | | |

**v**

_{1}

**v**

_{2}

**! v**

*n*| | | ⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥=

**a**

_{1}

**⋅v**

_{1}

**a**

_{1}

**⋅v**

_{2}

**! a**1

**⋅v**

*n*

**a**

_{2}

**⋅v**

_{1}

**a**

_{2}

**⋅v**

_{2}

**! a**2

**⋅v**

*n*! ! " !

**a**

*n*

**⋅v**1

**a**

*n*

**⋅v**2

**! a**

*n*

**⋅v**

*n*⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ = | | |

**Av**1

**Av**2

**! Av**

*n*| | | ⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥= | | | λ1

**v**1 λ2

**v**2 ! λ

*n*

**v**

*n*| | | ⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥= | | |

**v**1

**v**2

**! v**

*n*| | | ⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥ λ1 0 ! 0 0 λ

_{2}! 0 ! ! " ! 0 0 ! λ

*n*⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥

**= VΛ**

**Av**

*= λ*

_{i}

_{i}**v**

_{i}**⇔ AV = VΛ ⇔ A = VΛV**T

**⇔ A =**λ

_{i}*i*=1

*n*

### ∑

**v**

_{i}**v**

*T*

_{i}**V**−1

**= V**T

**V : orthonormal**( )

**A**= | ! |

**v**

_{1}

**! v**

*| ! | ⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥ λ*

_{n}_{1}! 0 " # " 0 ! λ

*⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥*

_{n}**− v**

_{1}− ! " !

**− v**

*− ⎡ ⎣ ⎢ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ ⎥*

_{n}*a b*

*c d*⎡ ⎣ ⎢ ⎤ ⎦ ⎥⎡ α 0

_{0}

_{β}⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

*a*

*c*

*b d*⎡ ⎣ ⎢ ⎤ ⎦ ⎥ =⎡

*a b*⎣ ⎢ ⎤ ⎦ ⎥⎡

_{c d}*αa αc*⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ =

_{βb βd}*αaa + βbb αac + βbd*

*αac + βbd αcc + βdd*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥=

*α aa acac*

*cc*⎡ ⎣ ⎢ ⎤ ⎦ ⎥ + β⎡

*bb bd*⎣ ⎢ ⎤ ⎦ ⎥ =

_{bd dd}*α a*

*c*⎡ ⎣ ⎢ ⎤ ⎦

*⎥ a c*⎡

_{⎣}⎤

_{⎦ + β}⎡

*b*⎣ ⎢ ⎤ ⎦

_{d}*⎥ b d*⎡

_{⎣}⎤

_{⎦}

**rotation**

**inverse rotation**

**stretching**

**Eigenspace**

*n unknowns*

*n equations*

**Ax**

**= b**

**VΛV**T

### (

### )

**x**

**= b**

**Λ V**T

**x**

### ( )

**= V**T

**b**

**Λy = d**

**V**T

**x**

**= y**

**x**

**= Vy**

**linear system**

**(globally coupled system)**
**eigenvalue decomposition**

**decoupled n equations****in eigenspace**

**Eigenvalue Analysis**

• It gives the **solution of Ax=b**, and **simplifies** the

solution of a certain problem by **reducing** a coupled
system to a collection of scalar problems.

• _{It gives insight into the }_{behavior of evolving }

**systems** governed by linear equation.

**A***m* **= VΛV**

### (

T### )

*m*

**= VΛV**

### (

T### )

### (

**VΛV**T

### )

**! VΛV**

### (

T### )

**= VΛV**T

**VΛV**T

### (

### )

**! VΛV**

### (

T### )

**= VΛIΛV**

### (

T### )

**! VΛV**

### (

T### )

**= VΛΛV**

### (

T### )

**! VΛV**

### (

T### )

**= VΛ**2

**V**T

### (

### )

**! VΛV**

### (

T### )

= !**= VΛ**

*m*

**V**T

**Ax**

**= b ⇔ VΛV**

### (

T### )

**x**

**= b ⇔ Λ V**

### ( )

T**x**

**= V**T

**b**

**⇔ Λy = d**

**Applications of Eigenvalue Analysis**

• Analyzing **natural frequency** in vibration

• Analyzing **stability** or **convergence** in iteration

• **Approximation** with lower computation

**Applications of Eigenvalue Analysis**

• Analyzing **natural frequency** in vibration

• _{Analyzing }_{stability}_{ or }_{convergence}_{ in iteration}

• **Approximation** with lower computation

*y _{n}*

*= 1+ Δt*( λ)

*n*

*y*

_{0}(λ < 0 in this case) : It converges when 1

*+ Δtλ ≤ 1.*

*⇔ −1 < 1+ Δt*λ < 1

*⇔ − 2 < Δtλ < 0*

*∴Δt < −*2 λ

**y**

_{n}_{+1}

**= y**

_{n}*)*

**+ Δt ⋅ f (y**_{n}**= y**

_{n}

**+ Δt ⋅Ay**_{n}**= y**

_{n}

**+ Δt ⋅ Λy**_{n}*( )*

**= I + Δt ⋅ Λ****y**

_{n}**Applications of Eigenvalue Analysis**

• Analyzing **natural frequency** in vibration

• _{Analyzing }_{stability}_{ or }_{convergence}_{ in iteration}

• _{Approximation}_{ with lower computation}

**A** = λ_{i}*i*=1
*n*

### ∑

**v**

_{i}**v**

*T ≈ λ*

_{i}

_{i}*i*=1

*m*

### ∑

**v**

_{i}**v**

*T*

_{i}*m*(

*≪ n*)

**Square-Root Matrix**

**A**−1

**= VΛ**−1

**V**T

**∵ VΛ**

### (

−1**V**T

### )

### (

**VΛV**T

### )

**= VΛ**−1

### ( )

**V**T

**V**

**ΛV**T

**= VΛ**−1

**IΛV**T

**= VΛ**−1

**ΛV**T

**= VV**T

**= I**

**A**1/2

**= VΛ**1/2

**V**T

**∵ VΛ**1/2

**V**T

### (

### )

### (

**V**Λ1/2

**V**T

### )

**= VΛ**1/2

### ( )

**V**T

**V**Λ1/2

**V**T

**= VΛ**1/2

**I**Λ1/2

**V**T

**= VΛ**1/2Λ1/2

**V**T

**= VΛV**T

**= A**

**∴A**1/2

**= VΛ**1/2

**V**T = λ

_{i}*i*=1

*n*

### ∑

**v**

_{i}**v**

*T*

_{i}**Similarity Transformation**

• **A and B are similar**, if there exists a non-singular
**matrix X such as B=XAXT**_{.}

• _{A and B }_{shares}_{ many }_{properties}_{.}

• _{If X is non-singular, then A and B have the }_{same}

characteristic polynomial and **eigenvalues**.