**Vector Space**

**Wanho Choi**

**(wanochoi.com)**

**Linear Combination of Vectors**

### •

**Combining**

** m n-dimensional vectors**

**m n-dimensional vectors**

**using **

**scalar multiplications**

** and **

**vector additions**

**y**

*= a*

_{1}

**x**

_{1}

*+ a*

_{2}

**x**

_{2}

*+ a*

_{3}

**x**

_{3}

*+!+ a*

_{m}**x**

_{m}### =

*a*

_{i}**x**

_{i}*i*=1

*m*

### ∑

*a*

_{i}**∈!, x**

_{i}### ∈!

*n*

### , m

### ∈"

## (

## )

**Vector Space**

### •

**A **

**n-dimensional space**

**n-dimensional space**

** in which any **

**linear **

**combination**

** of a set of vectors is also in that **

**space**

**y**

*= a*

_{1}

**x**

_{1}

*+ a*

_{2}

**x**

_{2}

*+ a*

_{3}

**x**

_{3}

*+!+ a*

_{m}**x**

_{m}### =

*a*

_{i}**x**

_{i}*i*=1

*m*

### ∑

**y**

### ∈!

*n*

*a*

_{i}**∈!, x**

_{i}### ∈!

*n*

### , m

### ∈"

## (

## )

**Linearly Dependent / Independent**

### •

** : linearly dependent**

### •

**Otherwise, they are **

**linearly independent**

**.**

**x**

_{1}

**,x**

_{2}

**,x**

_{3}

### ,

**!,x**

_{m}*iff a*

_{1}

*,a*

_{2}

*,a*

_{3}

### ,

*!,a*

_{m}### exist for

*a*

_{1}

**x**

_{1}

*+ a*

_{2}

**x**

_{2}

*+ a*

_{3}

**x**

_{3}

*+!+ a*

_{m}**x**

_{m}### =

*a*

_{i}**x**

_{i}*i*=1

*m*

### ∑

**= 0**

*a*

_{1}

*× a*

_{2}

*× a*

_{3}

*×!× a*

_{m}### ≠ 0

### (

### )

**Basis**

### •

**A set of n vectors**

**A set of n vectors**

** in a **

**n-dimensional space**

**n-dimensional space**

**, **

**which are **

**linearly independent**

** and every vector **

**in that space can be reproduced by a **

**linear **

**combination**

** of this set.**

*a*

_{1}

**e**

_{1}

*+ a*

_{2}

**e**

_{2}

*+ a*

_{3}

**e**

_{3}

*+!+ a*

_{n}**e**

_{n}### ∈!

*n*

*a*

_{i}**∈!, e**

_{i}### ∈!

*n*

**Orthonormal Basis**

### •

**A set of n vectors**

**A set of n vectors**

** in a **

**n-dimensional space**

**n-dimensional space**

**, **

**which are **

**linearly independent**

** and every vector **

**in that space can be reproduced by a **

**linear **

**combination**

** of this set.**

*a*

_{1}

**e**

_{1}

*+ a*

_{2}

**e**

_{2}

*+ a*

_{3}

**e**

_{3}

*+!+ a*

_{n}**e**

_{n}### ∈!

*n*

*a*

_{i}**∈!, e**

_{i}### ∈!

*n*

## (

## )

**e**

_{i}**⋅e**

_{j}### =

*0 (i*

*≠ j)*

*1 (i*

*= j)*

### ⎧

### ⎨

### ⎪

### ⎩⎪

**Span**

### •

**All **

**linear combination**

** of **

**basis vectors**

*a*

_{i}**∈!, e**

_{i}### ∈!

*n*

## (

## )

**Dimension & Rank**

### •

**A**

_{m×n}

**maps**

** vectors in **

### ℝ

n** to vectors in **

### ℝ

m**.**

### •

_{The }

_{The }

_{row space}

_{row space}

_{ of this matrix is the vector space }

_{ of this matrix is the vector space }

**generated by linear combinations of the row **

**vectors.**

### •

_{The }

_{The }

_{rank}

_{rank}

_{ of this matrix is }

_{ of this matrix is }

_{the maximum number }

_{the maximum number }

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

_{×n}

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

_{×n}

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

_{×n}

**r : the row rank of A**

**r : the row rank of A**

=
*n*
**A** *m*
*r*
**B** **C**_{n}*r*
*m*

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

_{×n}

**the row space of A is the span of the rows of C**

**the column space of A is the span of the columns of B**

**r : the row rank of A**

**r : the row rank of A**

=
*n*
**A** *m*
*r*
**B** **C**_{n}*r*
*m*

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

_{×n}

**the row space of A is the span of the rows of C**

**the column space of A is the span of the columns of B**

*a*_{11} *a*_{12}
*a*_{21} *a*_{22}
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥ =
*b _{1x}*

*b*

_{2 x}*b*

_{1y}*b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}*c*

_{1x}*c*

_{1y}*c*

_{2 x}*c*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ =

_{2 y}*b*

_{1x}c_{1x}*+ b*

_{2 x}c_{2 x}*b*

_{1x}c_{1y}*+ b*

_{2 x}c_{2 y}*b*

_{1y}c_{1x}*+ b*

_{2 y}c_{2 x}*b*

_{1y}c_{1y}*+ b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}c_{2 y}*a*

_{11}

*a*

_{12}⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ =

*b*

_{1x}c_{1x}*+ b*

_{2 x}c_{2 x}*b*

_{1x}c_{1y}*+ b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 x}c_{2 y}*= b1x*

*c*

_{1x}*c*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{1y}*+ b2 x*

*c*

_{2 x}*c*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

_{2 y}⎥: the 1st row vector
*a*_{21}
*a*_{22}
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥ =
*b _{1y}c_{1x}*

*+ b*

_{2 y}c_{2 x}*b*

_{1y}c_{1y}*+ b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}c_{2 y}*= b1y*

*c*

_{1x}*c*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{1y}*+ b2 y*

*c*

_{2 x}*c*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

_{2 y}⎥: the 2nd row vector

**r : the row rank of A**

**r : the row rank of A**

=
*n*
**A** *m*
*r*
**B** **C**_{n}*r*
*m*

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

_{×n}

**the row space of A is the span of the rows of C**

**the column space of A is the span of the columns of B**

*a*_{11} *a*_{12}
*a*_{21} *a*_{22}
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥ =
*b _{1x}*

*b*

_{2 x}*b*

_{1y}*b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}*c*

_{1x}*c*

_{1y}*c*

_{2 x}*c*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ =

_{2 y}*b*

_{1x}c_{1x}*+ b*

_{2 x}c_{2 x}*b*

_{1x}c_{1y}*+ b*

_{2 x}c_{2 y}*b*

_{1y}c_{1x}*+ b*

_{2 y}c_{2 x}*b*

_{1y}c_{1y}*+ b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}c_{2 y}*a*

_{11}

*a*

_{21}⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ =

*b*

_{1x}c_{1x}*+ b*

_{2 x}c_{2 x}*b*

_{1y}c_{1x}*+ b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}c_{2 x}*= c1x*

*b*

_{1x}*b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{1y}*+ c2 x*

*b*

_{2 x}*b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

_{2 y}⎥: the 1st column vector
*a*_{12}
*a*_{22}
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥ =
*b _{1x}c_{1y}*

*+ b*

_{2 x}c_{2 y}*b*

_{1y}c_{1y}*+ b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{2 y}c_{2 y}*= c1y*

*b*

_{1x}*b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥

_{1y}*+ c2 y*

*b*

_{2 x}*b*⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

_{2 y}⎥: the 2nd column vector

**r : the row rank of A**

**r : the row rank of A**

=
*n*
**A** *m*
*r*
**B** **C**_{n}*r*
*m*

**Row Rank = Column Rank**

**A**

_{m}

_{m}

_{×n}

_{×n}

**= B**

_{m}

_{m}

_{×r}

_{×r}

**C**

_{r}

_{r}

_{×n}

=
_{×n}

**the row space of A is the span of the rows of C**

**the column space of A is the span of the columns of B**

**r : the row rank of A**

**r : the row rank of A**

*m*
*n*
**A** *m*
*r*
**B** **C**_{n}*r*

**the row rank of A = the number of independent row vectors of C = r**

**the column rank of A = the number of independent column vectors of B = r**

**Row / Column Space**

*a*

_{11}

*a*

_{12}

*! a*

_{1n}*a*

_{21}

*a*

_{22}

*! a*

_{2n}### !

### !

### "

### !

*a*

_{m1}*a*

_{m 2}*! a*

_{mn}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

**A**

_{m}

_{m}

_{×n}

_{×n}

*a*

_{11}

*a*

_{12}

*! a*

_{1n}*a*

_{21}

*a*

_{22}

*! a*

_{2n}### !

### !

### "

### !

*a*

_{m1}*a*

_{m 2}*! a*

_{mn}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

Row Space: the subspace of !*n*** spanned by the row vectors of A**

### row vectors

### ∈!

*n*

### column vectors

### ∈!

*m*

Column Space

*a*

_{11}

*a*

_{12}

*! a*

_{1n}*a*

_{21}

*a*

_{22}

*! a*

_{2n}### !

### !

### "

### !

*a*

_{m1}*a*

_{m 2}*! a*

_{mn}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

*x*

_{1}

*x*

_{2}

### !

*x*

_{n}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

### =

*b*

_{1}

*b*

_{2}

### !

*b*

_{m}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

**Ax**

**= b**

*m× n*

*n*×1

*m*×1

*x*

_{1}

*a*

_{11}

*a*

_{21}

### !

*a*

_{m1}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

*+ x*

_{2}

*a*

_{12}

*a*

_{22}

### !

*a*

_{m 2}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

*+!+ x*

_{n}*a*

_{1n}*a*

_{2n}### !

*a*

_{mn}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

### =

*b*

_{1}

*b*

_{2}

### !

*b*

_{m}### ⎡

### ⎣

### ⎢

### ⎢

### ⎢

### ⎢

### ⎢

### ⎤

### ⎦

### ⎥

### ⎥

### ⎥

### ⎥

### ⎥

**Row / Column Space**

*The solution (x*

_{1}

*, x*

_{2}

### ,

*!, x*

_{n}### ) exists

**iff b is in the column space of A.**

**Ax**

**= 0**

*m× n* *n*×1 *m*×1

**Null Space**

Null Space

**: the set of vectors x such that Ax=0.**