Vector Space

Vector Space

Wanho Choi

(wanochoi.com)

Linear Combination of Vectors

•

Combining

m n-dimensional vectors

using

scalar multiplications

and

vector additions

y

= a

₁

x

₁

+ a

₂

x

₂

+ a

₃

x

₃

+!+ a

_m

x

_m

=

a

_i

x

_i i=1 m

∑

a

_i

∈!, x

_i

∈!

n

, m

∈"

(

)

Vector Space

•

A

n-dimensional space

in which any

linear

combination

of a set of vectors is also in that

space

y

= a

₁

x

₁

+ a

₂

x

₂

+ a

₃

x

₃

+!+ a

_m

x

_m

=

a

_i

x

_i i=1 m

∑

y

∈!

n

a

_i

∈!, x

_i

∈!

n

, m

∈"

(

)

Linearly Dependent / Independent

•

: linearly dependent

•

Otherwise, they are

linearly independent

.

x

₁

,x

₂

,x

₃

,

!,x

_m

iff a

₁

,a

₂

,a

₃

,

!,a

_m

exist for

a

₁

x

₁

+ a

₂

x

₂

+ a

₃

x

₃

+!+ a

_m

x

_m

=

a

_i

x

_i i=1 m

∑

= 0

a

₁

× a

₂

× a

₃

×!× a

_m

≠ 0

(

)

Basis

•

A set of n vectors

in a

n-dimensional space

,

which are

linearly independent

and every vector

in that space can be reproduced by a

linear

combination

of this set.

a

₁

e

₁

+ a

₂

e

₂

+ a

₃

e

₃

+!+ a

_n

e

_n

∈!

n

a

_i

∈!, e

_i

∈!

n

Orthonormal Basis

•

A set of n vectors

in a

n-dimensional space

,

which are

linearly independent

and every vector

in that space can be reproduced by a

linear

combination

of this set.

a

₁

e

₁

+ a

₂

e

₂

+ a

₃

e

₃

+!+ a

_n

e

_n

∈!

n

a

_i

∈!, e

_i

∈!

n

(

)

e

_i

⋅e

_j

=

0 (i

≠ j)

1 (i

= j)

⎧

⎨

⎪

⎩⎪

Span

•

All

linear combination

of

basis vectors

a

_i

∈!, e

_i

∈!

n

(

)

Dimension & Rank

•

A

_m×n

maps

vectors in

ℝ

n

to vectors in

ℝ

m

.

•

_The

_{row space}

_{of this matrix is the vector space}

generated by linear combinations of the row

vectors.

•

_The

_rank

_{of this matrix is}

_{the maximum number}

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

r : the row rank of A

= n A m r B C_n r m

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

the row space of A is the span of the rows of C

the column space of A is the span of the columns of B

r : the row rank of A

= n A m r B C_n r m

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

the row space of A is the span of the rows of C

the column space of A is the span of the columns of B

a₁₁ a₁₂ a₂₁ a₂₂ ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1x b_{2 x} b_1y b_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ c_1x c_1y c_{2 x} c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1xc_1x + b_{2 x}c_{2 x} b_1xc_1y + b_{2 x}c_{2 y} b_1yc_1x + b_{2 y}c_{2 x} b_1yc_1y + b_{2 y}c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ a₁₁ a₁₂ ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1xc_1x + b_{2 x}c_{2 x} b_1xc_1y + b_{2 x}c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b1x c_1x c_1y ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥+ b2 x c_{2 x} c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

⎥: the 1st row vector a₂₁ a₂₂ ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1yc_1x + b_{2 y}c_{2 x} b_1yc_1y + b_{2 y}c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b1y c_1x c_1y ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥+ b2 y c_{2 x} c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

⎥: the 2nd row vector

r : the row rank of A

= n A m r B C_n r m

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

the row space of A is the span of the rows of C

the column space of A is the span of the columns of B

a₁₁ a₁₂ a₂₁ a₂₂ ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1x b_{2 x} b_1y b_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ c_1x c_1y c_{2 x} c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1xc_1x + b_{2 x}c_{2 x} b_1xc_1y + b_{2 x}c_{2 y} b_1yc_1x + b_{2 y}c_{2 x} b_1yc_1y + b_{2 y}c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ a₁₁ a₂₁ ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1xc_1x + b_{2 x}c_{2 x} b_1yc_1x + b_{2 y}c_{2 x} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = c1x b_1x b_1y ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥+ c2 x b_{2 x} b_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

⎥: the 1st column vector a₁₂ a₂₂ ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = b_1xc_1y + b_{2 x}c_{2 y} b_1yc_1y + b_{2 y}c_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥ = c1y b_1x b_1y ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥ ⎥+ c2 y b_{2 x} b_{2 y} ⎡ ⎣ ⎢ ⎢ ⎤ ⎦ ⎥

⎥: the 2nd column vector

r : the row rank of A

= n A m r B C_n r m

Row Rank = Column Rank

A

_m

_×n

= B

_m

_×r

C

_r

_×n

=

the row space of A is the span of the rows of C

the column space of A is the span of the columns of B

r : the row rank of A

m n A m r B C_n r

the row rank of A = the number of independent row vectors of C = r

the column rank of A = the number of independent column vectors of B = r

Row / Column Space

a

₁₁

a

₁₂

! a

_1n

a

₂₁

a

₂₂

! a

_2n

!

!

"

!

a

_m1

a

_{m 2}

! a

_mn

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

A

_m

_×n

a

₁₁

a

₁₂

! a

_1n

a

₂₁

a

₂₂

! a

_2n

!

!

"

!

a

_m1

a

_{m 2}

! a

_mn

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

Row Space

: the subspace of !n spanned by the row vectors of A

row vectors

∈!

n

column vectors

∈!

m

Column Space

a

₁₁

a

₁₂

! a

_1n

a

₂₁

a

₂₂

! a

_2n

!

!

"

!

a

_m1

a

_{m 2}

! a

_mn

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

x

₁

x

₂

!

x

_n

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

=

b

₁

b

₂

!

b

_m

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

Ax

= b

m× n n×1 m×1

x

₁

a

₁₁

a

₂₁

!

a

_m1

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

+ x

₂

a

₁₂

a

₂₂

!

a

_{m 2}

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

+!+ x

_n

a

_1n

a

_2n

!

a

_mn

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

=

b

₁

b

₂

!

b

_m

⎡

⎣

⎢

⎢

⎢

⎢

⎢

⎤

⎦

⎥

⎥

⎥

⎥

⎥

Row / Column Space

The solution (x

₁

, x

₂

,

!, x

_n

) exists

iff b is in the column space of A.

Ax

= 0

m× n n×1 m×1

Null Space

Null Space

: the set of vectors x such that Ax=0.