Camera Models

(1)

Camera Models 1

2008-1

Multi View Geometry (Spring '08)

Prof. Kyoung Mu Lee

SoEECS, Seoul National University

Camera Models

Camera Models 2

Camera Models – Finite Cameras

• Pinhole camera model:

In homogeneous coordinates

Z fY Z y Y f

y= ⇒ =

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎟=

⎟⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

0 1 1

0 0

Z Y X f

f

Z fY fX

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎟=

⎟⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

0 1 1

0 1

1 Z

Y X f

f

Z fY fX

PX

x= P=diag(f,f,1)

[ ]

I|0

(2)

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

• Principal point offset model:

9Set the coordinates of the principal point (image center) be

9Then the mapping becomes

9Or in homogeneous representation

T y x p p , ) (

calibration matrix

T y x

T fX Z p fY Z p

Z Y

X, , ) ( / , / )

( a + +

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎟=

⎟⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛ + +

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

0 1 1

0 0

1

Z Y X p f

p f

Z Zp fY

Zp fX Z

Y X

y x x

x

a

[ ]

I|0Xcam

K x=

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

1

y x

p f

p f K

Camera Models 4

Finite cameras

• Camera Rotation and Translation:

center camera the

of s coordinate ous

inhomogene

~ : :rotation matrix ofcameracoordiantes s coordinate camera

ous inhomogene

~:inhomogene: ous worldcoordinates

~ ~ ~)

~

cam cam

C R X X

C X R(

X = −

world coordinates camera coordinates

1 X 0

RC R

1 1 0

C~ R

X_cam R ⎥

⎦

⎢ ⎤

⎣

⎡ −

=

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

⎥⎦

⎢ ⎤

⎣

⎡ −

= Z

Y X

[ ]

(3)

Camera Models 5

[ ]

^I^| ^C^~

KR

P= −

Finite cameras

• CCD Cameras:

9Non-square pixels

910 DOF

• Finite projective camera:

9Including skew parameter s(zero for most normal cameras)

911 DOF (5+3+3)

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

1

y y y

x x x

p m fm

M: 3x3 nonsingular

[

4

]

4]

[M|p MI|M p

P= = ⁻¹

{finite cameras}={P_4x3=[M|p₄]|det M≠0}

If m_xand m_ypixels per unit distance in xand ydirections

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

1

0 0

y x

x x

α α K

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

1 1

y x x

x

p f

p f m m

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

1

y x

x x

p p α α K

Camera Models 6

Camera Models - The projective camera

• General projective cameras: P=

[

M|p₄

]

C~

(4)

• Camera center:

9 is the null-space of the camera projection matrix

9For all A, all points on the line ACproject on image of A, therefore Cis the camera center

9 Image of camera center is (0,0,0)^T, i.e. undefined 9Finite Camera case:

9Infinite Camera case:

0 PC=

C A

X=λ +(1−λ)

PA PC PA

PX

x= =λ +

(1

−λ) =λ

⎟⎟⎠

⎞

⎜⎜⎝

=⎛− ⁻ 1

4 1p C M

0 0⎟⎟⎠, =

⎜⎜ ⎞

⎝

=⎛d Md C

C~

Camera Models 8

The projective camera

• Column vectors of P:

[ ]

[

0 0 0 1

]

worldorigin

, While,

. and , to Similarily

on point axis, - X of direction 0

0 0 1 ,

4

3 1

⇒

=

⇒

=

= _∞

T o

o

T X

X

D PD p

p p D PD p

2

π

(5)

Camera Models 9

• Row vectors of P:

9The principal plane:

qthe plane passing Cparallel to the image plane, set of X, s.t.,

9Axis plane:

qPoints on P¹,

qSimilarly, P²is a plane defined by Cand the line y=0 plane principal the

ng representi vector

the is

0 )

0 , , (

3

P

X P PX

⇒

=

⇒

= x y ^T ^T

0 line the and by defined plane

a is Thus,

on lies 0

axis - y on points )

, , 0 ( 0

1

1 1

1

=

⇒

=

⇒

=

⇒

=

x y

T

T T

C P

P C

C P

PX X

P ω

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

=

T T T

3 2 1

p p p P

Camera Models 10

The projective camera

• C is the intersection of P

¹

, P

²

and P

^3: ^C ⁰

P P P

=

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

T T T

2 2 1

Principal axis Camera

center C

Principal point

(6)

• The principal point:

P³

Normal direction of P³

= (p₃₁,p₃₂,p₃₃)^T

x₀

[

₄

]

³ ³

3

0 PPˆ M|P Pˆ Mm

x = = =

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

= 0 ˆ

33 32 31 3

p p p P

π

∞

Point on π_∞

• Principal axis vector:

9There is an ambiguity between m³and –m³

[ ] [ ]

camera the

of front the toward direction the

is

on t independen is

direction ) det(

) ˆ det(

ˆ) det(

ˆ

|ˆ

~ ˆ

| for

then , ) det(

Let

4 3 4

3 4

3

4 3

v

v m M

m KR m

M v

p M C R R K P

m M v

⇒

=

−

=

k k

k

m

³

(7)

Action of projective camera on point

• Forward projection:

9Projection of vanishing point:

• Back-projection:

9Finite camera case:

D=(d^T,0)^T

)1

( ⁻

+=P^T PP^T P

A ray passing through C and P⁺x

4

~ 1

p M C=− ⁻

Back projected point on π_∞ C

x

P X

= ⁺

I PP⁺=

• Depth of points:

9Consider the projection of a space point Xonto an image point x

. of direction (positive)

in the from of depth the

represent ,

1 and

0 det if , Thus

~) (~ ) ( Then,

1

~ 1

3 3

3

3 2 1

m C

X

m M

C X m C X P X P

X P P P PX x

ω ω

ω

=

>

−

=

−

=

⎥⎦

⎢ ⎤

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

T T

T

T T T

y x

Since PC=0

dot product of the ray from C to X, with the principal ray direction

(8)

• In general, for

(

;

)

^sign(det₃ ⁾

depth

m P M

X T

= ω

( )

^T

X= X,Y,Z,T

The projective cameras - Decomposition of P

• Finding the camera center:

• Finding the camera orientation and internal parameters:

9Decompose Mas M= KRusing RQfactorization method 9K: Upper triangular matrix

9R: rotation matrix (orthogonal)

9Keep Kto have positive diagonals to resolve ambiguity SVD using of space null ) , , , (

, C P

0

PC= = X Y ZT ^T →

p. 67

(9)

Decomposition of P

• When is skew non-zero?

9 for CCD/CMOS, always s=0

9Image from image, s≠0 possible (non coinciding principal axis)

⎥ ⎥

⎥

⎦

⎤

⎢ ⎢

⎢

⎣

⎡

=

1

y x

x x

p p s α α K

1 γ

arctan(1/s)

Decomposition of P

•

Euclidean vs. projective

9 general projective interpretation

9 Meaningfull decomposition in K,R,trequires Euclidean image and space

9Camera center is still valid in projective space 9 Principal plane requires affine image and space

9 Principal ray requires affine image and Euclidean space

[ ] [ 4 4 homography ]

0 1 0 0

0 0 1 0

0 0 0 1 homography 3

3

×

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

×

= P

(10)

• Camera at infinity

9Cameras center lying on the plane at infinity 9 affineand non-affinecameras

• Affine cameras:

9the last row of P, P^3Tis the form of (0,0,0,1)

9Thus,points at infinityare mapped to points at infinity

9Consider moving back a finite projection camera while zooming to keep the size of object be the same

[ ]

0 singular is

4

=

= PC M

p

| M P

ray principal the

of direction in the

center camera

the from origin world the of distance the

~ : of principal ray direction

the :

3 0 3

C r r d =− T

Affine cameras

• Moving back at unit speed for a time t, so that

• Next, by zooming with a zooming factor so that the image size remains fixed. Then,

~ 3

~ C r

C→ −t

t t

d_t ^T

at time ray principal the

of direction in the

center camera

the from origin world the of distance the

~ :

3 +

−

= r C

/d0

d k= _t

(11)

Affine cameras

• As d

_t

tends to infinity,

Affine camera

(12)

• For point on plane parallel with principal plane and through the world origin,

• For general points

⎟⎟⎠

⎜⎜ ⎞

⎝

⎛ +

= 1

β αr¹ r²

X P₀X=P_tX=P_∞X

⎟⎟⎠

⎜⎜ ⎞

⎝

⎛ + +Δ

= 1

β

αr¹ r² r³ X

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛ +

=

Δ

~

0 0

proj

d y x K X P x

x

proj affine

x x

0

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

=

= _∞

0

affine ~

~ d

y x K X P x

(

proj 0

)

0 0 0

affine-~ ~ -~

~ x x x

x d

d +Δ

=

Affine cameras-Error

• Thus, the error in employing an affine camera:

• Approximation error is small if

9Δ is small compared to the average depth d₀

9The distance of the point from the principal ray is small (i.e. small field of view)

(

proj 0

)

0 proj

affine

- ~ ~ - ~

~ x x x x

d

= Δ

(13)

Affine cameras-Decomposition

• Decomposition of

or

Thus,

P∞

0 x₀= for ~

⎥⎦

⎢ ⎤

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎥⎦

⎢ ⎤

⎣

=⎡ ^×

1 1 0 0 0

0 0 1 0

0 0 0 1 ˆ 1

2 ˆ

2

T 0T

t R 0

0 K

Affine cameras

• Summary of parallel projection

⎥ ⎥

⎥

⎦

⎤

⎢ ⎢

⎢

⎣

⎡

∞

=

1 0 0 0

0 0 1 0

0 0 0 1

P canonical representation

⎥ ⎦

⎢ ⎤

⎣

= ⎡

^×

1 0

0 K

₂ ₂

K calibration matrix

principal point is not defined

(14)

• Orthographic projection:

• Incorporating rotation and translation

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

1 1 1 0 0 0

0 0 1 0

0 0 0 1

1

Y X Z Y X y

x

[

^M ^t

]

0 r r 0

t

P R =

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎥=

⎦

⎢ ⎤

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

1 1 1 0 0 0

0 0 1 0

0 0 0 1

2 2

1 1

t t

T T

T

unit norm

5 DOF

Hierarchy of affine cameras

• Scaled orthographic projection:

• Weak perspective projection:

6 DOF

7 DOF

(15)

Hierarchy of affine cameras

• Affine camera:

• In inhomogeneous coordinates

⎥⎦

⎢ ⎤

⎣

⎡

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

⎥⎦

⎢ ⎤

⎣

=⎡ ^×

1 1 0 0 0

0 0 1 0

0 0 0 1 ˆ 1

2 ˆ

2

T 0T

t R 0

0 K

3x3 affine 4x4 affine

3x4 affine

linear mapping

8 DOF

Hierarchy of affine cameras

• Properties of the affine camera:

9Any projective camera with the principal plane is the plane at infinityis an affine camera

9The camera center is also lies on the plane at infinity 9Parallel lines in space are mapped to parallel image lines 9The vector dsatisfying M_2x3d=0 is the direction of parallel

projection, and (d^T,0)^Tis the camera center.

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

0 0

Y X Z Y X

PA

plane at infinity

= principal plane

point at infinity

d 0 P ⎥=

⎦

⎢ ⎤

⎣

⎡

A

0

(16)

• Non-affine camera at infinity

[ ]

0

zero not is row last but the singular is

4

=

= PC M

p

| M P

Other camera models

• Pushbroom cameras:

• Linear Pushbroom (LP) camera – satellite sensor, SPOT

(11dof)

(17)

Other camera models

• straight lines are not mapped to straight lines

⎥⎦

⎢ ⎤

⎣

=⎡

⎥⎦

⎢ ⎤

⎣

→⎡

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

=

⎥⎥

⎥

⎦

⎤

⎢⎢

⎢

⎣

⎡

ω ^~ ^/ω

~

1

y x y x Z Y X y

x P

hyperbola

~

+ + + = ⇒

⇒

α

xy

β

x

γ

y

δ

o

Other camera models

• Line cameras:

92D to 1D projection

95 DOF 0 PC=

camera center