More Single View Geometry

(1)

More Single View Geometry 1

2008-1

Multi View Geometry (Spring '08)

Prof. Kyoung Mu Lee

SoEECS, Seoul National University

More Single View Geometry

Actions of a Projective Camera on Planes

• Planar homography 9 For a point on

• The most general transformation that can occur between a scene plane and an image plane under perspective imaging is a plane projective transformation

(affine camera-affine transformation)

Hxπ

=

planar homography

π

(2)

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Actions of a Projective Camera on Lines

• Forward projection: Line to line

• Back-projection: line to plane

l P l→ ^T

l l

P l P X

l x PX x

line the of plane projected -

back the is

0 ) (

0 and

T T T

T

⇒

=

⇒

= l =

P^T π=

Actions of a Projective Camera on Conics

• Back projection: conic to cone

• Ex:

CP P Q C→ _CO= ^T

0 PC C P C Q

Q C

Q

=

= ( )

, of vector null the is and 3 rank has

T CO

CO CO

C CP

P

X CP P X

Cx x PX x

conic the of cone projected -

back the is

0 ) (

0

and

T T T

T

⇒

=

⇒

=

[ ] [ ] _⎥

⎦

⎢ ⎤

⎣

=⎡

⎥⎦

⎢ ⎤

⎣

=⎡

= | , | 0

T T

co 0

0 CK 0 K

K 0 C Q K 0 I K P

(3)

Images of Smooth Surfaces

• The contour generator Γ is the set of points X on S at which rays are tangent to the surface. The corresponding apparent contour γ is the set of points x which are the image of X, i.e. γ is the image of Γ

• The contour generator Γ depends only on position of projection center, γ depends also on rest of P

Actions of a Projective Camera on Quadrics

• Forward Projection: Quadric to conic

• The cone with vertex V and tangent to the quadric Q (degenerate quadric) is

QC π_Γ=

−

ip relationsh polar

pole

0 V Q_CO =

T T

CO (V QV)Q-(QV)(QV) Q =

(4)

Fixed Camera Center

• Projective imaging devices

~]

| [' ' '

~],

|

[I C P KR I C

KR

P= − = −

homography planar

,

: = ′ ′ ⁻¹

′=

⇒x Hx H (KR)(KR)

( )KR P R

K

P'= ' ' ^-1

⇒

Moving the image plane (zooming)

• Increase in focal length:

9 move image plane along the principal axis 9 magnification

f k f′

=

(5)

Camera rotation

x KRK 0]X

| K[I KRK 0]X

| K[R x'

X 0 I K x

-1

-1 =

=

= [ | ]

rotation conjugate

1 : KRK H= ⁻

{

eⁱ^θ e⁻ⁱ^θ

}

=

−values μ,μ ,μ three

angle rotation the

is

point), (vanishing axis

rotation the

of direction the

represent

1

θ e

• Compute the homography that warps the image quadrilateral to a rectangle with the correct aspect ratio

• warp the image

Applications – Synthetic Views

(6)

Applications – Planar panoramic mosaicing

• close-up: interlacing problem

(7)

Projective (reduced) notation

• If canonical coordinates are chosen

• Then the camera matrix becomes

and the camera center is

z This implies that all images with the same camera center are projectively equivalent

T 4

T 3

T 2

T

1 (1,0,0,0) ,X (0,1,0,0) ,X (0,0,1,0) ,X (0,0,0,1)

X = = = =

T 4

T 3

T 2

T

1 (1,0,0) ,x (0,1,0) ,x (0,0,1) ,x (1,1,1)

x = = = =

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

−−

= −

d

c d

b d

a 0

0 0

0 0 0 P

d T

c b

a , , , ) (

C= ⁻¹ ⁻¹ ⁻¹ ⁻¹

(8)

Moving the Camera Center

• When the camera center moves –motion parallaxoccurs

epipolar line

What does calibration give?

•

• The angle between two rays

• A calibrated camera (K is known) is a direction sensor which can measure the direction of rays

• An image line l defines a plane through the camera center with normal n=K^Tl measured in the camera’s Euclidean frame

( )( )

( )(

2

)

1 - T T - 2 1 1 - T T - 1

2 1 - T T - 1

2 T 2 1 T 1

2 T 1

) ( ) (

) ( cos

x K K x x K K x

x K K x

d d d d

d d

= θ=

x K d

Kd d

0

| I K x

x d

X

1

T T,1) ](

[

then , of points projected back

the be

~ Let

= −

⇒

=

= λ λ

in the camera Euclidean coordinates

) 0

) ( 0 ) ( 0

(d^Tn= ⇒ K⁻¹x^Tn= ⇒x^T K⁻^Tn = ⇒l=K⁻^Tn

(9)

The image of the Absolute Conic

• mapping between π_∞to an image is given by the planar homogaphy x=Hd, with H=KR

• image of the absolute conic (IAC) is

Independent on C

1 1

1

) ( ) ( ) (

on conic,

absolute an

For

−

∞

−

=

′=

T T

T

KK KR

I KR ω

π I Ω C CH

H C

(

^T

)

¹ ^-^T ^-¹

ω= KK ⁻ =K K

The image of the Absolute Conic

• Remarks:

9 depends only on the internal parameters K 9 The angle between two rays

9 Dual (line) conic (DIAC):

9 Once is identified K can be determined uniquely by Cholesky factorization

9 A plane intersects in a line, and this line intersects in two points (circular points of ). The imaged circular points lie on at the points at which the vanishing line of the plane intersects .

unchanged under any projective transformation of the image

Image of Q^*_∞

ω

or *

ω ω

π π_∞

π ^∞

Ω

π ω

ω

(10)

A simple calibration device

• Calibration steps:

Orthogonality in the image

• Using

2 0

1ωx = x^T 2 0

1ωx =

x^T l=ωx

( )( )

^⇒

=

2 T 2 1 T 1

2 T

cos 1

ωx x ωx x

ωx θ x

1

and

= ⁻ _∞ ⁻

=Hd ω H Ω H

x_i _i ^T

(11)

Vanishing Points

• Let the point on a line through A with direction be

• Then

• And

T T,0) (d D= D

A

X(λ)= +λ P=K

[ ]

I|0

Vanishing Points

• Camera rotation from vanishing points

• Assume two cameras with same K and rotation by R

• Vanishing points are depend on camera rotation but not position

• Thus, two corresponding directions are sufficient for determining R

i i

i i i

i i

i i i

i i

d R d

v K

v d K

d K v

v K

v d K

Kd v

ˆ ˆ

1 1 1 1

′=

′

= ′

⇒ ′

= ′

′

=

⇒

=

−

(12)

Vanishing Lines

• All planes with normal n have vanishing line

• Or

n K l= ⁻^T l

K n= ^T

Vanishing Lines

(13)

Orthogonality relations

• The angle between two scene lines using vanishing points:

• The angle between two scene planes using vanishing lines:

( )(

2

)

T 2 1 T 1

2 T

cos 1

ωv v ωv v

ωv

= v θ

( )(

2

)

T 2 1 T 1

2 T

cos 1

l ω l l ω l

l ω l

∗

= ∗

θ

Orthogonality relations

• Orthogonality relations:

9 The vanishing points of two perpendicular lines

9 The vanishing point of the normal direction to a plane and the plane vanishing line

9 The vanishing lines of two perpendicular planes

2 0

1ωv = v^T

0 ωv l ωv l l ω

v= ^∗ ⇒ = ⇒ ×( )=

2 0

1ω^∗l = l^T

(14)

Affine 3D measurements and Reconstruction

(15)

• Measuring a person’s height in a single image

Determining K from a single view

• Using five pairs of of vanishing points of perpendicular lines

• Similarly to pairs of vanishing point & lines, and vanishing lines & lines

•

• Thus, for a camera with zero skew and square pixels, three orthogonal vanishing points are sufficient for determining K

=0

j T iωv

v ω=K⁻^TK⁻¹ ^K

[ ]

21 12 22

11

21 12 1

12

then , addition

in If

. 0 then

and 0 If

ω ω α

α

ω ω ω

=

= ⁻ ⁻

y x

T ij

K K K

s ω K K

(16)

• Three sources of constraints on

9 Metric plane imaged with known homography

9 Vanishing points and lines corresponding to perpendicular directions

9 “internal constraints” such as zero skew or square pixels ω

(17)

Determining K using vanishing points and lines

v1 v₁

v1

v2

(18)

•

• Only one constraints is needed to determine f (and thus K)

• We can synthesize a fronto-parallel view using K and vanishing line l

⎟⎟⎠

⎜⎜ ⎞

⎝

= ⎛

=

= 1 1 1

then , 0 and

, ,

0 Assume

2 2

0 0

f diag f

y x

s _x _y

ω

α α

(19)

Single View Reconstruction

• Piecewise planar graphical model from a single image

9 Assume three orthogonal planes

9 Determine K using three vanishing points and square pixels 9 Proportioned width and height are determined by rectification 9 Find homography for texture mapping for each plane using K and l

Calibrating Conic

• Image of a cone with apex angle 45^oand axis coinciding with the principal axis

• Let and the point on the cone be , then the calibrating conic is an affine transformation of a unit circle

[ ]I 0 K

P= | X²+Y² =Z²

Unit circle

(20)

Calibrating Conic

• Orthogonality:

Thus, x and x’ are orthogonal if

x C x CSx x

x CS x ωx x

&

T T

= ′

= ′ ⁻¹

0

] 1 , 1 , 1 [

1, = −

=K⁻^TDK⁻ D diag C

C x Sx

x

of center the w.r.t.

of point reflection

= ;

&

• Finding calibrating conic using orthogonal vanishing points