More Single View Geometry

전체 글

(1)

More Single View Geometry 1

2008-1

Multi View Geometry (Spring '08)

Prof. Kyoung Mu Lee

SoEECS, Seoul National University

More Single View Geometry

More Single View Geometry 2

Actions of a Projective Camera on Planes

Planar homography 9 For a point on

The most general transformation that can occur between a scene plane and an image plane under perspective imaging is a plane projective transformation

(affine camera-affine transformation)

Hxπ

=

planar homography

π

(2)

More Single View Geometry 3

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Actions of a Projective Camera on Lines

Forward projection: Line to line

Back-projection: line to plane

l P l T

l l

P l P X

l x PX x

line the of plane projected -

back the is

0 ) (

0 and

T T T

T

=

= l =

PT π=

More Single View Geometry 4

Actions of a Projective Camera on Conics

• Back projection: conic to cone

• Ex:

CP P Q C CO= T

0 PC C P C Q

Q C

Q

=

= ( )

, of vector null the is and 3 rank has

T CO

CO CO

C CP

P

X CP P X

Cx x PX x

conic the of cone projected -

back the is

0 ) (

0

and

T T T

T

=

=

=

[ ] [ ]

=

=

= | , | 0

T T

co 0

0 CK 0 K

K 0 C Q K 0 I K P

(3)

More Single View Geometry 5

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Images of Smooth Surfaces

• The contour generator Γ is the set of points X on S at which rays are tangent to the surface. The corresponding apparent contour γ is the set of points x which are the image of X, i.e. γ is the image of Γ

• The contour generator Γ depends only on position of projection center, γ depends also on rest of P

More Single View Geometry 6

Actions of a Projective Camera on Quadrics

• Forward Projection: Quadric to conic

The cone with vertex V and tangent to the quadric Q (degenerate quadric) is

QC πΓ=

ip relationsh polar

pole

0 V QCO =

T T

CO (V QV)Q-(QV)(QV) Q =

(4)

More Single View Geometry 7

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Fixed Camera Center

• Projective imaging devices

~]

| [' ' '

~],

|

[I C P KR I C

KR

P= =

homography planar

,

: = 1

=

x Hx H (KR)(KR)

( )KR P R

K

P'= ' ' -1

More Single View Geometry 8

Moving the image plane (zooming)

• Increase in focal length:

9 move image plane along the principal axis 9 magnification

f k f

=

(5)

More Single View Geometry 9

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Camera rotation

x KRK 0]X

| K[I KRK 0]X

| K[R x'

X 0 I K x

-1

-1 =

=

=

= [ | ]

rotation conjugate

1 : KRK H=

eiθ eiθ

}

=

values μ,μ ,μ three

angle rotation the

is

point), (vanishing axis

rotation the

of direction the

represent

1

θ e

More Single View Geometry 10

• Compute the homography that warps the image quadrilateral to a rectangle with the correct aspect ratio

• warp the image

Applications – Synthetic Views

(6)

More Single View Geometry 11

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Applications – Planar panoramic mosaicing

More Single View Geometry 12

close-up: interlacing problem

Applications – Planar panoramic mosaicing

(7)

More Single View Geometry 13

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Applications – Planar panoramic mosaicing

More Single View Geometry 14

Projective (reduced) notation

• If canonical coordinates are chosen

• Then the camera matrix becomes

and the camera center is

z This implies that all images with the same camera center are projectively equivalent

T 4

T 3

T 2

T

1 (1,0,0,0) ,X (0,1,0,0) ,X (0,0,1,0) ,X (0,0,0,1)

X = = = =

T 4

T 3

T 2

T

1 (1,0,0) ,x (0,1,0) ,x (0,0,1) ,x (1,1,1)

x = = = =

⎥⎥

⎢⎢

−−

= −

d

c d

b d

a 0

0 0

0 0 0 P

d T

c b

a , , , ) (

C= 1 1 1 1

(8)

More Single View Geometry 15

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Moving the Camera Center

• When the camera center moves –motion parallaxoccurs

epipolar line

More Single View Geometry 16

What does calibration give?

• The angle between two rays

A calibrated camera (K is known) is a direction sensor which can measure the direction of rays

An image line l defines a plane through the camera center with normal n=KTl measured in the camera’s Euclidean frame

2

)

1 - T T - 2 1 1 - T T - 1

2 1 - T T - 1

2 T 2 1 T 1

2 T 1

) ( ) (

) ( cos

x K K x x K K x

x K K x

d d d d

d d

= θ=

x K d

Kd d

0

| I K x

x d

X

1

T T,1) ](

[

then , of points projected back

the be

~ Let

=

=

=

= λ λ

in the camera Euclidean coordinates

) 0

) ( 0 ) ( 0

(dTn= K1xTn= xT KTn = l=KTn

(9)

More Single View Geometry 17

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

The image of the Absolute Conic

• mapping between πto an image is given by the planar homogaphy x=Hd, with H=KR

• image of the absolute conic (IAC) is

Independent on C

1 1

1

) ( ) ( ) (

on conic,

absolute an

For

=

=

=

=

=

T T

T

KK KR

I KR ω

π I C CH

H C

T

)

1 -T -1

ω= KK =K K

More Single View Geometry 18

The image of the Absolute Conic

• Remarks:

9 depends only on the internal parameters K 9 The angle between two rays

9 Dual (line) conic (DIAC):

9 Once is identified K can be determined uniquely by Cholesky factorization

9 A plane intersects in a line, and this line intersects in two points (circular points of ). The imaged circular points lie on at the points at which the vanishing line of the plane intersects .

unchanged under any projective transformation of the image

Image of Q*

ω

or *

ω ω

π π

π

π ω

ω

(10)

More Single View Geometry 19

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

A simple calibration device

• Calibration steps:

More Single View Geometry 20

Orthogonality in the image

Using

2 0

1ωx = xT 2 0

1ωx =

xT l=ωx

( )( )

=

2 T 2 1 T 1

2 T

cos 1

ωx x ωx x

ωx θ x

1

and

=

=Hd ω H H

xi i T

(11)

More Single View Geometry 21

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Vanishing Points

Let the point on a line through A with direction be

• Then

• And

T T,0) (d D= D

A

X(λ)= +λ P=K

[ ]

I|0

More Single View Geometry 22

Vanishing Points

• Camera rotation from vanishing points

Assume two cameras with same K and rotation by R

• Vanishing points are depend on camera rotation but not position

• Thus, two corresponding directions are sufficient for determining R

i i

i i i

i i

i i i

i i

d R d

v K

v d K

d K v

v K

v d K

Kd v

ˆ ˆ

ˆ ˆ

1 1 1 1

=

=

=

=

=

(12)

More Single View Geometry 23

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Vanishing Lines

All planes with normal n have vanishing line

• Or

n K l= T l

K n= T

More Single View Geometry 24

Vanishing Lines

(13)

More Single View Geometry 25

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Orthogonality relations

• The angle between two scene lines using vanishing points:

• The angle between two scene planes using vanishing lines:

2

T 2 1 T 1

2 T

cos 1

ωv v ωv v

ωv

= v θ

2

)

T 2 1 T 1

2 T

cos 1

l ω l l ω l

l ω l

=

θ

More Single View Geometry 26

Orthogonality relations

• Orthogonality relations:

9 The vanishing points of two perpendicular lines

9 The vanishing point of the normal direction to a plane and the plane vanishing line

9 The vanishing lines of two perpendicular planes

2 0

1ωv = vT

0 ωv l ωv l l ω

v= = ×( )=

2 0

1ωl = lT

(14)

More Single View Geometry 27

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Affine 3D measurements and Reconstruction

More Single View Geometry 28

Affine 3D measurements and Reconstruction

(15)

More Single View Geometry 29

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Affine 3D measurements and Reconstruction

• Measuring a person’s height in a single image

More Single View Geometry 30

Determining K from a single view

• Using five pairs of of vanishing points of perpendicular lines

• Similarly to pairs of vanishing point & lines, and vanishing lines & lines

• Thus, for a camera with zero skew and square pixels, three orthogonal vanishing points are sufficient for determining K

=0

j T iωv

v ω=KTK1 K

[ ]

21 12 22

11

21 12 1

12

in If

. 0 then

and 0 If

ω ω α

α

ω ω ω

=

=

=

=

=

=

=

=

=

=

y x

T ij

K K K

s ω K K

(16)

More Single View Geometry 31

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Determining K from a single view

• Three sources of constraints on

9 Metric plane imaged with known homography

9 Vanishing points and lines corresponding to perpendicular directions

9 “internal constraints” such as zero skew or square pixels ω

More Single View Geometry 32

Determining K from a single view

(17)

More Single View Geometry 33

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Determining K using vanishing points and lines

More Single View Geometry 34

Determining K using vanishing points and lines

v1 v1

v1

v1

v2

(18)

More Single View Geometry 35

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Determining K using vanishing points and lines

More Single View Geometry 36

Determining K using vanishing points and lines

Only one constraints is needed to determine f (and thus K)

We can synthesize a fronto-parallel view using K and vanishing line l

⎟⎟

⎜⎜

=

=

=

=

= 1 1 1

then , 0 and

, ,

0 Assume

2 2

0 0

f diag f

y x

s x y

ω

α α

(19)

More Single View Geometry 37

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Single View Reconstruction

• Piecewise planar graphical model from a single image

9 Assume three orthogonal planes

9 Determine K using three vanishing points and square pixels 9 Proportioned width and height are determined by rectification 9 Find homography for texture mapping for each plane using K and l

More Single View Geometry 38

Calibrating Conic

• Image of a cone with apex angle 45oand axis coinciding with the principal axis

• Let and the point on the cone be , then the calibrating conic is an affine transformation of a unit circle

[ ]I 0 K

P= | X2+Y2 =Z2

Unit circle

(20)

More Single View Geometry 39

Multi View Geometry (Spring '08) K. M. Lee, EECS, SNU

Calibrating Conic

• Orthogonality:

Thus, x and x’ are orthogonal if

x C x CSx x

x CS x ωx x

&

T T

T T

= ′

= ′

= ′

= ′ −1

0

] 1 , 1 , 1 [

1, =

=KTDK D diag C

C x Sx

x

of center the w.r.t.

of point reflection

= ;

&

More Single View Geometry 40

Determining K using vanishing points and lines

• Finding calibrating conic using orthogonal vanishing points

Updating...

관련 주제 :