a) (Samsung Electronics) b) (Kwangwoon University)
Corresponding Author : (Jin-Sung Lee)
E-mail: [email protected] Tel: +82-10-3418-2285
ORCID: http://orcid.org/0000-0002-9466-2958
・Manuscript received November 4, 2015; Revised December21, 2015;
Accepted January 4, 2016.
3
a), a) , b)
Crosstalk Reduction of Glasses-free 3D Displays using Multiview Image Processing
Sung-Yeol Kima), Jin-Sung Leea) , and Sang Mi Choib)
3 (Crosstalk) .
3 . 3
,
. , .
3 .
Abstract
In this paper, we present a new method to reduce crosstalk of a glasses-free three-dimensional (3D) display using a multi-view image processing technique. Since crosstalk makes the current view image mixed with its neighboring ones, the output 3D image becomes severely blurred. We apply adaptive depth retargeting and view gradient-based crosstalk inverse filtering onto a multi-view image to minimize crosstalk of the glasses-free 3D display. In addition, overflow and underflow pixels are compensated by epipolar image pixel interpolation so that visual artifacts are minimized. Experimental results show that the proposed method reduces crosstalk more efficiently than the previous work while improving 3D image quality.
Keyword : depth retargeting, crosstalk inverse filtering, epipolar image, glasses-free 3D display
I.
3 ,
3
. , 3 [1]
(Regular Paper)
21 1 , 2016 1 (JBE Vol. 21, No. 1, January 2016) http://dx.doi.org/10.5909/JBE.2016.21.1.66
ISSN 2287-9137 (Online) ISSN 1226-7953 (Print)
7
6 1
v1 v2 v3 v4 v5 v6 v7
7 1 2 1 2 3 crosstalk
(a) (b) (c) (d)
1. . (a) , (b) 2 , (c) 3 , (d) CIF
Fig. 1. Image blur and artifacts in an autostereoscopic display. (a) slanted structure, (b) 2D image, (c) 3 image, and (d) visual artifacts of CIF
3
. 3
. ,
3 .
.
[2] .
.
3 ,
2008 LG HD 9
, Philips
. , LG , SONY, Dolby,
Dimenco 4K UHD
, 2 /3
3 TV . 2015
8K 3D TV
. 3
. (Morie)
3
. , 1(a) ,
(crosstalk) .
3 (blur)
. 1(b) 1(c) 2
3 . ,
.
, 3
.
[3,
4, 14, 15] [5, 6, 7]
.
. .
. ,
.
(crosstalk inverse filtering, CIF)[5] . CIF . ,
. 3
3 . ,
(overflow)
(underflow) 3
. 1(d) , CIF ,
3 .
(depth retargeting, DR)[8] . DR
(Cornsweet effect)[9] .
, ,
.
, . DR
.
DR CIF 3
.
DR ,
. ,
CIF
. CIF
.
. II
. III
. IV
, V .
II.
(Crosstalk In- verse Filter, CIF) (Depth Retargeting,
DR) . CIF 2010
Chang [5] . CIF
3
. 3
(1) X
.
∑∑
1 1 K i
K
j j
j x y X jiV x y
U
= =
) , ( ) , (
= ) , (
, x, y . Vj j
, Uj j
. K 3
, X 1 K×K
.
CIF X X' ,
(2) V' .
∑∑
1 1 K i
K
j j
j x y X jiV x y
V
= =
) , ( ) ,
′(
= ) ,
′(
, (1) (2)
Ū (3) .
) , (
= ) , ( ) ,
′( ) , (
= ) , (
= =
y x V y x V i j X i j X y
x
U j
K i
K
j j
j ∑∑
1 1
(3) , CIF
Ū Vj
. ,
. 8
0~255 . ,
X'
.
1(d) ,
3 .
DR
다시점 영상 생성 (Multiview Image Generation) 적응적 깊이 재배치
(Adaptive Depth Retargeting)
EPI 생성 (EPI Generation)
시점 미분값기반 CIF (View Gradient-based CIF) EPI 화소 보간법
(EPI Pixel Interpolation) 무안경 3D 영상 생성
(3D Image Generation)
2D Image Depth Image H
W N
… 시점 N
시점 1
v4 v3 v5
v2 v6
v1 vK
X ′′
5 6 7 1 2 3 4
6 7 1 2 3 4 5
1 2 3 4 5 6 7
3 4 5 6 7 1 2
2 3 4 5 6 7 1
7 1 2 3 4 5 6
4 5 6 7 1 2 3
×
View N
View 1
3D Image 3.
Fig. 3. Overall flow of the proposed crosstalk reduction method
2010 Lang [8] .
DR
. DR
.
, DI
DS . ,
C , DI
, (4)
.
) , ( - ) , (
= ) ,
(x y D- x y D x y CiI Ii1 iI
Ci i
, DiI i
. C
.
C DS DF
. 2 DR .
DS
C
DF = +
2. DR
Fig. 2 Depth retargeting using Cornsweet effect
III.
1.
3
. 2 .
+ =
DS C' DF
+ =
4. 3 DR .
Fig. 4. Adaptive depth retargeting of autostereoscopic 3 displays.
. 1)
DR .
2)
. 3) (Epipolar
Image, EPI)[10] . 4) X'
CIF . 5)
EPI . ,
3 EPI 3
.
2.
3
DR . DI
DS .
40% ,
3
20% . DS DI
80% .
DR 3
(4) C C'
. DI(x, y) 2 DI'(x, y) , DI'(x,
y) 0 C'(x, y) 0 .
(edge) C
.
. 4
DR . 4 ,
DR 2 C' 0
. DF
(5) C' DS .
) , ( + ) ,
′(
×
= ) ,
(x y μ C x y D x y
DF S
, μ C' .
DF
.
3. CIF
DR EPI . EPI
3 2
. EPI
. , 3 , K
W×H ,
EPI W×K , EPI
W×K×H .
, EPI CIF
.
X' CIF
. CIF A={a1, a2, ,aK} ,
EPI P={p1, p2, ,pK} ,
v4
v3 v5
v2 v6
v1 vK
X ′
v4
v3 v5
v2 v6
v1 vK
X ′′ 시점 미분값에 의한가중치 조절
(a) (b)
5. CIF . (a) (b)
Fig. 5. View gradient-based CIF weight control. (a) the previous method and (b) the proposed method
G={g1, g2, ,gK} (6) .
2
)}
- ( + ) - (
•{
= i-1 i i+ i
i
p p abs p p λ abs
g 1 1
, pi gi i
, λ1 . CIF
A'={a'1, a'2, ,a'K} (7) .
i i i
i a
λ a β a g •( - )+
′= i
2
, βi i
, λ2 . βi 0.5×ai
, λ1 λ2 1 32 .
X" (8) .
′= X′
1 3 2
1 1 2 1
a a a
a a a
a a a
K K
K
... ′
′
′
...
...
...
...
... ′
′
′ ... ′
′
′
-
X" CIF (9)
.
∑∑
1 1 K i
K
j j
j x y X jiV x y
V
= =
) , ( ) ,
′(
= ′ ) ,
′(
5 K 7 , CIF
.
. CIF
. , CIF
.
4. EPI
CIF
. ,
.
, EPI . EPI
.
(masking) , EPI
(partial EPI bilateral filtering) .
1
, 0 ,
. EPI
1 0
. EPI
(10) .
∑
∑
∈
∈
Q
n S n C n x
Q
n S n C n x n
new
x F x x F p p
p p p F x x F
p (| - |) (| - |)
|) ′ - (|
|) - (|
=
, FS FC
(a) (b) (c) (d) (e)
6. . (a) 2 , (b) , (c) Chang , (d) Wang , (e)
Fig. 6. Results of test data. (a) original 2D image, (b) Before crosstalk reduction, (c) Chang’s method, and (d) Wang’s method, (e) the proposed method
. S C FS FC
. Q .
[11] FS FC CIF
EPI , CIF
EPI . , px pn EPI , p'n CIF
EPI . , p'n 1
FC ( : 0.001)
. , CIF EPI
EPI 3
. 3
3
.
IV.
7 55 3
. 3 86%
.
15 .
1920x1080 .
7 .
[5,13] [16,17]
.
.
6 . 6(a)
2 , 6(b)
, 6(c) Chang , 6(d)
Wang , 6(e) . 6(b)
6(e) 3
3 DSLR .
6(b) ,
. , 6(c), 6(d),
6(e)
Test Data Glasses-based (%) Before Crosstalk
Reduction (%) Chang's Method (%) Proposed (%)
video 1 99.9 75.7 80.7 88.9
video 2 99.9 72.7 74.8 87.7
video 3 99.9 79.3 80.3 89.3
video 4 99.9 81.3 85.3 92.4
video 5 99.9 60.6 75.6 87.6
video 6 99.9 76.8 80.8 90.7
video 7 99.9 74.3 75.3 87.5
video 8 99.9 76.9 77.9 88.7
video 9 99.9 66.6 70.6 87.9
video 10 99.9 81.1 81.1 96.2
video 11 99.9 82.5 85.5 91.6
video 12 99.9 84.9 90.9 95.0
video 13 99.9 75.7 80.7 88.7
video 14 99.9 81.1 85.1 91.5
video 15 99.9 82.1 85.1 89.3
Average BM Ratio 99.9 76.8 80.5 90.2
1. BM
Table 1. Image sharpness comparison via blur metric ratio
. ,
. ,
DR CIF
3
.
3 ,
(blur metric, BM)[12]
. BM ,
. 2
BM
(11) .
- ) - (
×
= (%)
D D D
BM BM BM BM
2 2
1 3
100
, BM2D 2 BM ,
BM3D 3 BM
. BM 100%
3 2
. 1
BM . 1
, BM 76.8%, Chang
80.5% , BM 90.2%
. BM
13.4%, Chang 9.7% .
3 5% BM
99.9% . 3
3
.
V.
3
.
, 3
. 15 ,
3
. , BM
90.5% 13.4%,
9.7% .
(References)
[1] N. A. Dodgson, “Autostereoscopic 3D displays,” Computer, vol. 8, pp.31-36, 2005.
[2] P. Kauff, N. Atzpadin, C. Fehn, M. Müller, O. Schreer, A. Smolic, and R. Tanger, “Depth map creation and image-based rendering for ad- vanced 3DTV services providing interoperability and scalability,”
Signal Processing Image Communication, vol. 22, no. 2, pp. 217-234, 2007.
[3] H.H. Hui, M.H. Kuo, H.H. Huang, S. C. Chuang, C. H. Chen, and C. H.
Tsai, “High resolution autostereoscopic 3D display with proximity pro- jector array." SID Symposium Digest of Technical Papers, vol. 39, no.
1, pp. 760-763, 2008.
[4] S. S. Kim, B. H. You, H. Choi, B. H. Berkeley, D. G. Kim, and N. D.
Kim, “World's first 240Hz TFT LCD technology for Full HD LCD TV and its application to 3D display." SID Symposium Digest of Technical Papers, vol. 40, no. 1, pp. 424-427, 2009.
[5] Y. C. Chang, C. Y. Ma, and Y. P. Huang, “Crosstalk suppression by image processing in 3D display,” SID Symposium Digest of Technical Papers, vol. 41, pp.124 127, 2010.
[6] J. S. Lipscomb and W. L. Wooten, “Reducing crosstalk between stereo- scopic views,” IS&T/SPIE 1994 International Symposium on Electronic Imaging: Science and Technology, pp. 92-96, 1994.
[7] H. Hong, “Reduction of spatially non-uniform 3D crosstalk for stereo- scopic display using shutter glasses,” Displays, vol. 33, no. 3, pp.136-141, 2012.
[8] M. Lang, A. Hornung, O. Wang, S. Poulakos, A. Smolic, and M. Gross,
“Nonlinear disparity mapping for stereoscopic 3D,” ACM Trans. on Graphics, vol. 29, no. 4, paper no. 75, 2010.
[9] P. Didyk, T. Ritschel, E. Eisemann, K. Myszkowski, and, H.P. Seidel,
“A perceptual model for disparity,” ACM Trans. on Graphics, vol. 30, no. 4, paper no. 96, 2011.
[10] J. X. Chai, X. Tong, S. C. Chan, and H. Y. Shum, “Plenoptic sam- pling,” Proc. of the 27th annual conference on Computer graphics and interactive techniques, pp. 307-318, 2000.
[11] M. Elad, “On the origin of the bilateral filter and ways to improve it,”
IEEE Trans. on Image Processing, vol. 11, no. 10, pp. 1141 1150, 2002.
[12] P. Marziliano, F. Dufaux, S. Winkler and T. Ebrahimi, “Perceptual blur and ringing metrics: application to JPEG2000,” Signal processing Image communication, vol. 19, no. 2, pp. 163-172, 2004.
[13] C. Y. Ma, Y. C. Chang, and Y. P. Huang, “Multi-zone digital crosstalk reduction by image processing in 3D display,” Journal of Display Technology, vol. 10, no. 6, pp. 488-493, 2014.
[14] M. Zhou, H. Wang, W. Li, S. Jiao, T. Hong, S. Wang, X, Sun, X. Wang, J. Y. Kim, and D. Nam, “A unified method for crosstalk reduction in Multiview displays,” Journal of Display Technology, vol. 10, no. 6, pp.
500-507, 2014.
[15] J. Park, D. Nam, G. Sung, Y. Kim, D. Park, and C. Kim, “Active cross- talk reduction on multi-view displays using eye detection,” SID Symposium Digest of Technical Papers, vol. 42, pp.920 923, 2011.
[16] Q. H. Wang, X. F. Li, L. Zhou, A. H. Wang, and D. H. Li, “Crosstalk reduction by correcting the subpixel position in a Multiview autoster- eoscopic three-dimensional display based on a lenticular sheet,”
Applied Optics, vol. 50, no. 7, pp. B1~B5, 2011.
[17] J. Kang and H. Lee, “Autostereoscopic 3D image enhancement via sub-view based crosstalk inverse compensation,” Proc. Of Global 3D Tech Forum, pp. 1~2, 2015.
- 2001 : - 2003 : - 2008 :
- 2009 ~ 2011 : University of Tennessee
- 2012 ~ : VD
- ORCID : orcid.org/0000-0003-3214-0641
- : 3D
- 1998 : - 2000 : - 2008 :
- 2008 ~ : DMC
- ORCID : orcid.org/0000-0002-9466-2958
- : 3D
- 1984 :
- 2003 : IT
- 2015 ~ : 3D
- 2002 ~ 2009 : CIO
- 1985 ~ : - 2009 ~ : 3D
- ORCID : orcid.org/0000-0002-1931-4742
- : , , 3D , , 3D
