麳 싋髜ꦘ꼐yꥧꊃ댣驤싈쏻 묃騳싈꾏ꁯ쏻
땯騳꾇싸ꖛ 魫긛 &3 ꫳ싣 鮟ꨄ뒳 듣되싋 &11 닄묄
ꢧ썳먏 ꢧ鮟쏃 鮯딛驓
싋髜시驤ꁯ싈骿
^KFPRRQ
UOJKN`#NDXNU
MJNLP#NDXDFNU
&RPSUHVVLRQRI&118VLQJ/RZ5DQN$SSUR[LPDWLRQDQG&3
'HFRPSRVLWLRQ0HWKRGV
+\HRQ&KHRO0RRQ*L+ZD0RRQDQG-DH*RQ.LP
.RUHD$HURVSDFH8QLYHUVLW\
됃 달
뫋魫 &11 &RQYROXWLRQDO 1HXUDO 1HWZRUN 뒯 데기 ꫳꛇ 馌먣 듧끌 ꉠ ꁓ대싋 ꮳ땳 ꫳ닫댿꺋 됟꾇싋 꺠ꀔ뒳 ꩣ덛뙫驏 듷뒫鸇 &11 ꠗꃧ듇 騳긟ꔸ ꟃꠗꜛ饯 ꝓ됟 뱓뜿댿 ꊟꓫ ꠗꦃ듫 ꍿꀃ ,R7 ,QWHUQHW RI 7KLQJV 디믇뎯 馈뒯 땯땳ꖔ 쏇騬댿 땰되ꆇ鮟댿ꀃ 떋싋듣 ꊟꛧꁓ ꊟꓫ꺋 &11 ꠗꃧ듇 듳ꢣ 꺠ꀔ뒳 뒏뜯싇꺋 麓쀧됻뷛 ꠗꃧ뒳 닄묄싇ꀃ 鮟ꨄꉓ듣 덟髛ꆇ驏 듷ꁓ ꩧ 黫ꢧ댿꺋ꀃ 싸ꖛ ꫳ싣 鮟꾏듧 땯騳꾇싸ꖛ 魫긛 /RZUDQN DSSUR[LPDWLRQ 뎯 &3 &DQRQLFDO 3RO\DGLF ꫳ싣 鮟ꨄ뒳 騟싘싇덛 &11 ꠗꃧ뒳 닄묄싇ꀃ 鮟ꨄ뒳 떋늷싋ꁓ 떋늷싇ꀃ 鮟ꨄ뒯 騳뮤듇 뒏쎄댿 기驯댵듣 싇鸇듇 싸ꖛꫳ싣 鮟ꨄꜻ뒳 땰되싇ꀃ 鮟뗣듇 鮟ꨄ驫 ꁛꜛ 닄묄 꺠ꀔ뒳 鼁듣鮟 둳싇덛 &11 듇 騳뮤 븯듴댿 ꊟꓫ ꆿ 饯뜯 ꫳ싣 鮟ꨄ뒳 꺏빌땰뒫ꗋ 땰되싋ꁓ 떋늷鮟ꨄ듇 꺠ꀔ駯뜌뒳 둳싇덛 데기 ꫳꛇ &11 ꠗꃧ듧 9** 5HV1HW 魧ꜛ驏 0RELOH1HW9 ꠗꃧ 닄묄댿 땰되싇덯驏 ꠗꃧ듇 騳뮤 뒏쎄댿 ꊟꓫ ꆿ 饯뜯듇 ꫳ싣 鮟ꨄ뒳 꺏빌땰뒫ꗋ 땰되싗뒫ꗋ냗 땯騳꾇싸ꖛ 魫긛 鮟ꨄꜻ 땰되싋 騬됟 ꩣꁓ a ꦟ듇 ꅈ듫싋 닄묄뒗댿꺋 ꫳꛇ 꺠ꀔ듣 쌔기ꆗ뒳 쏄듧싇덯ꁓ
꺋ꗏ
&11 &RQYROXWLRQDO 1HXUDO 1HWZRUN 뒳 鮟ꦇ뒫ꗋ 싇ꀃ 듧驤끏騬Ꝍ뒯 뫋魫 뱣쉗뺟 ꮳ땳 데기 듧끌 쏃뜷 馋꺏 ꉠ ꁓ대싋 듀되댿꺋 ꑟ댣鸋 꺠ꀔ뒳 ꩣ듣驏 듷ꁓ 魧ꕛ鸇 꺠ꀔ듇 쌔기뒳 둳싋 騳뮤듇 鮹듣 싈뀤싏 饯뚀믇 ZHLJKW 꾇듇 뜌饯ꗋ ꠗꃧ듇 뷛鮟 묃ꗏ 驫떄댿꺋 떀魫싣닫 싏 쀨띄 Ꝥ IHDWXUH PDS 뒳 땯디싇鮟 둳싋 ꟃꠗꜛ饯 뷛駻 뜌饯싇駻 ꆇ댷ꁓ 듣댿 ꊟꓫ 덟긟 껼ꄳ鸇 ꟃꠗꜛ饯 떋싋ꆋ ꠗꦃ듫 ,R7 鮟鮟댿 듧驤끏騬Ꝍ뒳 묃ꗏ싇鮟댿ꀃ 떋싋듣 ꊟꛧꁓ ꊟꓫ꺋 鮟뗣듇 싈뀤ꆋ 麓쀧됻뷛 ꠗꃧ듇 꺠ꀔ뒳 뫋ꁯ싋 뒏뜯싇꺋 ꠗꃧ듇 뷛鮟ꛫ 뙳듣ꀃ 듧驤끏騬Ꝍ 닄묄 鮟ꨄꉓ듣 덟髛ꆇ驏 듷ꁓ>@ &11 ꠗꃧ뒳 닄묄싇ꀃ 鮟ꨄ뒯 饰 ꠗꃧ댿 셛싗ꆋ 饯뚀믇듇 꾇ꛫ 뙳듣ꀃ ꃟ댿 뚀땿뒳 ꆿ驏 듷ꁓ ꁯ쇋땰듧 鮟ꨄ뒫ꗋꀃ ꠗꃧ듇 꺠ꀔ댿 뷟 데쌔뒳 뙫뜯 늹뒯 饯뚀믇뎯 黧ꉋ듇 덟騟뒳 鶹ꀃ 饯뜯믇鮟 SUXQLQJ 鮟ꨄ >@驫 饯뚀믇 싸ꖛ뒳 馋 듣기듇 싸ꖛꗋ ꫳ싣싇덛 饯뚀믇듇 꾇ꛫ 뙳듣ꀃ 싸ꖛ ꫳ싣 鮟ꨄ ꉠ듣 듷ꁓ>@ ꩧ 黫ꢧ댿꺋ꀃ 싸ꖛ ꫳ싣 鮟꾏듧 땯騳꾇싸ꖛ 魫긛 /5 /RZ 5DQN DSSUR[LPDWLRQ 뎯 &3 &DQRQLFDO 3RO\DGLF ꫳ싣 鮟ꨄ뒳 騟싘싇덛 &11 ꠗꃧ뒳 닄묄싇ꀃ 鮟ꨄ뒳 떋늷싋ꁓ
싸ꖛ ꫳ싣 鮟ꨄ
麳 싋髜ꦘ꼐yꥧꊃ댣驤싈쏻 묃騳싈꾏ꁯ쏻 싸ꖛ ꫳ싣 鮟ꨄ뒯 릗됿 듣기듇 &11 饰 騳뮤듇 饯뚀믇 싸ꖛ뒳 馋 듣기듇 싸ꖛꗋ ꫳ싣싗뒫ꗋ냗 饯뚀믇듇 꾇 덟긟ꔸ뒳 뙳듣ꀃ ꦘꨄ듣ꁓ ꁯ쇋땰듧 싸ꖛ ꫳ싣 鮟ꨄ뒯 릗됿 싸ꖛ뒳 69' 6LQJXODU 9DOXH 'HFRPSRVLWLRQ 鮟ꨄ뒳 듣되싇덛 馋듇 싸ꖛꗋ ꫳ싣싇ꀃ 땯騳꾇싸ꖛ 魫긛 鮟ꨄ驫 릗됿 듣기듇 싸ꖛ뒳 ꁓ꾇듇 騳꾇 UDQN 뺿꺋 WHQVRU 듇 꺏쎄騟싘 쎄빋ꗋ ꫳ싣싇ꀃ &3 ꫳ싣 鮟ꨄ ꉠ듣 듷ꁓ 땯騳꾇싸ꖛ 魫긛 鮟ꨄ뒯 릗됿 싸ꖛ뒳 69' ꫳ싣 鮟ꦇ뒫ꗋ 馋듇 릗됿 싸ꖛꗋ ꫳ싣싇ꀃ 駲뒫ꗋ 끌 驫 馈듣 쇋썳ꆋꁓ
ܹ
ൌ ܷ
ܸ
் (1) 㡂₆㍲G ܹǡ ܷǡ்ܸ⓪G ṗṗG ܯ ൈ ܰSG ܯ ൈ ܴSG ܴ ൈ ܰG 䋂₆⯒G Ṗ㰖ⳆSG ܹ⓪G ṗG juuG ⳾◎㦮G ݅⻞㱎G 䂋㦮G Ṗ㭧䂮G 䟟⪂㦚G 㦮⹎䞮ⳆSG ܷǡ ்ܸ ⓪G ݅ ⻞㱎G 䂋㦮G Ṗ㭧䂮G 䟟⪂⪲䎆G 䟊♮⓪G 䟟⪂㦚G 㦮⹎䞲┺UG jw ⓪G Z 㹾㤦G 㧊㌗㦮G ἶ㹾㤦G 䎦㍲✺㦚G TXG 䎦㍲✺㦮G ㍶䡫ἆ䞿㦒⪲G 䟊䞲┺UG 㡞⯒G ✺㠊SG 㔳G OYP㢖G ṯ㧊G Z 㹾㤦G 䎦㍲Gܺ⯒GܴGṲ⪲G 㧊⬾㠊㰚G Z Ṳ㦮G TX 㦮G 䎦㍲⪲G 䟊䞶G 㑮G 㧞┺UG 㡂₆㍲GܴG 㦖G Ἒ㑮G Ṩ㧊ἶSG 䟊╏G ₆⻫㦚G 㩗㣿䞮₆G 㥚䟊G 㧛⩻♮⓪G 䞮㧊䗒G 䕢⧒⹎䎆O G P㧊┺UG ➆⧒㍲SG ܴ Ṩ㠦G ➆⧒G 㞫㿫㥾㧊G ㌗╖㩗㦒⪲G ἆ㩫♲┺UG ؠ ܽோٔ ܾோٔ ܿோ ோ ୀଵ (2) ⽎G ⏒ⶎ㠦㍲⓪G [ 㹾㤦G ₆㦮G 䄾⽒⬾㎮OPG Ṗ㭧䂮G 䎦㍲✺㦚G jwG 䟊G ₆⻫㦚G 㩗㣿䞮㡂G 㞫㿫䞲┺[UG YkG ₆㦮G 䄾⽒⬾㎮G 䂋㦮G Ṗ㭧䂮G Ṩ✺㦖G [kG 䡫䌲OYkG 䞚䎆G 䋂₆SG 㧛⩻㺚⍦G 㑮SG 㿲⩻㺚⍦G 㑮P⪲G ῂ㎇♮㠊G 㧞┺UG ➆⧒㍲SG 䄾⽒⬾㎮G Ṗ㭧䂮G 䎦㍲Gܭ㠦G ╖䞲G jwG 䟊⓪G ┺㦢G 㔳G OZPὒG ṯ㦒ⳆSG ⽎G ⏒ⶎ㠦㍲⓪G Z Ṳ㦮G Ἒ䂋㦒⪲G 䟊䞮⓪G ₆⻫㦚G 㩲㔲䞲┺UG ܭ௧ǡ௦ǡǡؠ ܷǡ௦ ሺଵሻ ܷǡǡሺଶሻܷ௧ǡ ሺଷሻ ோ ୀଵ (3) 㡂₆㍲Gܷǡ௦ሺଵሻSGܷǡǡሺଶሻ SGܷ௧ǡሺଷሻ⓪G ṗṗGܴ ൈ ܵSGܴ ൈ ܦ ൈ ܦSG ܴ ൈ ܶG 䋂₆⯒G Ṗ㰚┺UG 㧊G ➢SG ܵ㢖G ܶ⓪G ṗṗG 㧛⩻ὒG 㿲⩻㦮G 㺚⍦G 㑮SG ܦ ൈ ܦ ⓪G 䄾⽒⬾㎮G 䞚䎆G 䋂₆⯒G 㦮⹎䞲┺UG ➆⧒㍲SG ܷǡ௦ሺଵሻ SG ܷ௧ǡሺଷሻ㦖G 䞚䎆G 䋂₆ṖG X 㧎G wOw~PG 䄾⽒⬾㎮G 䂋㦮G 䡫䌲⪲G ῂ㎇♮ⳆSGܷǡǡሺଶሻ⓪G 䞚䎆G 䋂₆ṖGܦ ൈ ܦ㧎G kOk~PG 䄾⽒⬾㎮G 䂋㦒⪲Gῂ㎇♲┺UG ⁎ⰒG X 㦖G 㔳G OZP㦚G ⁎Ⱂ㦒⪲G ⋮䌖⌎G ộ㧊┺UG ➆⧒㍲SG ἆῃG 䞮⋮㦮G 䄾⽒⬾㎮G 䂋㦚G Y Ṳ㦮G wSG X Ṳ㦮G kG 䄾⽒⬾㎮G 䂋㦒⪲G 䟊䞮ỢG ♲┺UG
GG
G
(a) 䄾⽒⬾㎮ 䂋 (b) CP 䟊♲ 䄾⽒⬾㎮ 䂋 ⁎Ⱂ 1. CP 䟊 ₆⻫[4]G
끓썇騟驫
쇋 뒯 닄묄뒳 싇뜯 늹뒯 됿ꩧ ꠗꃧ듇 꺠ꀔ뒫ꗋ 7RS $FFXUDF\ ꀃ ꠗꃧꗋꫯ뺟 덷뮐ꆋ 뷣ꔇ뀓 뚀 기둳 馋댿 떄ꁤ듣 듷뒳 騬됟듇 떄쏄ꄳꛫ 듇ꥧ싋ꁓ 꺠ꀔ뒳 뮐떄싇鮟 둳싣 긛되ꆋ ꃟ듣뺟꺺뒯 ,/695& ,PDJH1HW /DUJH 6FDOH 9LVXDO 5HFRJQLWLRQ &RPSHWLWLRQ 듇 駯뜌 YDOLGDWLRQ ꃟ듣뺟꺺 馋댿 데기댿 ꁯ싋 ꫳꛇ 꺠ꀔ듣ꁓ>@ 쇋 ꀃ ꩧ 黫ꢧ댿꺋듇 饰 끓썇 ꨳ 뗟駣뒳 鸇븯鸧 駲듣ꁓ 鮟뗣듇 鮟ꨄꉓ뒯 騳뮤듇 뒏쎄댿 기驯댵듣 ꁗ듫 ꫳ싣 鮟ꨄꉓ뒳 땰되싇덯ꁓ ꊟꓫ꺋 鮟뗣듇 鮟ꨄ ꁯꮳ 꺠ꀔ 쌔기뒳 둳싣 饰 끓썇댿꺋ꀃ 騳뮤듇 뒏쎄댿 ꊟꓫ 땰되싇ꀃ 싸ꖛ ꫳ싣 鮟ꨄꉓ뒳 ꁓꛣ駻 땰되싇덯ꁓ 뎳땳 덟騟 )& )XOO\ &RQQHFWHG 뮤듇 騬됟 ꠗꉏ 끓썇뗟駣댿꺋 땯騳꾇싸ꖛ 魫긛 /5 鮟ꨄ뒳 땰되싇덯驏 뱗ꩫꙗ껇 뮤댿ꀃ 馋듇 鮟ꨄ뒳 ꠗꆿ 꺏빌땰뒫ꗋ 땰되싇덯ꁓ ꍿ싋 싣ꁨ 끓썇댿꺋 ꁗ꾋쓷 싸ꖛ ꫳ싣 鮟ꨄꜻ뒳 땰되싇 꺠ꀔ듇 껿끓듣 ꦋ까싇ꀃ 驯騳ꗋ ꨳꄳ듇 딛싈뀤 5HWUDLQLQJ 뒳 땰되싇덯뒫 딛싈뀤뒳 둳싣 ,/695& 듇 싈뀤 ꃟ듣뺟 꺺뒳 긛되싇덯ꁓ Table 1. 㤦⽎ ⳾◎ ㎇⓻ (ILSVRC 2012)Prgho# Prgho#Vl}h#+PE,# Wrs08#Dffxudf|#
+(,#
YJJ049# 85:13# <3138#
UhvQhw83# <;15# <41<6#
PrelohQhwY5# 461;# <3139#
麳 싋髜ꦘ꼐yꥧꊃ댣驤싈쏻 묃騳싈꾏ꁯ쏻
Table 2. 㔺䠮 㫆Ị
# Od|hu#W|sh#
5G#Frqy1# SZ#)#GZ#Frqy1# IF#
Edvholqh#+OU,# OU# OU#
OU#
Whvw#4# FS# FS#
Whvw#5# FS# OU#
GG
䚲G Z 㦖G 䟟⪂G 䟊G ₆⻫㦚G 㩗㣿䞲G ἓ㤆㦮G 㞫㿫G ㎇⓻G 㔺䠮ἆὒ㧊┺UG ṗG 㔺䠮⼚G 㞫㿫㥾㦖G i 㦒⪲G Ṗ㩫䞲G syG ₆⻫㦚G 㩗㣿䞲G 㞫㿫♲G ⳾◎㦮G {T\G ⮮G ㎇⓻㧊G 㤦⽎G ⳾◎G ╖゚G ZLG ⹎Ⱒ㦮G ㏦㔺㧒G ➢⯒G ₆㭖㦒⪲G 䞮㡖┺UG 䟟⪂G 䟊G ₆⻫㧊G 㧚ⶊG ㎇⓻G ㏦㔺G 㠜㧊G 㭒㠊㰚G ⳾◎㦚G XU\¥XYUX ⺆G 㞫㿫䞾㦚G 䢫㧎䞮㡖㦒ⳆSG 䔏䧞G }nnTX] 㠦㍲⓪G ₆㫊㦮G Ṗ㰖䂮₆G ₆⻫XG ╖゚G 㟓G XUZ ⺆G ▪G 㞫㿫㧊G ♾㦚G ⽊㡂㭖┺UG ⡦䞲SG 㩖Ἒ㑮䟟⪂G ⁒㌂G ₆⻫Ⱒ㦚G 㩗㣿䞲G iG ₆⻫⽊┺G Ἒ䂋㦮G 㥶䡫㠦G ➆⧒G sy ὒG jw ₆⻫㦚G ㍶䌳㩗㦒⪲G 㩗㣿䞲G 㩲㞞G ₆⻫㧊 }nnTX]SG yu\W 㠦㍲G 㡗㌗G ⮮G ㎇⓻G 䟻㌗㧊G 㧞㦢㦚G 䢫㧎䞮㡖┺UG G ⁎ⰒG Y ⓪G yu\W 㠦㍲㦮G ṗG 㔺䠮G ⼚G ㎇⓻G 㞫㿫㥾㠦G ➆⯎G ㎇⓻⯒G ⋮䌖⌎G ộ㧊┺UG 䄾⽒⬾㎮G 䂋㠦㍲G 㡺㰗G jw ⯒G 㩗㣿䞲G {G X 㧊G iG ₆⻫⽊┺G 㩚㼊G 㞫㿫㥾G ῂṚ㠦㍲G ⮮G ㎇⓻㧊G Ṧ㏢♮㠞㰖ⰢSG 䄾⽒⬾㎮G 䂋㦮G 㥶䡫㠦G ➆⧒G ㍶䌳㩗㦒⪲G 䟊G ₆⻫㦚G 㩗㣿䞲G {G Y 㠦㍲⓪G ⌄㦖G 㞫㿫㥾G ⻪㥚OWUZ\G¥GWU[\PG 㠦㍲⓪G ₆㫊G iG ₆⻫⽊┺G ⮮G ㎇⓻㦮G 䟻㌗㧊G 㧞㦢㦚G 䢫㧎䞮㡖┺UG G Table 3. 㔺䠮ἆὒ Prgho# Whvw# Wrs08# Dffxudf|# +(,# Frpsuhvvlrq# udwlr#+{,# YJJ049# Eddvholqh# ;:1<5# 4514# Whhvw#4# ;;144# Kddqv#^4`# ;;138# <13# UhvQhw83# Eddvholqh# ;<177# 51;9# Whhvw#4## ;;1:<# Whhvw#5## ;<1;:# PrelohQhwY5# Eddvholqh# ;;153# 4187# Whhvw#4## ;;13:#
G
G
⁎Ⱂ 2. ResNet50 㠦㍲㦮 㞫㿫⮶㠦 ➆⯎ ⮮ ㎇⓻G騟ꗏ
ꩧ 黫ꢧ댿꺋ꀃ 싸ꖛ ꫳ싣 鮟꾏듧 땯騳꾇싸ꖛ 魫긛 /RZUDQN DSSUR[LPDWLRQ 뎯 &3 &DQRQLFDO 3RO\DGLF ꫳ싣 鮟ꨄ뒳 듣되싇 덛 싈뀤ꆋ &11 ꠗꃧ뒳 닄묄싇ꀃ 鮟ꨄ뒳 떋늷싇덯ꁓ 데기 ꫳꛇ 듇 ꁯ쇋땰 &11 ꠗꃧ듧 9** 5HV1HW 魧ꜛ驏 0RELOH1HW9 댿 ꁯ싇덛 饯뜯듇 싸ꖛ ꫳ싣 鮟꾏뒳 꺏빌땰뒫ꗋ 땰되싋 떋늷 鮟ꨄ듇 닄묄 꺠ꀔ뒳 駯뜌싇덯驏 끓썇騟驫 /5 뎯 &3 ꫳ싣 鮟ꨄ뒳 뱗ꩫꙗ껇 騳뮤듇 뒏쎄댿 ꊟꓫ 꺏빌땰뒫ꗋ 땰되싇ꀃ 떋늷 鮟ꨄ듣 ꁗ듫 싸ꖛ ꫳ싣 鮟ꨄ뒳 땰되싋 騬됟ꩣꁓ 닄묄 꺠ꀔ 쌔기듣 듷뒻뒳 쏄듧싇덯ꁓ$FNQRZOHGJHPHQW
듣 黫ꢧ뒯 麳ꄳ 떄ꫯ 驫싈鮟꾏떄ꩣ뼤끏ꫯ 듇 딛됿뒫 ꗋ 싋髜덟髛딛ꁗ듇 뜯됿뒳 ꦊ늳 꾇싸ꆋ 덟髛듳 1R 5)$㺎
ἶ ⶎ 䠢
^4` S., Han, et al, “Deep Compression: Compressing Deep Neural Networks with pruning, trained quantization and Huffman coding,” Computer Vision and Patter Recognition, In Proc. ICLR 2016, May 2016.
^5` C. Aytekin, F. Cricri, T. Wang, E. Aksu, “Response to the Call for Proposals on Neural Network Compression: Training Highly Compressible Neural Networks,” ISO/IEC JTC1/SC29/WG11, m47379, Mar. 2019.
^6` M. Jaderberg, A. Vedaldi, and A. Zisserman, “Speeding up Convolutional Neural Networks with Low Rank Expansions,” In Proc. CVPR, Jun. 2014.
^7` V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, and V. Lemptisky, “Sppeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition,” In Proc. CVPR, Jun. 2015.
^8` [Available at Online]
http://www.imagenet.org/challenges/LSVRC/2012/
G
