Improving Recognition of Patent's Claims with Deep Neural Networks

❻

❻⩂

⩂┳

┳

₆

₆⹮

⹮

䔏

䔏䠞

䠞㦮

㦮

㫛

㫛㏣

㏣

㼃

㼃ῂ

ῂ䟃

䟃

㧎

㧎㔳

㔳

Ṳ

Ṳ㍶

㍶

*☯ῃ╖䞯ᾦ 䅊䜾䎆Ὃ䞯ὒ **☯ῃ╖䞯ᾦ 㦋䞿ᾦ㥷㤦

[email protected], [email protected], [email protected] [email protected], [email protected]

G

Improving Recognition of Patent’s Claims with Deep Neural

Networks

Ju-yeon Park*, Yeji Shin*, Minsu Kim*, Dongho Kim**, Jihie Kim** *Dept. of Computer Science and Engineering, Dongguk University

** Dongguk Institute of Convergence Education

Abstract

䔏䠞⯒ 䐋䟊 ₆㑶㦮 ῢⰂ⯒ 㩫㦮䞮ἶ ⽊䢎䞮⓪ 㧒㧊 ⰺ㤆 㭧㣪䟊㰦㠦 ➆⧒ 䔏䠞 ⶎ㍲⯒ ⿚㍳䞮 ⓪ 㡆ῂ ⡦䞲 㭧㣪䟊㰖ἶ 㧞┺. 䔏䧞 䔏䠞㦮 㼃ῂ䟃㦚 㫛㏣䟃ὒ ☛Ⱃ䟃㦚 ῂ⿚䞮ἶ, ὖ⩾♲ 㧎㣿㦚 㺔㞚⌊⓪ 㧒㦖 ὖ⩾ 䔏䠞✺㦚 ⿚㍳䞮⓪◆ ⰺ㤆 㭧㣪䞮┺. ⽎ 㡆ῂ⓪ 㾲⁒ 䎣㓺䔎 ⿚㍳ ⿚㟒㠦 䣣₆ 㩗 ㎇⓻ Ṳ㍶㦚 㧊⊞ BERT(Bidirectional Encoder Representations From Transformers) 㠎㠊 ⳾◎㦚 ㌂㣿䞮 ἶ Neural Network 㦮 䕢㧎 䓲┳ ὒ㩫㦚 䐋䟊 㼃ῂ䟃㦮 ☛Ⱃὒ 㫛㏣㦚 ῂ⿚䞮㡖ἶ, 㧎㣿䞮⓪ 䟃㦮 ⻞ 䢎㢖 㧎㣿 ⶎῂ⪲ 㧊⬾㠊㰚 㧎㣿 䕾䎊㦚 䐋䟊 㫛㏣䟃㦮 㧎㣿 䟃㦚 㺔㞚⌊㠞┺. 㧊 ⹿⻫㦚 2003 ⎚ 㧊䤚㦮 xml 䡫㔳㦮 ⹎ῃ 䔏䠞 ◆㧊䎆㠦 ㌂㣿䞲 ἆὒ, 㩫䢫☚ 99% 㦮 ㎇⓻㦚 䢫⽊䞮㡖┺. 1. Introduction Ͷ 㹾 ㌆㠛䡗ⳛ 㔲╖⯒ ⰴ㧊䞮㡂 ₆㑶㦖 㩦㩦 ザ ⯊Ợ ⼖䢪䞮ἶ ⹲㩚䞮ἶ 㧞┺Ǥ 㧊⩂䞲 㔲╖㦮 㭧㕂 㠦㍲ ₆㑶㦮 ῢⰂ⯒ 㩫㦮䞮ἶ ⽊䢎䞮⓪ 㧒㦖 ⰺ㤆 㭧㣪䞮┺Ǥ 㾲⁒ ⁖⪲⻢ ₆㠛✺ Ṛ㦮 ╖′⳾ 䔏䠞 䂾 䟊 ㏢㏷㧊 㯳Ṗ䞾㠦 ➆⧒ 䔏䠞ⶎ㍲⿚㍳㦚 䢲㣿䞲 㼊Ἒ㩗㧎 㡆ῂ☚ 㣪ῂ♮ἶ 㧞┺Ǥ 䔏䠞 ῢⰂ ⻪㥚㦮 㩫䢫䞲 䟊㍳㦚 㥚䟊㍲⓪ ῢⰂ⻪㥚㠦 ╖䞲 ㌗㥚Ȁ䞮㥚  䙂䞾 ὖἚ㦮 ⳛ䢫䞲 ῂ⿚㧊 䞚㣪䞮┺Ǥ 㼃ῂ ⻪㥚⓪  ☛Ⱃ䟃ὒ 㫛㏣䟃✺⪲ ῂ㎇♮⓪◆SG 㧊⩂䞲G ☛Ⱃ䟃ὒG 㫛㏣䟃✺㦮G ⳛ䢫䞲G ῂ⿚㦚G 䐋䟊G 䔏䠞ⶎ㍲⿚㍳㠦G ☚㤖 㦚G 㭚G 㑮G 㧞┺U 㾲⁒㦮G 䔏䠞⓪G tsVzntsG 䡫䌲⪲G 㩲Ὃ♮㠊SG 㧎㣿 ♲G 㫛㏣䟃㧊G 䌲⁎⪲G 䚲䡚♲┺UG ⹎ῃ㦮G 䔏䠞G ◆㧊䎆G ὖ⩾SG ὒỆ㦮G YWWZ ⎚G 㧊㩚㦮G 䔏䠞⓪G 䎣㓺䔎G 㩫⽊⪲SG 㧎㣿䞮⓪G 䟃㦮G 䌲⁎ṖG ♮㠊㧞㰖G 㞠₆G ➢ⶎ㠦G 㰗㩧G 䕦 ⼚䟊㟒G 䞲┺UG ⡦䞲SG 㫛㏣G ὖἚ⓪G ┺㟧䞲G 䚲䡚㦒⪲G 㧊 ⬾㠊㪎㧞ἶSG 㡺₆ṖG 㧞㦚G 㑮G 㧞₆G ➢ⶎ㠦G 㭒㦮䟊㟒G 䞲┺UG G ➆⧒㍲G ⽎G 㡆ῂ⓪ YWWZ ⎚G 㧊䤚㦮G 䌲⁎G 㼃ῂ⻪㥚ṖG ῂ⿚♲G \ Ⱒ㡂Ṳ㦮G ◆㧊䎆⯒G 䐋䟊G YWWZ ⎚G 㧊㩚G 䔏䠞G ◆㧊䎆㦮G 㼃ῂ䟃G 㧎㣿G ὖἚ⯒G ⳛ䢫䞮ỢG 䞮⓪G ộ㦚G ⳿ 䚲⪲G 䞲┺UG 䔏䠞㦮G 㼃ῂ䟃㦮G ㌗䞮G 䙂䞾G ὖἚG 䕢㞛㦖G 㽳G Y ┾Ἒ⪲G 㧊⬾㠊㰖ⳆSG 㟓G _GaGX 㦮G 㫛㏣䟃ὒG ☛Ⱃ䟃G Ṛ㦮G 䞯㔋◆㧊䎆㦮G 㟧ὒG 㰞㦮G ‶䡫㦚G 䟻㌗㔲䋺₆G 㥚 䟊G 㾲⁒ 䎣㓺䔎 ⿚㍳ ⿚㟒㠦 䣣₆㩗 ㎇⓻ Ṳ㍶㦚 㧊 ⊞ ily{Oi‹™ŒŠ›–•ˆ“G l•Š–‹Œ™G yŒ—™ŒšŒ•›ˆ›–•šG m™–”G {™ˆ•š–™”Œ™šP‚X„G ⳾◎㦚G ㌂㣿䞮ἶSG 䔏䠞G ⿚ ⮮G ◆㧊䎆⯒G 㧊㣿䞮㡂G 䕢㧎G 䓲┳G 䞲┺UG ⶎⰻ㦮G ῂ㫆 ㈦ⰢG 㞚┞⧒G ⶎⰻ㦮G 㧊䟊⯒G 䙂䞾㔲䋾G ily{G ⳾◎㦚G 䐋䟊G 䔏䠞G 㼃ῂ䟃㦮G 㫛㏣G ὖἚ⯒G ⳛ䢫䞮ỢG ῂ⿚䞮ἶSG ◆㧊䎆G ⿞‶䡫G ⶎ㩲☚G 㢚䢪㔲䋾┺UG Ⱎ㰖Ⱏ㦒⪲G 㧎㣿 䞮⓪G 䟃㦮G ⻞䢎㢖G 㧎㣿G ⶎῂ⪲G 㧊⬾㠊㰚G 㧎㣿G 䕾䎊㦚G 㧊㣿䞮㡂G 㫛㏣䟃㦮G 㧎㣿G 䟃㦮G ⻞䢎⯒G 㺔⓪┺UG 㧊 ⹿ ⻫㦚 2003 ⎚ 㧊䤚㦮 xml 䡫㔳㦮 ⹎ῃ 䔏䠞 ◆㧊䎆㠦 ㌂㣿䞲 ἆὒ, 㩫䢫☚ 99% 㦮 ㎇⓻㦚 䢫⽊䞮㡖┺G G G 2. Related Work 䔏䠞⯒ ⿚㍳䞮⓪ ┺㟧䞲 㡆ῂ✺㧊 㰚䟟♮ἶ 㧞⓪ Ṗ㤊◆, 㧊 㡆ῂ㠦㍲⓪ 䔏䠞㦮 㼃ῂ䟃✺㦚 ⿚㍳䞮㡂 ☛Ⱃ䟃ὒ 㫛㏣䟃㦚 ῂ⿚䞮ἶ㧦 䞲┺. 㧊⹎ 䞯㔋♲ BERT ⳾◎㦮 䕢㧎 䓲┳㦚 㧊㣿䞮㡂 䔏䠞㦮 ⿚⮮⯒ 㰚䟟䞮⓪ 㡆ῂ☚ 㧞┺.[2] 㧊 㡆ῂ㠦㍲ ⓪ 䔏䠞㦮 ┺⯎ ⿖⿚㧊 㞚┢ 㼃ῂ䟃 Ⱒ㦚 ⿚㍳䞮㡂 䔏䠞㦮 ⿚⮮⯒ 㰚䟟䞲┺. ṯ㦖 BERT ⳾◎㦮 䕢㧎 䓲

500

┳㦚 㧊㣿䞮㡂 㼃ῂ䟃㦚 ⿚㍳䞮㰖Ⱒ 㾲㫛 ⿚⮮䞮⓪ ἆὒⶒ㧊 䔏䠞 㩚㼊㦮 ⿚⮮⧒⓪ 㩦㠦㍲ 㹾㧊Ṗ 㧞┺. 㼃ῂ䟃㦮 ⿚⮮ 㞢ἶⰂ㯮㦚 䐋䟊 䔏䠞 㼃ῂ䟃㦮 ῂ 㫆⿚㍳㦚 䞮⓪ 㡆ῂ☚ 㫊㨂䞲┺.[3] 㧊 㡆ῂ㠦㍲⓪ 㼃 ῂ䟃㦮 䡫㔳㩗 䔏㰫ὒ 㼃ῂ╖㌗(subject-matter)㦚 㧊㣿 䞲 ☛Ⱃ䟃ὒ 㫛㏣䟃㦚 ⿚⮮䞮⓪ 㞢ἶⰂ㯮㦚 㩲㞞䞲┺. 䞮㰖Ⱒ, 㧊 㡆ῂ㠦㍲⓪ ῃ⌊ 䔏䠞 ⶎ㍲㠦 䞲㩫♮Ⳇ, 㼃ῂ䟃㦮 “n 䟃(㼃ῂ䟃 n)㠦 㧞㠊㍲/㠦㍲”㦮 ⶎῂ㦮 㡂 ⿖⯒ 䐋䟊 ┾㑲䞮Ợ 㫛㏣䟃ὒ ☛Ⱃ䟃㦚 ῂ⿚䞮Ợ ♲ ┺. 㤆Ⰲ㦮 㡆ῂ㠦㍲⓪ BERT ⳾◎㦚 ☚㧛䟊 ⽊┺ 㫛 ㏣䟃㦚 ῂ⿚ 䞲┺⓪ 㩦㠦㍲ 㫖 ▪ ⏨㦖 㢚㩚㎇㦚 ⽊ 㡂㭚 㑮 㧞┺. ❻ ⩂┳ 㦚 䢲㣿䞲 䔏䠞⯒ ⿚⮮䞮⓪ 㡆ῂ[4]㠦㍲⓪ 䔏䠞 ⶎ㍲㦮 IPC(International Patent Classification) 㢖 IPC sub ⿚⮮⯒ 㰚䟟䞲┺. 䔏䠞 ⶎ㍲ ⿚㍳㦚 㥚䟊 䔏 㰫 ⻷䎆✺㦚 㿪㿲䞲 䤚, 㧎䆪▪ 䂋ὒ ⍺䔎㤢䋂 䂋㦚 䢲㣿䞮㡂 䔏㰫 䞯㔋㦚 㰚䟟䞲 䤚㠦 Softmax 䣢‖⯒ 䐋䟊 ⿚⮮⯒ 䞮Ợ ♲┺. 㧊 㡆ῂ㠦㍲⓪ 䔏䠞 ⶎ㍲ ⿚ ㍳㠦 㝆㧊⓪ Softmax 䣢‖ 㧊㣎㠦☚ 䔏䠞 ⶎ㍲㦮 䔏 㰫㦚 㺔⓪ 䞯㔋 ὒ㩫㧊 䞚㣪䞮Ợ ♮⸖⪲ 䔏䠞 ⶎ㍲ ⿚㍳ 㧊㩚㦮 㩚㻮Ⰲ ὒ㩫㧊 ⽋㧷䞮Ợ ♲┺. ⶎ㧻㦮 䡫䌲㏢ ┾㥚㦮 ⿚㍳㦚 䐋䟊 䔏䠞 䟃Ṛ㦮 ῂ 㫆 ⿚㍳㦚 㰚䟟䞲 㡆ῂ☚ 㧞┺.[5] 㧒⽎㠊 䔏䠞⯒ ₆ ⹮㦒⪲ ⶎ㧻㦮 ┾㠊㠦 ➆⧒ 㽳 6 Ṳ㦮 ⶎ㧻 ῂ㫆⪲ ῂ⿚䞮ἶ, 㧊⩆ ῂ㫆⯒ ⹪䌫㦒⪲ 㼃ῂ䟃㦚 䔎Ⰲ 䡫䌲 ⪲ 㔲ṗ䢪䞮㡂 䔏䠞㦮 Ṗ☛㎇㦚 ⏨㧎┺. ➆⧒㍲, 䔏䠞 㼃ῂ䟃㠦 ╖䞲 ▪ 㩫䢫䞲 㧊䟊⯒ 䞶 㑮 㧞㰖Ⱒ, 㧊 㡆ῂ㠦㍲⓪ ┾㑲䧞 㼃ῂ䟃㦮 ῂ㫆Ⱒ㦚 ⋮䌖⌒ ㈦ 㠊 ⟶䞲 㧎㣿 ὖἚ☚ 㞢 㑮 㠜₆ ➢ⶎ㠦 㼃ῂ䟃㦮 㩚㼊 㩗㧎 ⰻ⧓㦚 㧊䟊䞮⓪ ◆㠦⓪ 㠊⩺㤖㧊 㧞┺. 㧊 ⏒ⶎ㠦㍲⓪ 䔏䠞㦮 㼃ῂ䟃㦚 ⿚㍳䞮㡂 㧎㣿 ⶎ ῂ㢖 㧎㣿䞮⓪ 䟃㦮 ⻞䢎⪲ 㧊⬾㠊㰚 㧎㣿 䕾䎊㦚 㧊 㣿䞮㡂 BERT 䞯㔋 ⳾◎㠦 㩗㣿㔲䋾 䤚, 㧊 䞯㔋 ⳾ ◎㦚 㧊㣿䞮㡂 Ṛ┾䞮Ợ 㼃ῂ䟃㦮 㧎㣿 ὖἚ⯒ 䕢㞛 䞮㡂 䔏䠞 㼃ῂ䟃㦮 ⌊㣿㦚 ⽊┺ 㩫䢫䞮Ợ 䕢㞛䞮⓪ ◆ ☚㤖㧊 ♮ἶ㧦 䞲┺. 3. Approach 䡚㨂 ⹎ῃ㦮 䔏䠞⓪ 㼃ῂ䟃㠦㍲ 㧎㣿䞮⓪ 䟃㦮 㩫⽊ ⯒ 䌲⁎ 䡫㔳㦒⪲ (⁎Ⱂ 1)ὒ ṯ㧊 㩲㔲䞮ἶ 㧞┺. (⁎Ⱂ 1) ⹎ῃ 䔏䠞 ⶎ㍲ 㡞㔲 䟃㦮 㧎㣿 ὖἚ 㩫⽊⯒ ⋮䌖⌒ 㑮 㧞⓪ 䌲⁎⓪, 䟃 㦮 㩫⽊⯒ Ṗ㰖ἶ 㧞⓪ <CLAIM ORDER>㢖 㧎㣿 ὖ Ἒ⯒ 㩲㔲䞮⓪ <claim-ref>䌲⁎⪲, 㧊 㧎㣿 䌲⁎✺㦚 㭧㕂㦒⪲ ⿚㍳䞮㡖┺. 㧎㣿G 䌲⁎㢖G 㧎㣿G 䌲⁎㦮G 㞴SG ⛺G ⶎ㧻✺㦚G O⁎Ⱂ YP 㢖G ṯ㧊G 䕢㕇G 䞮㡖┺UG ◆㧊䎆⯒G ⿚㍳䞲G ἆὒSG ṗG ‹ˆ›ˆG 㫛㏣䟃G 㭧G 㧎㣿G 䌲⁎㧊㰖ⰢG ‚Š“ˆ”G RG 㒁㧦„G ⶎῂṖG 㠜⓪G ἓ㤆⓪G 㽳G XS]X_SXX] Ṳ㦮G ◆㧊䎆G 㭧G ^]`] Ṳ⪲G 㟓G WUWW\L㠦G ⿞ὒ䞮㡖┺UG ➆⧒㍲SG ☛Ⱃ䟃ὒG 㫛㏣䟃ⰢG 㧮G ῂ⿚䟊⌎┺ⳊSG ㌆㑶㩗㦒⪲G ``U\LG 㧊㌗㦮G 䢫⮶⪲G 㩫䢫䞲G 㧎㣿G 䌲⁎G 㰖䂾G Ṗ⓻䞶G ộ㧊⧒ἶG 㡞㌗䞮㡖┺UG G (⁎Ⱂ 2) ◆㧊䎆 䕢㕇 㡞㔲 㧊㢖G ṯ㦖G ‚Š“ˆ”G RG 㒁㧦„G 䔏㰫㦚G 䢲㣿䞮㡂G O⁎ⰒG ZPὒG ṯ㧊G 㼃ῂ䟃G ⿚⮮㦮G 㩚⹮㩗㧎G 䝚⪲㎎㓺⯒G ῂ㿫 䞮㡖┺UG G G (⁎Ⱂ 3) 㩚⹮㩗㧎 䝚⪲㎎㓺 G ṗG 㫛㏣䟃ὒG ☛Ⱃ䟃㦚G ῂ⿚䞮₆G 㥚䟊㍲G ❻⩂┳G ⳾ ◎㦚G 䢲㣿䞮ἶ㧦G 䞮㡖㦒ⳆG 㤆㍶G juu ⳾◎㦚G 䢲㣿䞮 㡖┺UG ṗG 㼃ῂ䟃㦮G ⍺⻞㱎G 㧊㌗㦮G 㠊㩞㦚G ▪⹎G ◆㧊 䎆⪲G ⽊ἶG 㧮⧒⌊㠊G 㫛㏣G 㡂⿖⯒G ⧒⻾ⰗG 䞲G ⛺G s––’œ— 䎢㧊なG O⻷䎆䢪G ♲G 㧛⩻G ◆㧊䎆P㦚G ῂ㿫䟊G 㩚

501

㻮Ⰲὒ㩫㦚G Ⱎ㼺┺UG 䞮㰖ⰢG juu ⳾◎㦮G 䔎⩞㧊┳G ἆ ὒ⓪G 㡞㌗⽊┺G ⌄㦖G ^WLG 㑮㭖㦒⪲G ⋮䌖⌂㦒ⳆSG 㧊㠦G ╖䞲G 㤦㧎㦒⪲㍲G ⶎ㧻G ῂ㫆G ₆⹮㦮G 㞢ἶⰂ㯮ὒG ◆㧊 䎆㦮G 䘎䟻㎇O›™œŒaG`WLSGˆ“šŒaGXWLP㦚G 㰖⳿䞮㡖┺UG 㼁⻞㱎G ⶎ㩲㩦㧎G ⶎ㧻G ῂ㫆₆⹮㦮G 㞢ἶⰂ㯮㦚G Ṳ㍶ 䞮₆G 㥚䟊G juu ⳾◎㠦㍲G ⶎⰻ㧊䟊G ₆⹮㦮G 㞢ἶⰂ㯮 㧎G usw ₆⹮G ❻⩂┳G ⳾◎⪲G ⼖ἓ䞮㡖┺UG ⡦䞲G ◆㧊䎆G 䘎䟻㎇㦚G Ṳ㍶䞮₆G 㥚䞮㡂G ㌂㩚G 䞯㔋♲G ⳾◎㦚G 䓲┳ 䞮⓪G 䕢㧎G 䓲┳G ⹿㔳㦚G 䢲㣿䞮㡖┺UG 㧊➢G 䢲㣿䞲G ⳾ ◎㦖G ily{ ⳾◎㧊ⳆG ily{ ⓪G ῂ⁖㧊G ὋṲ䞲G ㌂㩚G 䤞⩾♲G 㧦㡆㠊G 㻮Ⰲ㦮G ❻⩂┳G ⳾◎㧊ⳆG 㧒⿖G ㎇⓻G 䘟 Ṗ㠦㍲G 㧎Ṛ⽊┺G ⏨㦖G 㩫䢫☚⯒G ⽊㧎┺UG 㧊⩂䞲G ㌂㩚G 䤞⩾♲G ⳾◎㦚G 䓲┳䞮㡂G 䢲㣿䞲┺ⳊG ◆㧊䎆㦮G ⿖㫇SG ゚㣿G ⹥G 㔲Ṛ㩗㧎G ⶎ㩲✺㦚G ┺㑮G 䟊ἆ䞶G 㑮G 㧞┺UG G ily{ ⳾◎㦚G 䓲┳䞮₆G 㥚䟊㍶G 䟊╏G ⳾◎㧊G 㧊䟊䞶G 㑮G 㧞⓪G 䡫䌲⪲G ◆㧊䎆⯒G 㩚㻮ⰂG 䞮⓪G ὒ㩫㧊G 䞚㣪䞮 ┺UG 㤆㍶G ⳾◎G 䞯㔋㦚G 㥚䟊G 䢲㣿♶G 䞯㔋G ◆㧊䎆G ㎡G O㟓G XWWSWWW 㡂ṲP㦚G U›š 䡫䌲㦮G 䢫㧻㧦⪲G 㭖゚䞮㡖 ┺UG ┺㦢㦒⪲G 䟊╏G ◆㧊䎆⓪G O⁎ⰒG ZPὒG ṯ㧊G Š–“œ”•G W 㠦G ṗG 䟟㦮G pkSG Š–“œ”•G X 㠦G ṗG 䟟㦮G ⧒⻾O㫛㏣㎇G 㡂⿖PSG Š–“œ”•G Y 㠦G ˆ“—ˆ ◆㧊䎆SG Š–“œ”•G Z 㠦G ṗG 䟟 㦮G ⶎ㧦㡊㦮G 䡫䌲⪲G ῂ㎇䞮㡖┺UG G G O⁎ⰒG ZPG 㩚㻮ⰂG ◆㧊䎆G ῂ㎇G G 䡚㨂₢㰖㦮G 㩚㻮ⰂG ♲G ◆㧊䎆⯒G 㧎Ṛ㧊G 㧊䟊䞶G 㑮G 㧞⓪G ⶎ㧦䚲䡚㦮G 䡫䌲⧒ἶG 䞲┺ⳊSG ┺㦢㦒⪲G ily{ ⳾◎㧊G 㧊䟊䞶G 㑮G 㧞⓪G 䔏㰫䚲䡚㦮G 䡫䌲⪲G 㿪ṖG 㩚㻮 ⰂG ὒ㩫㦚G 㰚䟟䞮㡖┺UG ṗG 䟟㦖G ⧒⻾ὒG 䎣㓺䔎㦮G 䏶 䋆䢪ṖG 㰚䟟♮ⳆG ṗṗ㦮G 䏶䋆G 㕣㦖G 㞴㍶G Š–“œ”•G W 㦮G ‹ 㠦G ➆⧒G ῂ⿚♲┺UG G G O⁎ⰒG [PG{™ˆ••ŽG—ˆ™ˆ”Œ›Œ™G G 㞴㍲G Ⱒ✺㠊㰚G 㩚㻮ⰂG ◆㧊䎆⯒G ㌂㩚G 䞯㔋G ♲G ily{G ⳾◎㠦G 㩗㣿䞮㡂G ™Œ›™ˆ••ŽG ὒ㩫㦚G Ệ㼦G 㫛㏣ 䟃ὒG ☛Ⱃ䟃㦚G ῂ⿚䞮⓪G ⳾◎㦚G Ⱒ✺㠊⌊㠞┺UG 䟊╏G ⳾◎㦮G 䎢㓺䔎G ὒ㩫㦚G 㥚䟊G 㟓G YZSWWW 㡂Ṳ㦮G 䎢㓺䔎G ◆㧊䎆G ㎡㦚G 㭖゚䞮㡖ἶG 㞴㍶G 㩚㻮ⰂG ὒ㩫㦚G Ệ䂲G 䤚 㠦G Œˆ“œˆ›Œ 䞮㡖┺UG G O⁎ⰒG \PGily{ ⳾◎㦮G ™Œ›™ˆ••ŽG G 4. Result NLP BERT ⳾◎㦮 䓲┳ ἆὒ⓪ ┺㦢ὒ ṯ┺. 10 Ⱒ Ṳ㦮 ◆㧊䎆⯒ 䞯㔋㔲䋾 䤚, 2 ⰢṲ㦮 ◆㧊䎆⯒ 䐋䟊 䎢㓺䔎⯒ 㰚䟟䞲 ἆὒ⓪ <䚲 1>, <䚲 2>㢖 ṯ┺. 䟃 䟃⳿⳿ ṨṨ MCC

(Matthews correlation coefficient) 99.97% Evaluate loss 0.07% <䚲 1> BERT ⳾◎ ㎇⓻ 㔺 㔺㩲㩲 ἆἆὒὒ (㫛㫛㏣㏣䟃䟃 19242 ṲṲ, ☛☛ⰓⰓ䟃䟃 4351 ṲṲ) true false ⿚ ⿚⮮⮮ ἆ ἆὒὒ true 19240 Ṳ 0 Ṳ false 2 Ṳ 4351 Ṳ <䚲 2> BERT ⳾◎ 䎢㓺䔎 ἆὒ <䚲 1> ⏨㦖 ⰺ䓲 ㌗ὖἚ㑮(MCC) Ṩ㦒⪲ 䎢㓺䔎 ◆㧊䎆Ṗ ⿚⮮♮㠞┺. <䚲 2> ⿚⮮ ἆὒ㠦㍲⓪ 㽳 19242 Ṳ㦮 㫛㏣䟃 㭧 㔺㩲⪲ 㫛㏣㦒⪲ ῂ⿚䞲 Ṳ㑮 ⓪ 19240 Ṳ, loss ⓪ 2 ṲṖ ⋮㢪┺. ⡦䞲, 㽳 4351 Ṳ 㦮 ☛Ⱃ䟃 㭧 㔺㩲⪲ ☛Ⱃ㦒⪲ ῂ⿚䞲 Ṳ㑮⓪ 4351 Ṳ, loss ⓪ 0 ṲṖ ⋮㢪┺. ἆ⪶㩗㦒⪲ 0.07%㦮 ⌄㦖 ㏦㔺⮶(Evaluate loss)⪲ 䎢㓺䔎 ◆㧊䎆Ṗ 㢂⹪⯊Ợ ⿚ ⮮♮㠞┺⓪ ộ㦚 㞢 㑮 㧞┺.

502

5. Discussion 䔏䠞㦮G 㼃ῂ䟃✺㦚G ☛Ⱃ䟃ὒG 㫛㏣䟃㦒⪲G ῂ⿚䞮ἶSG 㫛㏣䟃㦮G ἓ㤆G 㧎㣿䞮ἶG 㧞⓪G 䟃㦚G 㺔⓪G ily{G 䞯㔋G ⳾◎㦮G 㩫䢫☚ṖG ``LG 㧊㌗㦒⪲G ⏨ỢG ⋮㢪┺UG 䔏䠞㦮G 㫛㏣䟃✺㧊G ╖⿖⿚G 㧎㣿G ⶎῂ㢖G 㧎㣿䞮⓪G 䟃㦮G ⻞䢎G 䡫㔳㦒⪲G 㧎㣿G 䕾䎊㧊G 㩫䡫䢪♮㠊G 㧞₆G ➢ⶎ㠦G ┾㑲 䞲G ily{G 䞯㔋G ⳾◎㦚G 䐋䟊㍲☚G ⏨㦖G 㩫䢫☚⯒G 㠑㦚G 㑮G 㧞㠞┺UG ⁎⩝㰖ⰢG 㧎㣿䞮⓪G 䟃㦮G ⻞䢎ṖG ⳛ㔲♮㠊G 㧞㰖G 㞠Ệ⋮SG 㧎㣿䞮⓪G ⻞䢎ṖG 㡂⩂G ṲOŒUŽUG XSYSZG –™G XTZP㧎G 㡞㣎㩗㧎G ἓ㤆ṖG 㫊㨂䞮⓪◆SG 㰖⁞㦮G ily{G 䞯㔋G ⳾◎⪲⓪G 㧊⩆G 㡞㣎G 䅖㧊㓺✺㦚G 㩫䢫䞮ỢG 㺔㦚G 㑮G 㠜┺UG ➆⧒㍲G 㩫䡫䢪♮㠊G 㧞㰖G 㞠㦖G ㏢㑮㦮G 㡞㣎G 䅖㧊㓺✺㦚G 㻮Ⰲ䞶G 㑮G 㧞☚⪳G 䞮₆G 㥚䟊G ┾㑲䞲G 䕾䎊 㦚G 䐋䟊G 㧎㣿G 䟃㦚G 㺔⓪G ộ㧊G 㞚┞⧒G ⶎⰻ㦮G 㧊䟊⯒G 䙂䞾䞲G 㧦㡆㠊G 㻮Ⰲ⯒G 䢲㣿䞲G 㼃ῂ䟃G 㧎㣿G ⹿㔳㦚G 䞯 㔋䞮㡂SG 㧊⯒G 㺔㞚⌎┺ⳊG 䞮⋮G 㧊㌗㦮G 㼃ῂ䟃✺㦚G ⳾ ⚦G 㺔㞚⌒G 㑮G 㧞ỢG ♮㠊G ▪G ⏨㦖G 㩫䢫☚⯒G 㠑㦚G 㑮G 㧞㦚G ộ㧊┺UG * ⽎ 㡆ῂ⓪ ὒ䞯₆㑶㩫⽊䐋㔶⿖ ⹥ 㩫⽊䐋㔶₆䣣䘟 Ṗ㤦㦮 SW 㭧㕂╖䞯㰖㤦㌂㠛㦮 㡆ῂἆὒ⪲ 㑮䟟♮㠞 㦢" (2016-0-00017) 焾 焾処怾竒

[1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, 24 May, 2019, Google AI Language, 10 Apr, 2020, <https://arxiv.org/abs/1810.04805>

[2] Jieh-Sheng Lee, Jieh Hsiang, “Patent BERT: Patent Classification with Fine-Tuning a pre-trained BERT Model”, 1 Jul, 2019, National Taiwan University, 10 Apr, 2020, <https://arxiv.org/abs/1906.02124>

[3] ㏷⹒䢎, 㧚㏢⧒, ῢ㣿㰚 “䔏䠞 㼃ῂ䟃㦮 ῂ㫆⿚㍳ 㦚 㥚䞲 㼃ῂ䟃 ⿚⮮ 㞢ἶⰂ㯮”, 䞲ῃ䐋㔶╖䣢, 2018, 102-103(2 pages)

[4] Bing Xia, Baoan LI, Xueqiang LV “Research on Patent Classification Based on Deep Learning”, Advances in Intelligent Systems Research, 2016

[5] Akihiro S., Manabu O., Yuzo M., Makoto I. “Patent Claim Processing for Readability”, “03: Proceedings of the ACL-2003 workshop on Patent corpus processing”, 2003, 56-65(10 pages)



503