A Radial Basis Function Approach to Pattern Recognition and Its Applications

        

           

         

 Representational Capability

        

     

     

  !!  

δ# 

       $ %  

      

 




 


        

+",  %&'     

   $

A Radial Basis Function Approach to Pattern Recognition and Its Applications

Miyoung Shin^a) and Cheehang Park

            

   ,    

 

  

 

     

12   

















     

 

∈ ∈ =

=

        '    

         

  ′
′

′    ′

      
 6





 +ε

=

 

 


ε    

  ε 
ε 0  ε
= 

  







= 
 









_ 


∈ !





 = 

    ∈


























=

=
=
= −
=

 







  

−

 

  





 



 − 

=  
− 

+ εε


_





  *

   

        











   

  
*   

 
    


   
− 



  

  



 
− 

=  
− 

+  
−  




 


















=
+





 
  $ 

Model complexity Estimation error = (bias)² + variance Error

variance

(bias)²

    


 



     









 *  

  $   

  

   
− 

       

 
           
    





 . 
  







   





 

     
9 







$
5 

  



 
 

 

'   




^=


  

Input Layer Hidden Layer Output Layer x₁

x_d

y w₁

w_m φ₁

φm











 

 

∑ ∑

= = = −

=

= ^





   



 






φ
φ
µ σ  

φ⋅ µ_σ_

 

 = µµµµ σ

  













=  
! 









φ_
φ

φ=





φ

! φ_⋅  

  

∑



=

= ^

  

 φ

⋅ φ

  φ

=φ
−µ σ

 −








 +
⁻



 +

 ^

   µ_

         

     σ

µµµµ(σ )  (  )

 ,    
µµµµ  σ

           


   

   

     

   

Error

Too much bias (underfitting)

Too much variance (overfitting)

Best Model

Validation Error (Expected)

Model Complexity Training Error (Expected)

$  

$, 

      


     

 µµµµ σ    

$    

  

µµµµ     

  


   !  "     

     

Stage 1 m w Determination of nonlinearity

Determination of linearity RC criterion (δ)

Stage 3 Stage 2

σ µ

σ  

  δ ' σ 

     

σ




δ  

0 

    
   

=

 @@
C δ =

   =

 
         

#$ %

           

RC Algorithm

Step 1:.   σ    

%A  δ    ≤σ ≤  δ 

  B!
C!B
C

Step 2: ?       δ 




×

   $

Step 3:

  $  

$ 


Step 4:

  

1 2 3 4 5 6

0

Input Variable (x) 0

0.5 1 1.5 2

−2

−0.5

−1

−1.5 Output Variable (y)

Training samples and True function


 
  

(ε)  !B"6 ε +

=







 π  ε



  π 

  _ = π_^−= " 
 !BB    

 =


^
=_      

        F $  



  '    _{ }

  

  F          





Step 1: σ δ

. =   ≤σ≤   ≤δ ≤ Step 2:



  σ

  ≥
≥!≥_ ≥ '    

 6





  

× δ

≤

= ≤<  + 

   

(!BBδ

  σ
  σ  

  

×

 F =! 





!



!



!



!



!



!



!



!



!













































−

−

−

−

−

−

−

−

−

=

−

−

−

−

−

−

−

−

−

σ σ

σ

σ σ

σ

σ σ

σ







































!

#

#

#

#

!

!



* σ="







###$



"%&



"'#



"%& "#&  ###$

"#& ( # ###$ ##&$

(%'

 (#&



##$



##"#



##$ ##&$ (%' ( # ##&$ ###$ ( # "#&

###$  "#& "%&

 ###$ "%& "'#































=

!!

!# # #

#

# !!!!!

 ^

     σ
δ= _


 σ

  !

 " #

 $ %

& % %

& ' $

& # $

'  δ =
  σ
   <     ./?  <  

'     $    



$













=

σ 
        σ  

Step 3: µµµµ





 σ





 σ

 

 σ ="  =(

×
  $

$

σ 
 

 σ _{ µ µ}
µ_ µ_ µ µ_ µ_ µ_ µ_

          

        

      

       

      

      

 σ 
 ()


 σ
_{ 
      }

  ! #% **'+ − '! −%$+ '! *+"$ −*$%% −#$$ *'*

 " # #" +#$ −%"$* −$#!' % " % !! −% %!

 $ % −"+* " **" '$$ −"'*

& % % −*$*! −*!# +%% $% −*'!%

& ' $ −** * * # − "$! '#

& # $ − '%! !"! −%% ! %"+

Step 4:

Φ

    





=Φ⁺ 

 Φ⁺ Φ  !  

'$    × Φ

 ×    ×
        

 )







 µµµµσ

=   

 σ   µµµµ

=(  σ ="

     













 −

−

+











 −

−

−

=











  





 









  





_  ^ ^













 −

−

+











 −

−

+



"







!

""



 %

"





&&



!

'



 ^ ^





"





((



!

%



'











 −

−

− 

 

  

   _

1 2 3 4 5 6

0

Input Variable (x) 0

0.5 1 1.5 2

−2

−0.5

−1

−1.5

Output Variable (y)

#
 $%
 &' σ'()*

    


 
 


       $     

 ,             

   $ 

 σ
  ,-


 σ
_) &

  !  *+!  !#"

 " #  +  !$#

 $ %  "$#  ##'

& % %  "%#  ##"

& ' $  "!%  #'"

& # $  "!!  ##



 


%     %&'

    
_=_^
^_^_=_  



 

 ^



∑

= −

=







   

 ^



 F %&'  ^
=

"



  _{ }     $ 


 

  σ 

  *$ 

 G.9 /

/ =" σ =%  

%&'  

**#G  

II III

IV V

I

4 5 6

7

8 9

0.2 0.3 0.4 0.5 0.6 0.2 0.22 0.24 0.26 0.28 0.3

VI

Plot of Training and Prediction Errors

Error

Width of Basis Functions Number of Basis Functions Training Prediction

' +

   

   AH
5  

7B

4    



#'  

δ σ

       δ µ

µµ µ

σ*_+,

1

2

3

4

0 10 20 30 40 50

0 5 10 15 20 25 30

heartc 2.dat: classification errors with respect to M and sigma (delta=1%)

Classification Error (%)

M Sigma

F E D

C B A

training error validation error test error

δ*_+,

 σ
) &

. *% $$ *" *+$

 *! *$% *%+

 % *" *%* *$%

/ " ** *'% *$%

- "% ! *$% *+$

 $ # *#* *#*

 
5    

' >    


 

  ,  

  

 
8δ = $ σ =(!76

σ σ

 (





'

= σ =    

  

δ*_/$_+,

 σ
) &

. *% '% ! *+$

 " * % *#*

 % *+ *"+ **+

/ " *" *%+ *%+

- "% *" *$% *%+

 $ * *$% *%+

δ*_/_+,

 σ
) &

. *% *% *" **

 # %! *!#

 % $# *% *!#

/ " " *% *+$

- "% " *"+ *#*

 $ *! *"+ *%+

δ 0 σ
) & 

* % *" *%* *$% *+#

% "% *" *$% *%+ *+#

* $ *! *"+ *%+ *!#

 (



=

δ  B!C


' δ   

 σ

   %&'    σ = (='

δ =
C

 δ

=> @δ =B7
C B!
C





 σ  

5      

     *

  

     $      ,

%&'          

   

   

    

 !"#$
%"


,-"
 !"#$
%"
 


.
/
) 
0
%--
12"#34
) 
#"
"#
5*)#



9
&
%3"((
10"
0"$

'))(
)
.: ;#
5+

*)#

 6
 
   5
=-#

 !"#$
%"
7(



 8 


9
>
."*) 
1
)
7)) ;"
?+(
) 
#"
:"++

 
@
/"*) 
A
" " #;4
) 
'
&:)#
12":)(
2"#34
) 


8


9
$
) 

9
&)4" 
1.)#
=")  -

2"#34

=+


8


@
" 

.
2
3) 
) 
%

/) #
1#- )(
=")#

@B:)"
=")  -
5(-#*

'))(
)
.: ;#
2"#+

);


 8 



 
&"#*)
&
/)*) 4
) 
9
2
0#4(
1A#*)#


0*"+7)$ -
%))*"#"

@#)##;)(
"(C
)
#*D)#

5);6
E
)*'+,
2)!(("
0" """
@5



8

9 
>4 
 !"#$
%"
0"
0
A# 
 


=
%";"(#
1%'A2+5
@"#

2":)(
2"#34
" ;*)4

%,("*
) 
" ;*)4 -
':("6
0"; ;)(
'"#
?

 !"#)#
F)(:"
/"*) $
@"#"*,"




@ 
) 
5
=
/"(
15
'.
()"
)"
.)*"34



@ 
) 
5
=
/"(
1F 3("-"
&;!"$
) 
7)()#

.()
5(



7
2
7) 4
*'9
<("$




";"!"
#"
@
) 
@
"-""


;*:#"
;" ;"
*
H "
 !"#$


@":(
F")


) 

"";#!"($

) 
#"
%&
"-""

;*:#"
) 
 *)+

#
;" ;"
*
@$);:"
 !"#$
2"3 H4
@5

)$

.
"
:##)  -

%&
34
"
)3)"
#"
5((+ !"#$

&;#)(
%D"
*
@$);:"
 !"#$
@"


;:" #($
34 -
)
)
" 
*"*,"

#"; ;)(
#)
)#
#"
A(";+

# ;
) 
0"(";**: ;)#
'"");
E ##:#"

F")
>"
"+

");
 #""#
 ;(:"
)#)
*  -
)(-#*
)##"
";- # 
)+

)(
,)
: ;#
*"( -
) 
"!)(:)#

)
!)"#$

,("*


*";)(
)- 
) 
#3)"
" - "" -

)("
$;
*
@":(
2)# )(
 !"#$

F")


#"
@
"-""

;*:#"

;" ;"
*
F")
5!) ;"
 ##:#"


@;" ;"
) 
0"; (-$
F")

 
) 
#"

%&
"-""

;*:#"
;" ;"
*
 !"+

#$

%)

.) ;"


&: -
#"
()#


$")
"
)
,""
 !(!"
)
I";#
(")"


"!")(
()-"
I";#
:;
)
:(#*")
*:#"
@$#"*
&"!"(+

*" #
) 
>-
@""
%))(("(
*:#"
@$#"*
&"!"(*" #
>

"");
 #""#
 ;(:"
*:(#*")
$#"*
#,:#"
$#"*
*+

("3)"
-:3)"
"#34
!#:)(
;*:# -
) 
*,("
)-" #
)+

;#";#:"
>"

;:" #($
A";:#!"
&";#

E *)#
@:#

&! 
A(";# ;
) 
0"(";**: ;)# 
'"");
E ##:#"