서론 I.
[1,2].
.
.
.
. ,
.
* (Corresponding Author)
: 2011. 5. 20., : 2011. 6. 5., : 2011. 6. 20.
, :
([email protected]/[email protected])
2010 ( )
(NRF-2010-D00065).
[GRRC 2011-B2, U-city ].
[3], [4]
,
[5,6].
(radial basis function neural networks; RBFNNs) (multilayer)
,
. function approximation, regularization, noisy interpolation,
[7].
[8], [9]
,
.
FCM (Fuzzy C-Means) [10], K-Means [11], Mountain [12], Subtractive [13]
.
(GA: Genetic
K-Means :
K-Means-Based Polynomial-Radial Basis Function Neural Network Using Space Search Algorithm: Design and Comparative Studies
, * (Wook-Dong Kim1 and Sung-Kwun Oh1)
1The University of Suwon
Abstract: In this paper, we introduce an advanced architecture of K-Means clustering-based polynomial Radial Basis Function Neural Networks (p-RBFNNs) designed with the aid of SSOA (Space Search Optimization Algorithm) and develop a comprehensive design methodology supporting their construction. In order to design the optimized p-RBFNNs, a center value of each receptive field is determined by running the K-Means clustering algorithm and then the center value and the width of the corresponding receptive field are optimized through SSOA. The connections (weights) of the proposed p-RBFNNs are of functional character and are realized by considering three types of polynomials. In addition, a WLSE (Weighted Least Square Estimation) is used to estimate the coefficients of polynomials (serving as functional connections of the network) of each node from output node. Therefore, a local learning capability and an interpretability of the proposed model are improved. The proposed model is illustrated with the use of nonlinear function, NOx called Machine Learning dataset. A comparative analysis reveals that the proposed model exhibits higher accuracy and superb predictive capability in comparison to some previous models available in the literature.
Keywords: radial basis function neural networks, K-Means clustering algorithm, weighted least square estimation, space search optimization algorithm, polynomial function
Copyright© ICROS 2011
Algorithms) (PSO: Particle Swarm Optimization)
. [21,22]
[14], [15], [16], [17]
.
(SSOA:
Space Search Optimization Algorithm) K-Means .
.
. ,
. K-Means
.
.
, .
,
1 .
.
. (WLSE: Weighted Least Square Estimation) [18] ,
(SSOA) .
(LSE: Least Square Estimation) (BP: Back Propagation)
.
,
.
.
. II
K-Means
, III
. IV
NOx
NOx [19]
. V
.
클러스터링 기반 다항식 방사형 II. K-Means
기저 함수 신경 회로망
K-Means [11]
, K-Means
. 1 2
, .
1.K-Means 클러스터링 기반 다항식 방사형 기저 함수 신
경 회로망 구조 K-Means
. , K-Means
.
. 1
K-Means
.
(receptive field) , (1)
.
.(1)⋯
(2)(1)
⋯
(2)i(i=1, ...,c) , j( j=1, ...,
m) . (2)
1. K-Menas .
Fig. 1. Architecture of the K-Means clustering-based polynomial- Radial Basis Function Neural Networks.
.
.
( )
.
.
.
, 4 ,
, , .
(3) . , i(i=1, ..., c)
.
(3)(Weighted Least Squares Estimator: WLSE) .
(LSE: Standard Least Squares Estimation) .
,
(interpretability)
. ,
.
(4) .
(4)ai , Y
, Wi i
. Xi i
.
⋮ ⋮ ⋱⋯ ⋯⋯ ⋮
⋮ ⋯⋮ ⋱ ⋯ ⋯ ⋮
(5)N i
(6) .
(6) (7) RMSE (Root Mean Square Error) (8) MSE(Mean
Square Error) .
(7)
(8)N ,
.
클러스터링 알고리즘 2. K-Means
K-Means [11]
.
. , .
.
,
. K-Means .
단계
[ 1] , 0
1 .
∈
(9) , c
, N .
단계
[ 2]
v .
×
(10)
i(i=1,...,c) j( j=1,...,m)
k(k=1,...,N) .
단계
[ 3]
1 .
∥ ∥
(11)
∈ (12)
r .
단계
[ 4] (13) ,
r = r +1 [ 2] .
∥ ∥≤ (13)
(14) .
(14)3 .
공간탐색 최적화 알고리즘을 이용한 모델 최적화 방법 III.
.
공간탐색 최적화알고리즘 1.
(SSOA: Space Search Optimization Algorithm)
.
. 단계
[ 1] ⋯
. , P
D .
단계
[ 2]
.
단계
[ 3]
.
단계
[ 4] (population) M
. 단계
[ 5]
.
≤ ≤ (15)
q(q=1,...,D) , k(k=1,...,M) .
. 1 .
단계
[ 6]
.
× (16)
space .
단계
[ 7]
. 단계
[ 8]
. 단계
[ 9] Fworst(t)
.
(17)
(18) 단계
[ 10] ,
.
단계
[ 11] ,
[step 4] .
공간탐색 최적화 알고리즘을 이용한 모델 최적화방법 2.
. K-Means
.
.
2 .
K-Means
.
. 5
. ,
.
2. K-Means
.
Fig. 2. K-Means clustering-based radial basis function neural network using space search optimization algorithm.
3. .
Fig. 3. Structure of chromosome of space search optimization algorithm.
3 3
. 12 ,
. (7)
(8) RMSE MSE ,
.
시뮬레이션 및 결과 고찰 IV.
.
3 2 -1
synthetic ,
NOx .
5-fold resampling cross validation .
SSOA GA
.
1 SSOA
GA .
비선형 함수 데이터 1.
,
4 500
. 300(60%)
200(40%) .
2 GA
, (8) MSE (Mean
Square Error) .
2 K-Means
.
.
(constant) 1 2
.
GA SSOA
. 4
( ; PI) 0.174
±0.003 , ( ; EPI)
0.178±0.007 . SSOA
0.061±0.001, 0.068±
0.006 , GA 0.084±
0.007, 0.092±0.010 .
(a) A nonlinear function. (b) Data set for experiment.
4. .
Fig. 4. Nonlinear function and associated data set.
2. SSOA GA
.
Table 2. Comparison of performance index of SSOA-based model and GA-based model for nonlinear function.
N T
Without optimization
With optimization SSOA-based model GA-based model
PI EPI PI EPI PI EPI
2
C 0.293
±0.004 0.287
±0.012 0.226
±0.007 0.223
±0.153 0.262
±0.007 0.259
±0.010 L 0.243
±0.006 0.237
±0.011 0.185
±0.005 0.188
±0.004 0.215
±0.006 0.213
±0.007 Q 0.192
±0.004 0.197
±0.011 0.111
±0.007 0.120
±0.011 0.141
±0.006 0.152
±0.007 M 0.241
±0.006 0.236
±0.011 0.151
±0.017 0.157
±0.015 0.206
±0.005 0.205
±0.007
4
C 0.283
±0.005 0.275
±0.010 0.204
±0.008 0.202
±0.005 0.222
±0.006 0.218
±0.008 L 0.236
±0.005 0.231
±0.011 0.146
±0.003 0.149
±0.011 0.171
±0.005 0.172
±0.007 Q 0.174
±0.003 0.178
±0.007 0.061
±0.001 0.068
±0.006 0.084
±0.007 0.092
±0.010 M 0.233
±0.006 0.229
±0.012 0.135
±0.003 0.138
±0.010 0.166
±0.008 0.169
±0.014
6
C 0.284
±0.005 0.277
±0.010 0.210
±0.006 0.207
±0.009 0.232
±0.004 0.227
±0.010 L 0.235
±0.006 0.230
±0.011 0.137
±0.009 0.142
±0.010 0.164
±0.010 0.166
±0.010 Q 0.176
±0.003 0.181
±0.007 0.066
±0.004 0.073
±0.010 0.095
±0.009 0.102
±0.011 M 0.233
±0.006 0.228
±0.012 0.129
±0.006 0.133
±0.009 0.164
±0.008 0.167
±0.014 N: No. of nodes T: Polynomial Type
C: Constant L: Linear Q: Quadratic M: Modified Quadratic
1. .
Table 1. Initial setup parameters of optimization algorithm.
SSOA GA
Parameters Values Parameters Values Generation size 100 Generation size 100 Population size 50 Population size 50 No. of chromosome
for crossover 8 Selection Roulette wheel weight of crossover [-0.5 1.5] Crossover One point
Mutation Uniform Crossover rate 1 Mutation rate 0.01 Mutation Uniform
- - Mutation rate 0.05
- - Elitism Used
-1 -0.5 0 0.5 1 -0.5 -1
0.5 0 10 0.2 0.4 0.6 0.8
-1 -0.5 0 0.5 1
-0.5 -1 0.5 0 -0.21
0 0.2 0.4 0.6 0.8
(a) Constant. (b) Linear.
-1 -0.5 0 0.5 1
-0.5 -1 0.5 0 1 -0.5 0 0.5 1
-1 -0.5 0 0.5 1
-0.5 -1 0.5 0 -11 -0.5 0 0.5 1
(c) Quadratic. (d) Modified quadratic.
5. SSOA .
Fig. 5. Shape of the model output according to polynomial type of SSOA-based model.
5 4 SSOA
. 5(a)
quadratic , modified quadratic, linear .
10 20 30 40 50 60 70 80 90 100
0.145 0.15 0.155 0.16 0.165 0.17 0.175 0.18 0.185
Generations MS
E o f T rain ing data se t
SSA-based model GA-based model
10 20 30 40 50 60 70 80 90 100
0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.2
Generations MS
E o f T rain ing data set
SSA-based model GA-based model
(a-1) 4 nodes (a-2) 6 nodes (a) Linear.
10 20 30 40 50 60 70 80 90 100
0.05 0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13
Generations MS
E o f T rain ing da tase t
SSA-based model GA-based model
10 20 30 40 50 60 70 80 90 100
0.08 0.09 0.1 0.11 0.12 0.13 0.14
Generations MS
E o f T rain ing data set
SSA-based model GA-based model
(b-1) 4 nodes (b-2) 6 nodes (b) Quadratic.
10 20 30 40 50 60 70 80 90 100
0.13 0.14 0.15 0.16 0.17 0.18
Generations MS
E o f T rain ing data se t
SSA-based model GA-based model
10 20 30 40 50 60 70 80 90 100
0.1 0.12 0.14 0.16 0.18 0.2
Generations MS
E o f T rain ing data se t
SSA-based model GA-based model
(c-1) 4 nodes (c-2) 6 nodes (c) Modified quadratic.
6. SSOA GA .
Fig. 6. Convergence process for the training dataset in each case of SSOA and GA.
6 quadratic
SSOA GA
. ,
MSE , MSE
. SSOA
. 데이터
2. NOx
NOx NOx
5 -1 260
.
Tamb (The ambient temperature at site), COM (The compressor speed), LPT (The low pressure turbine speed), Pcd (The compressor discharge pressure), Texh (The turbine exhaust temperature) .
130(50%) 130(50%) ,
MSE .
3
. GA
SSOA .
NOx
.
6 quadratic
3. NOx SSOA GA
.
Table 3. Comparison of performance index of SSOA-based model and GA-based model for NOx dataset.
N T
Without optimization
With optimization SSOA-based model GA-based model
PI EPI PI EPI PI EPI
4
C 442.01
±18.86 491.40
±59.04 64.19
±8.19
68.85
±9.60 51.41
±14.38 60.47
±17.89 L 15.42
±0.84 19.22
±1.73 0.87
±0.19 1.52
±0.59 0.93
±0.30 1.76
±0.26 Q 2.05
±0.16 3.06
±0.50 0.03
±0.01 0.32
±0.28 0.05
±0.01 0.27
±0.01 M 7.71
±0.45 10.15
±1.07 0.33
±0.11 1.17
±0.45 0.35
±0.09 0.95
±0.22
6
C 421.74
±25.93 461.00
±56.54 51.75
±2.45
60.98
±9.57 47.75
±10.20 50.81
±9.34 L 14.99
±0.98 19.17
±1.79 0.60
±0.07 1.57
±0.41 0.70
±0.12 1.33
±0.38 Q 1.91
±0.24 3.22
±1.21 0.02
±0.01 0.43
±0.36 0.05
±0.01 0.24
±0.06 M 7.19
±0.57 9.82
±0.87 0.33
±0.03 1.19
±0.35 0.37
±0.06 0.90
±0.27
8
C 351.64
±37.08 387.24
±105.5 55.08
±5.44
65.68
±11.44 40.21
±8.55 44.86
±8.93 L 13.37
±1.22 17.17
±3.17 0.62
±0.09 1.08
±0.22 0.70
±0.11 1.21
±0.26 Q 2.16
±0.38 3.39
±0.89 0.02
±0.01 0.43
±0.47 0.13
±0.19 0.20
±0.14 M 6.74
±0.22 9.36
±1.21 0.32
±0.09 3.54
±4.66 0.40
±0.05 1.58
±0.41 N: No. of nodes T: Polynomial Type.
C: Constant L: Linear Q: Quadratic M: Modified Quadratic.
. 1.91±0.24 , SSOA 0.02±0.01, GA 0.05±0.01
.
SSOA GA
. 4
. FNN (fuzzy set)
, BP(back propagation)
.
(GA) complex . Muti-
FNN HCM
. SSOA
(PI) (EPI)
.
.
결론 V.
SSOA K-Means
.
K-Means , SSOA .
. (WLSE)
. SSOA
. K-Means clustering
SSOA ,
.
참고문헌
[1] L. Zhao, Y. Yang, and Y. Zeng, “Eliciting compact T-S fuzzy models using subtractive clustering and coevolutaionary particle swarm optimization,”
Neurocomputing, vol. 72, no. 10-12, pp. 2569-2575, June 2009.
[2] B. Niu, Y. Zhu, X. He, and H. Shen, “A multi-swarm optimizer based fuzzy modeling approach for dynamic systems processing,” Neurocomputing, vol. 71, no. 7-9, pp. 1436-1448, March 2008.
[3] A. Kandal, L. Li, and Z. Cao, “Fuzzy inference and its application to control systems,” Fuzzy Sets and Systems, vol. 48, no. 1, pp. 99-111, 1992.
[4] Z.-B. Xu, H. Q, J. Peng, and B. Zhang, “A comparative study of two modeling approaches in neural network,”
Neural Networks, vol. 17, no. 1, pp. 73-85, Jan. 2004.
[5] L. Sanchez, I. Couso, and J. Casillas, “Genetic learning of fuzzy rules based on low quality data,” Fuzzy Sets and Systems, vol. 160, no. 17, pp. 2524-2552, Sep. 2009.
[6] S. Kiranyaz, T. Ince, A. Yildirim, and M. Gabbouj,
“Evolutionary artificial neural networks by multi- dimensional particle swarm optimization,” Neural Networks, vol. 22, no. 10, pp. 1448-1462, Dec. 2009.
[7] A. A. Frolov, D. Husek, I. P. Muraviev, and P. Y.
Polyakov, “A boolean factor analysis by attractor neural network,” IEEE Transactions Neural Networks, vol. 3, pp. 698-707, May 2007.
[8] R. A. Aliev, B. G. Guirimov, B. Fazlollahi, and R. R.
Aliev, “Evolutionary algorithm-based learning of fuzzy neural networks. part 2: recurrent fuzzy neural networks,” Fuzzy Sets and Systems, vol. 160, no. 17, pp.
2553-2566, Sep. 2009.
[9] S. B. Roh, S. K. Oh, and W. Pedrycz, “A fuzzy ensemble of parallel polynomial neural networks with information granules formed by fuzzy clustering,”
Knowledge-Based Systems, vol. 23, no. 3, pp. 202-219, April 2010.
[10] J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum, New York, 1981.
[11] S. P. Lloyd, “Least squares quantization in PCM,” IEEE Transactions on Information Theory, vol. 28, no. 2, pp.
129-137, Mar. 1982.
[12] J. Wang, J. Liu, and L. Liu, “A mountain means clustering algorithm,” 7th World Congress on Intelligent Control and Automation (WCICA 2008), pp. 5045-5049, 2008.
[13] N. R. Pal and D. Chakraborty, “Mountain and subtractive clustering method: Improvements and generalizations,” International Journal of Intelligent Systems, vol. 15, no. 4, pp. 329-341, April 2000.
[14] W. Shen, X. Guo, C. Wu, and D. Wu, “Forecasting
4. .
Table 4. Comparative analysis of performance of several selected models for the NOx emission process.
Model Type N PI EPI
Regression model - - 17.68 19.23
Standard Neural
Networks - 25 0.02 3.85
Polynomial networks - 21 0.94 1.09 FNN(GAs+complex)
[19]
Constant 30 6.27 8.78 Linear 30 3.73 5.29 Multi-FNN [20] Constant 120 2.81 5.16 Linear 120 0.720 2.03
Proposed model
Without
OptimizationQuadratic
4 2.05±0.16 3.06±0.50 6 1.91±0.24 3.22±1.21 8 2.16±0.38 3.39±0.89 With
OptimizationQuadratic 6 0.02±0.01 0.43±0.36 8 0.02±0.01 0.43±0.47 N: No. of rules(nodes)
stock indices using radial basis function neural networks optimized by artificial fish swarm optimization,”
Knowledge-Based Systems, vol. 24, no. 3 pp. 378-385, April 2011.
[15] A. Azadeh, M. Saberi, and S. M. Asadzadeh, “An adaptive network based fuzzy inference system-auto regression-analysis of variance algorithm for improvement of oil consumption estimation and policy making: The cases of Canada United Kingdom, and South Korea,”
Applied Mathematical Modelling, vol. 35, no. 2, pp.
581-593, Feb. 2011.
[16] D. Wu, K. Warwick, Z. Ma, J. G. Burgess, S. Pan, and T. Z. Aziz, “Prediction of Parkinson’s disease tremor onset using radial basis function neural networks,”
Expert Systems with Applications, vol. 37, no. 4, pp.
2923-2928, April 2010.
[17] J. Li and X. Liu, “Melt index prediction by RBF neural network optimized with an MPSO-SA hybrid algorithm,”
Neurocomputing, vol. 74, no. 5, pp. 735-740, Feb. 2011.
[18] O. Nelles and R. Isermann, “Basis function networks for interpolation of local linear models,” Proceedings of IEEE Conference on Decision and Control, vol. 1, pp.
470-475, 1996.
[19] S. K. Oh, W. Pedrycz, and H. S. Park, “Hybrid identification in fuzzy-neural networks,” Fuzzy Set and Systems, vol. 138, no. 2, pp. 399-426, Sep. 2003.
[20] S. K. Oh, W. Pedrycz, and H. S. Park, “Rule-based multi-FNN identification with the aid of evolutionary fuzzy granulation,” Knowledge-Based systems, vol. 24, no. 1, pp. 1-13, Jan. 2004.
[21] B. S Kim, S. K. Park, B. H. Choi, E. T. Kim, H. J.
Lee, and H. J. Kang, “Collision risk assessment for pedestrians' safety using neural network,” Journal of Institute of Control Robotics and Systems (in Korean), vol. 17, no. 1, pp. 1-11, Jan. 2011.
[22] K. S. Seo and B. Y. Hyun, “GPU implementation techniques of genetic algorithm and comparative studies,”
Journal of Institute of Control Robotics and Systems (in Korean), vol. 17, no. 32, pp. 328-335, Jan. 2011.
김 욱 동
2009 .
2011 . 2011 ~
.
, , Granular
Computing,
Computational Intelligence .
오 성 권
1981 .
1983 , 1993 .
1983 ~1989 (
). 1996 ~1997 Manitoba Post-Doc.
1993 ~2004 . 2005 ~
. 2002 ~ , ․ ․ ,
.
, - , , computational
intelligence, .
