단일 채널에서 블라인드 음원분리를 통한 하이브리드 BCI시스템 최적화
양 다린1, 트렁 하우 뉘엔1, 김종진1, 정완영1＊
1부경대 학교 전자 공학과
The Optimization of Hybrid BCI Systems based on Blind Source Separation in Single Channel
Da-Lin Yang1, Trung-Hau Nguyen1, Jong-Jin Kim1 and Wan-Young Chung1*
1Department of Electronics Engineering, Pukyong National University
요 약 현재의 연구에서는 소음을 제거하기 위해 블라인드 소스 분리(BSS)접근 방식에 의해 최적화된 두뇌-컴퓨 터 인터페이스(BCI)를 제안했다. 모터 이미지(MI)신호와 정상 상태 시각적 제거 전위(SSVEP)신호는 신호 대 잡 음비(SNR)의 증가로 인해 쉽게 검출되었다. 또한, MI와 SSVEP사이의 조합은 일반적으로 현재 BCI에서 생성되 는 명령 수를 증가시킬 수 있다. 현재 시스템은 계산 시간을 줄이고 BCI를 실제 용도에 가깝게 하기 위해 단 일 채널 EEG신호를 사용했다. 또한, 복잡한 신경 네트워크(CNN)가 다중 클래스 분류 모델로 사용되었다. 우리 는 비 MS/BCI와 BBS/BCI사이의 정확성 측면에서 성능을 평가했다. 결과적으로 BBS+BCI의 정확도는 비 BBS+BCI의 정확도보다 16.15±25.12%더 높은 수준에 도달했다. 사용하지 않을 때보다 BBS를 사용함으로써 전 반적으로 제안된 BCI시스템은 비교적 정확한 다차원 제어 애플리케이션에 적용될 가능성을 입증했다.
• 주제어 : BCI, MI, SSVEP, BBS, CNN
Abstract In the current study, we proposed an optimized brain-computer interface (BCI) which employed blind source separation (BBS) approach to remove noises. Thus motor imagery (MI) signal and steady state visual evoked potential (SSVEP) signal were easily to be detected due to enhancement in signal-to-noise ratio (SNR). Moreover, a combination between MI and SSVEP which is typically can increase the number of commands being generated in the current BCI. To reduce the computational time as well as to bring the BCI closer to real-world applications, the current system utilizes a single-channel EEG signal. In addition, a convolutional neural network (CNN) was used as the multi-class classification model. We evaluated the performance in term of accuracy between a non-BBS+BCI and BBS+BCI. Results show that the accuracy of the BBS+BCI is achieved 16.15±5.12% higher than that in the non-BBS+BCI by using BBS than non-used on. Overall, the proposed BCI system demonstrate a feasibility to be applied for multi-dimensional control applications with a comparable accuracy.
• Key Words : Brain-computer interface (BCI), Motor Imagery (MI), Steady State Visual Evoked Potential (SSVEP), Blind Source Separation (BBS), Convolutional Neural Network (CNN).
Received 19 January 2018, Revised 03 January 2018, Accepted 29 March 2018
Ⅰ. Introduction
Brain computer interface (BCI) has recently established a new pathway for communication between human and the world beside the conventional neuromuscular pathways. Recent single-channel BCI system has focused on finding new feature extraction algorithms, classification methods and combining existing approaches to develop hybrid BCIs. In his study, we propose an hybrid BCI utilizing MI and SSVEP hybrid feature.
MI is typically observed on the motor cortex during user's movement imagining of the left, right hand or foot. The movement imagining could cause event-related de-synchronization (ERD) in the contralateral side and event-related synchronization (ERS) in the ipsilateral side of the brain.
Moreover, mu rhythms (8-13 Hz) are found to be strongly connected to motor activities [2]. SSVEP is a response of the brain to visual stimuli.
SSVEP-based BCIs normally applied in spelling assistance, game control etc. in which subjects is asked to gaze on stimulator consisting different- frequency stimuli. Both techniques have recently obtained good achievements. However hybrid system has shown to be able to enhance classification accuracy as well as to increase information transfer rate (ITR). Normally, a hybrid BCI uztilizing MI and SSVEM is obtained by simultaneously acquiring EEG signals from motor cortex and occipital cortex that typically requires at least two channels. Considering the simple system and computational time, the proposed BCI acquires a single-channel EEG (either C3 or C4) which can be used to simultaneously captured MI and SSVEP.
Blind Source Separation (BSS) plays an important role in signal processing. It is used to extract or recover a set of source signals from a set of mixed signals, based on probability theory [11-13]. In this study, principle component analysis (PCA) and independent component analysis (ICA)
were employed to remove artifact signals (i.e., EOC, EMG) from the EEG as shown in Figure 1.
Recently, CNN has became one of the most powerful technique used in the field of data mining such as data classification, segmentation etc.. An important advantage of a CNN-based classification model is that it can be used to classify many classes. Thus, the model can generate more commands resulted from several mental tasks of the brain.
The paper is organized as follows: the methodology including system block diagram, experimental paradigm, and signal processing are presented in section Ⅱ, experimental results are shown in sectionⅢ, and conclusion is described in section Ⅳ.
Fig. 1. Demonstration of the system block diagram
Fig. 2. The names and the distribution of EEG electrodes.
Ⅱ. Methodology
2.1 Headset
A 16-channel EEG headset (Cognionics Inc., San Diego, USA) was used in the current system.
Figure 2 shows the position of the channel (red mark) used in the proposed system (either C3 or C4) according to 10-20 international system. EEG signal acquiring from channel C3 is band-pass filtered from 4 Hz to 30 Hz before being wirelessly transferred to PC for analysis. The EEG signal was sampled at 500 Hz.
2.2 Experimental paradigm
Total 11 healthy adults (4 females and 7 males, aged between 23-38 years, mean 28.9 years), who have normal vision and no experience about the system participated in the experiment. Subjects were asked to seat in a comfortable chair and instructed to follow the experiment procedure. A 21-inch LED monitor with 1920*1080 resolution and 60 Hz refresh rate was used as the stimulator.
One of four different flickering frequencies (i.e., frequency 1: 6.6Hz, frequency 2: 7.5Hz, frequency 3: 8.57Hz, and frequency 4: 9.6Hz) is selected as a stimulation frequency. The experiment includes 10 runs. Each run repeated the following tasks for 10 times (frequency 1, frequency 2 + MI, frequency 3, frequency 4 + MI, do nothing). Each task takes 8s to be finished. Users were given 2s for relax after each run.
The procedure for data analysis for data analysis in Fig.3 is organized as following:
(1) Use of a band-pass filter (5-25 Hz) to limit the mu band (8~13Hz) and SSVEP frequency band (6.6-9.6 Hz);
(2) Employing PCA and ICA to extract the EEG singal and then remove noises. MI and SSVEP signals are recovered for analysis;
(3) Performing Short-time Fourier transform (STFT)
to get 2D images which are typical the input of the CNN model.
(4) Training the CNN model and then validate the model.
2.3 Signal Processing
2.3.1 Principal Component Analysis
In this paper, PCA was performed to obtain four most important components:
PC1=X*eigVec(:,1);
PC2=X*eigVec(:,2);
PC3=X*eigVec(:,3);
PC4=X*eigVec(:,4),
where X is the raw EEG signal and eigVec is the diagonal elements of the covariance matrix.
2.3.2 Independent Component Analysis ICA has been used as a preprocessing technique prior to the feature extraction step to remove the artifact in BCI applications [13]. In addition, in several studies, ICA have been used as a classifier [14].
Assume that at the time instant k, the observed n-dimensional data vector is X(k) = [x1(k), ..., xn(k)]. The acquired EEG signal X(k) is assumed to be mixed with m unknown sources
(1)
Fig. 3 Paradigm of a hybrid brian computer interface system analysis procedure.
The source signals, s1(k),....,sm(k) are supposed to be stationary and independent with each other.
So that, based on the probability theory, the solution is solved in the form:
(2)
where B is called the separating matrix [7].
2.3.3 STFT Analysis
STFT is performed on EEG signal every 4 s.
The time window for fast Fourier transform is 500 ms with an overlapping 300 ms. Then the STFT is done by moving this time window along the signal [8].
(3)
The general formula of STFT transform of a single EEG signal h(n), (n=0, 1,…N-1) is given by the Equation (3), where w(n) is the Hamming window.
Fig. 4 The CNN structure with an unit of 50ms to input into the NN and one command output after training.
2.3.4 Convolutional Neural Network
A data was collected to train the CNN model.
Totally, there are 100 samples were used. As mentioned earlier, the input sample of the CNN model is the STFT of 4-s EEG signal. Therefore, in the STFT image, information of time and frequency are obtained at the same time. As a
result, it is a feature-rich input for the CNN-based classifier as shown in Fig. 4 [9].
The proposed model was validated using 10-fold cross validation method. The CNN model was built using Matlab.
Ⅲ. Experimental Result
3.1 Offline Results
As Fig. 5, we design the cue in C# interface to collect the offline data. The CNN model scans through the STFT image in the convolutional layer.
A 5-by-5 kernel filter is used in the current CNN.
The CNN comprises of four outputs corresponding to five commands of the proposed BCI (turn left, turn right, go forward, go backward, and stop as shown in Table 1).
Command Task
SSVEP 6.6Hz Go forward
SSVEP 7.5Hz+MI Turn left
SSVEP 8.57Hz Go backward
SSVEP 9.6Hz+MI Turn right
MI Stop
Table 1 The navigation task and correlated command
Fig. 5 The offline cue for instructing the subject.
Fig. 6 The STFT feature in different tasks.
An example of hybrid feature is shown in Fig.
6. Apparently, after passing EEG signal through ICA, the mu rhythms and four SSVEP frequencies can be easily observed.
As mentioned in section 2, two directions are controlled by the SSVEP alone, and the rest directions are controled by SSVEP and MI together. Table 1 shows the comparison of system accuracy among three different features (MI only, SSVEP only, and MI + SSVEP). The highest average accuracy of 90.02±2.57% was obtained in the hybrid task, whereas 79.51±4.58% and 82.41±
3.42% accuracies were obtained in the model utilizing MI task only and in the model utilizing SSVEP task only, respectively. The accuracy of the model utilizing hybrid feature is significant higher than that in the model utilizing single task (either MI or SSVEP) (p<0.001, paired t-test) as shown in Fig.7.
3.2 Online Results
The signal processing module is designed based on combination between C# and Matlab programming languages. The real-time graphic user interface (GUI) is constructed using C# whereas most signal processing modules are implemented in Matlab.
These two main modules (GUI and signal processing) can communicate with each other using Dynamic Linking Library (DLL).
Real-time EEG is captured whenever the model detect any coming data in the COM port. After receiving enough 4-s data, the system start to process them. The BCI command generating from the CNN model is transfer to the GUI to update the current status of the game map as shown in Fig. 7.
（a）
(b)
Fig. 7 (a) The accuracy of different tasks for 11 subjects(p<0.001) with comparison of the MI task, SSVEP task and hybrid task. (b) The accuracy of different systems for 11 subjects(p<0.001) with comparison of the system of employing BSS and the
non-used one.
Ⅳ. Conclusion
In this paper, we illustrate a hybrid BCI system utilizing hybrid MI and SSVEP feature. Although the proposed system uses only one channel, we can control a multi-dimensional game. ICA and PCA are both employed to extract a clearly hybrid MI and SSVEP feature.
Fig. 8 The real-time experiment interface.
The hybrid feature based on STFT analysis is used to obtain 2D image for the CNN model. An average accuracy of 90.02±2.2% is achieved which is comparable high in comparison with the work shown in [17] which utilized four-channel hybrid
BCI system (77%±2.5%). Another hybrid system with EC/EO and MI achieved an accuracy of 82.68±
2.6% on average [18]. Futhermore, the result has confirmed the powerful role of the BSS to remove noises as well as to enhance the SNR.
REFERENCES
[1] T. Ebrahimi, J. M. Vesin, and G. Garcia, “Brain -computer interface in multimedia communication,”
IEEE Signal Processing Magazine, vol. 20, pp.
14-24, 2003.
[2] Nicolas-Alonso et al. “Brain computer interfaces, a review,” Sensor, vol. 12(2), pp. 1211-1279, 2012.
[3] L.-W. Ko and S. S. K. Ranga, “Combining CCA and CFP for enhancing the performance in the hybrid BCI system,” in 2015 IEEE Symposium Series on Computational Intelligence, pp. 103–
108, Cape Town, South Africa, 2015.
[4] L.-W. Ko, S.-C. Lin, M.-S. Song, and O. Komarov,
“Developing a few-channel hybrid BCI system by using motor imagery with SSVEP assist,” in 2014 International JointConference on Neural Networks (IJCNN), pp. 4114–4120, Beijing, China, 2014.
[5] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain- computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, pp.
767–791, 2002.
[6] FastICA MATLAB Package [Online]. Available:
HTTP: http://www.cis.hut.fi/projects/ica/fastica [7] A. Hyvärinen, “Survey on independent component
analysis,” Neural Computing Surveys, vol. 2, pp. 94–128, 1999
[8] C.W.N.F. Che Wan Fadzal, W. Mansor, L. Y.
Khuan, “An Analysis of EEG Signal Generated From Grasping and Writing”, In Proceedings of 2011 IEEE Conference on Computer Application
& Industrial Electronics (ICCAIE 2011) , IEEE.
[9] Müller-Gerking J, Pfurtscheller G and Flyvbjerg H 1999 Designing optimal spatial filters for single-trial EEG classification in a movement task Clin. Neurophysiol. 110 787–98
[10] Hyvarinen, A., 1999. Fast and robust fixed -point algorithms for independent component analysis. IEEE T. Neural Netw., 10: 626-634.
[11] Sanjeev, N.J., et al., 2012. Blind source separation and ICA techniques: A review. Int.
J. Eng. Sci. Technol., 4(04): 1490-1503
[12] Olyaee, S., M.S.E. Abadi, R. Ebrahimpour and M.R. Moradian, 2010. A comparative study of two blind source separation approaches to resolve the multisource limitation of the nutating rising-sun reticle based optical trackers. Int. J.
Comput. Electr. Eng.,2(2): 1793-8163.
[13] Gao, J., Y. Yang, P. Lin, P. Wang and C. Zheng, 2010. Automatic removal of eye-movement and blink artifacts from EEG signals. Brain Topogr., 23:105-114.
[14] Nicolas-Alonso, L.F. and G.G. Jaime, 2012.
Brain Computer Interfaces, a review. Sensors, 12: 1211-1279; DOI: 10.3390/s12020121
[15] L.-W. Ko, S.-C. Lin, M.-S. Song, and O.
Komarov, “Developing a few-channel hybrid BCI system by using motor imagery with SSVEP assist,” in 2014 International Joint Conference on Neural Networks (IJCNN), pp.
4114-4120, Beijing, China, 2014
[16] C. Brunner, B. Z. Allisona, D. J. Krusienskib, V. Kaisera, G. R. MullerPutza, G. Pfurtschellera, and C. Neupera, “Improved signal processing approaches in an online simulation of a hybrid brain-computer interface,” J. Neurosci. Methods, vol. 188, no. 1, pp. 165-173, 2010.
[17] Yang D, Nguyen HT, Chung WY. An Online Synchronous Hybrid BCI System for Multidimensional Control Using MI and SSVEP. Summer Conference of IEIE, 2017 Jun:904-7.
[18] Jiang Y, Lee H, Li G, Chung WY. A Hybrid Brain-Computer Interface System for Multidimensional Control Using Motor Imagery and Eye Closure. Journal of Medical Imaging and Health Informatics. 2017 Nov 1;7(7):
1580-8.
저자소개 Da-Lin Yang
2016 Dept. of Computer Science and Tech., Qingdao Univ., China. (Bachelor).
Sep. 2016~present pursuing M.S. degree, Electronics Eng., Pukyong National Univ..
※ Her research interests include Brain-computer interface and signal processing.
Trung-Hau Nguyen
June 2008 Dept. of Mechatronics Eng., HCM City Univ. of Tech., Vietnam (Bachelor).
Sep. 2015 Dept. of Biomedical Eng., Pukyong National Univ. (Master).
Sep. 2015~present pursuing his Ph.D degree, Dept. of Electronics Eng., Pukyong National Univ.
※ His research interests include the development of wearable healthcare.
Wan-Young Chung
1987 Dept. of Electronic Eng., Kyungpook National Univ.. (Bachelor).
1989: Dept. of Electronic Eng., Kyungpook National Univ.. (Master).
1998: Graduate School of Science and Eng., Kyushu Univ., Fukuoka, Japan. (Ph.D).
1993~1999: Semyung Univ.. (Assistant Professor).
1999~2008: Dongseo Univ.. (Associate Professor).
Since Sep. 2008: Dept. of Electronic Eng., Pukyong National Univ.. (Full Professor).
※ His areas of interest include Ubiquitous Healthcare, Wireless Sensor Network Applications, and Gas Sensors.
Jong-Jin Kim
1983: Dept. of Electronic Eng., Kyungpook National Univ.. (Bachelor).
1985: Dept. of Electronic Eng., KAIST. (Master).
1995: Dept. of Electronic Eng., Kyungpook National Univ.. (Ph.D).
Since 1987: Dept. of Electronic Eng., Pukyong National Univ.. (Professor).
※ His area of interest include Computer Archtectures and Ubiquitous Healthcare.
