We defined five driver states from low alertness to high alertness (drowsiness, combat drowsiness, normal, low load, and high load). Finally, a real-time detection system has been developed to classify the driver's status into normal or overloaded using a multi-layer ECG-based ANN model.
INTRODUCTION
- Background of the Study
- Objectives of the Study
- Significance of the Study
- Organization of the Dissertation
The first goal was to develop an ANN model to classify driver cognitive load based on ECG. Subsequently, Chapter 3 explains the development of the ANN model to classify driver cognitive load based on ECG.
LITERATURE REVIEW
- Cognitive Load
- Drowsiness
- Human Heart
- Electrocardiography and Heart Rate Variability
- Relationship between HRV and Driver States
- Artificial Neural Network
Driver drowsiness generally refers to the driver's tendency to fall asleep at the wheel (Liu et al., 2009). Heart rate is usually derived by converting the average heart period (in milliseconds) to heart rate (in beats per minute) (Lohani et al., 2019; Berntson et al. 2007).
CLASSIFICATION OF DRIVER’S COGNITIVE LOAD LEVELS
- Participants
- Equipment
- Experimental Procedures
- ECG Processing
- ANN Model
- Statistical Analysis
- ECG Measures Performance
In the first step, two sensitive ECG measurements were selected from six ECG measurements for each participant. In the second step, three levels of perceived cognitive load were personally determined for each participant by combining the four driving tasks. Thus, the participant's perceived level of cognitive load was defined as low (driving and driving with the 0-back tasks), medium (driving with the 1-back task), and high (driving with the 2-back task).
Determining the perceived level of cognitive load was more important in this study to prevent a traffic accident. 1 (less than the mean level of cognitive load) the perceived level of cognitive load is low and for the opposite of normalized ECG values (> 1) the perceived level of cognitive load is high. The independent variables were the driver's levels of cognitive load (three levels: low, medium and high cognitive load).
The performance of all ECG measurements related to change in levels of driver cognitive load was assessed for systematic trends and statistical significance.
CLASSIFICATION OF DRIVER’S DROWSINESS LEVELS
- Participant
- Equipment
- Experimental Design
- ECG Processing
- Structure of the ANN Model
- Statistical Analysis
- Subjective Drowsiness Score
- ECG Measures and ANN Model Performance
In the first step, the goals and procedures of the experiment were explained to each participant and the participant's consent was obtained. We manually selected two of the six normalized ECG measurements for each participant as significant inputs for the ANN model. The accuracy of the ANN model is affected by the number of units in the hidden layer.
Meanwhile, the output layer of the ANN model had three units: 1) normal, 2) fighting dormancy, and 3) dormancy. Additionally, a cross entropy function was used to evaluate the performance of the proposed ANN model. The performance of all ECG measures in relation to changing driver drowsiness states was evaluated in terms of systematic trends and statistical significance.
The average classification accuracies for the training and testing sets of the ANN model were 100% and 99.3%.
CLASSIFICATION OF DRIVER’S STATES BASED ON ECG
- ECG under Cognitive Load
- ECG under Drowsiness
- ECG Processing
- ANN Model Development
- Statistical Data Analysis
- ECG Measures Performance
- ANN Model Performance
The performance of the time-domain ECG measures as the driver's states changed from drowsy to high cognitive load. Furthermore, several ANN models with standard feed-forward and back-propagation were constructed to distinguish driver states based on time domain for ECG measures. Meanwhile, the dependent variables were the three time-domain ECG measures (3 levels: average IBI, SDNN, and RMSSD).
In addition, Tukey's tests of the mean IBI, SDNN and RMSSD of the ECG measurements statistically categorized the five driving conditions into four, two and three groups, respectively, as shown in Figure 5.4. In addition, the effects of several variations of time-domain ECG measurements (eg, mean IBI and SDNN; mean IBI and RMSSD; and mean .. IBI, SDNN and RMSSD) as ANN inputs with the accuracy of the proposed ANN model were examined. Then, the performances of ANN models using amplitude-normalized ECG measurements also differed for the training (66–76%) and testing (45–54%) datasets.
Accuracy of the ANN model on training and testing datasets of different normalization methods and input nodes of ECG measures.
DEVELOPMENT OF REAL-TIME DETECTION SYSTEM OF
Development of the Real-time Detection System
- Acquisition of ECG Data Sets
- Data Pre-processing and Selection of Sensitive ECG Measures
- Establishment of Multi-Layer Artificial Neural Network
- Implementation of the Real-time Detection System
Finally, two sensitive ECG measures were selected from the six ECG measures for each participant, in which they varied as reported in previous studies (Mehler et al Gabaude et al., 2012; Brookhuis and De Waard , 2001) with the change of driving conditions (from normal driving to overload driving, or vice versa). An analysis of variance (ANOVA) was performed in Minitab v14.0 (Minitab Inc., USA) to statistically examine the significance of ECG measures as driver status changed (α = 0.05). The training data was used during network training and for adjusting the unit weights of the link layers.
Meanwhile, the test data was used as a completely independent measure of the network's performance for data that was not seen during the training process. As a result, the multilayer ANN model showed perfect accuracy (100%), sensitivity (100%) and specificity (100%) for the training and testing data sets. The window space and update rate of the real-time detection system were 120 seconds and 1 second, respectively.
The flow diagram of the proposed real-time detection system of the driver's cognitive load is shown in Figure 6.4.
Performance Evaluation of the Real-time Detection System
- Methods and Materials
- Experimental design
- Results
In the second session, the post-experiment (main experiment) was performed to evaluate the real-time detection system for the same subject who participated in the first session. Third, the two sensitive measures selected from the pre-experiment were entered into the real-time detection system. In this study, the driver's status under normal or overloaded driving conditions was classified based on the ECG data obtained in real time.
Moreover, the experimental results of the proposed real-time system using the two sensitive measures revealed a status change after the driving condition changed from normal to overload, as shown in Figure 6.8. As can be observed, although the real-time detection system correctly identified the status change of both participants, some delays were observed until the status change was detected, as shown in Figure 6.8. In this study, the driving condition changed at 240 seconds which was just after normal driving (the time to start the secondary task); however, the real-time detection system detected the status change at 246 seconds for participant #1, indicating a delay of 6 seconds.
2, the real-time detection system identified the state change after 243 seconds, which corresponds to a delay of 3 seconds.
DISCUSSION
Classification of Driver’s Cognitive Load and Drowsiness
Changing driver's condition for the cognitive load and drowsiness significantly affected to mean IBI of ECG measures. Mean IBI gradually decreased to cope with an oxygen demand as the cognitive load level increased. A cognitive load promotes oxygen demand by cells and leads to the production of more cardiac output by increasing heart rate.
The sympathetic nervous system is generally activated under high cognitive load and stabilizes the heart rate to produce more cardiac output (Camm et al., 1996; Low, 2013). Since LF was predominantly influenced by the sympathetic nervous system (Billman, 2013; Bezerianos et al., 1999), high cognitive load increased LF by activating the sympathetic nervous system. Mean IBI decreased as cognitive load increased in this study; while mean IBI increased due to sleepiness (Lal and Craig, 2001; Rodriguez-Ibañez et al., 2012) or driving fatigue (Lal and Craig, 2002).
Therefore, it is suggested that cognitive load modulates the sympathetic and parasympathetic nervous systems in an opposite manner to drowsiness and fatigue while driving.
Development of Driver’s States Detection System
In addition, the LF/HF ratio calculated in this study increased when the difficulty of the cognitive load increased; on the other hand, the LF/HF ratio decreased significantly with sleepiness ( Elsenbruch et al., 1999 ; Tasaki et al., 2010 ; Miyaji, 2014 ) or motion fatigue ( Calcagnini et al., 1994 ; Patel et al., 2010 ). ; Yang et al., 2010). The proposed ECG-based evaluation methodology can be adopted for the development of a system that can identify periods of elevated driver cognitive load or drowsiness level. As mentioned above, systematic opposite trends were observed between high cognitive load and sleepiness on ECG measures.
Mean IBI, SDNN, and RMSSD decreased as cognitive load increased; however, they increased as drowsiness increased, which is consistent with the results reported in the existing studies (Mehler et al., 2011b; Lal and Craig, 2001). The results of this study may also confirm that the high cognitive load and drowsiness control the sympathetic and parasympathetic nervous systems of cardiac function in a completely different way (Chowdhury et al., 2018; Tjolleng et al., 2017; Dong et al., 2011; Piotrowski and Szypulska, 2017; Miyaji, 2014; Tasaki et al., 2010). Previous studies have reported that the cardiac response of participants may vary due to the inter-variability among them (Hong et al., 2014; Lee et al., 2010; Lal and Craig, 2001).
Development of real-time driver cognitive load detection system This study also proposed a real-time driver cognitive load detection system.
Development of Real-time Detection System of Driver’s Cognitive Load
The proposed real-time detection system detected the status change with several delays (average delay = 4.5 seconds) for both participants. First, the delay observed in the detection system can be affected by the state of transition from the normal driving state to the overload driving state. Therefore, the detection system can be adopted to create an intelligent vehicle that can support a driver by providing timely intervention and/or warnings (eg, visual, audio, vibration) to prevent vehicle accidents and near misses. .
However, in-depth future research is required to generalize the findings of this study. First, further experiments involving large samples with different demographic characteristics such as age, gender, and nationality are suggested, as the experiments in this study were limited in terms of sample size and demographic characteristics of the participants. Second, this study mainly focused on detecting two levels of the driver's cognitive load states (normal or overload).
Therefore, a field study with a real vehicle is necessary to confirm the results of this study.
CONCLUSION
Driver drowsiness detection using wavelet analysis of heart rate variability and a support vector machine classifier. A comparison of heart rate and heart rate variability indices to distinguish between single-task driving and driving under secondary cognitive workload. Heart rate variability-based recovery during sleep compared with movement analysis, subjective ratings, and cortisol awakening responses.
Heart rate variability for classification of alert versus sleep-deprived drivers in real-time driving conditions. An on-road evaluation of the impact of cognitive workload on physiological arousal in young adult drivers. Changes in heart rate variability indices due to drowsiness in professional drivers measured in a real environment.
Validity of spectral analysis based on heart rate variability from 1-minute or less ECG recordings.
