Predicting Length of Stay in Intensive Care Units after Cardiac Surgery: Comparison of Artificial Neural Networks and Adaptive Neuro-fuzzy System

(1)

I. Introduction

The intensive care unit (ICU) is one of the most specialized and costly parts of a hospital [1], in comparison to other sections, it has one of the highest mortality rates and admits a relatively large number of patients [2]. On the other hand, as hospitals are always struggling to improve their services and reduce costs, ICUs are constantly working on the evaluation, comparison, and improvement of their performance to achieve these goals. For this purpose, measurements of outcome indices, including hospital mortality and length of stay, are usually carried out [3].

Predicting Length of Stay in Intensive Care Units after Cardiac Surgery: Comparison of Artificial

Neural Networks and Adaptive Neuro-fuzzy System

Hamidreza Maharlou, MSc^1,2, Sharareh R. Niakan Kalhori, PhD³, Shahrbanoo Shahbazi, MD⁴, Ramin Ravangard, PhD^1,5

1Department of Health Services Management, School of Management and Medical Information Sciences, Shiraz University of Medical Sciences, Shiraz, Iran; ²Stu- dent Research Committee, Shiraz University of Medical Sciences, Shiraz, Iran; ³Department of Health Information Management, School of Allied Medical Sciences, Tehran University of Medical Sciences, Tehran, Iran; ⁴Shiraz Anesthesiology and Critical Care Research Center, Shiraz University of Medical Sciences, Shiraz, Iran;

5Health Human Resources Research Centre, School of Management and Medical Information Sciences, Shiraz University of Medical Sciences, Shiraz, Iran

Objectives: Accurate prediction of patients’ length of stay is highly important. This study compared the performance of artificial neural network and adaptive neuro-fuzzy system algorithms to predict patients’ length of stay in intensive care units (ICU) after cardiac surgery. Methods: A cross-sectional, analytical, and applied study was conducted. The required data were collected from 311 cardiac patients admitted to intensive care units after surgery at three hospitals of Shiraz, Iran, through a non-random convenience sampling method during the second quarter of 2016. Following the initial processing of influential factors, models were created and evaluated. Results: The results showed that the adaptive neuro-fuzzy algorithm (with mean squared error [MSE] = 7 and R = 0.88) resulted in the creation of a more precise model than the artificial neural network (with MSE = 21 and R = 0.60). Conclusions: The adaptive neuro-fuzzy algorithm produces a more accurate model as it applies both the capabilities of a neural network architecture and experts’ knowledge as a hybrid algorithm. It identifies nonlinear components, yielding remarkable results for prediction the length of stay, which is a useful calculation output to support ICU management, enabling higher quality of administration and cost reduction.

Keywords: Forecasting, Neural Networks, Decision Support Techniques, Length of Stay, Heart Diseases, Cardiac Surgical Procedures, Intensive Care Unit

Healthc Inform Res. 2018 April;24(2):109-117.

https://doi.org/10.4258/hir.2018.24.2.109 pISSN 2093-3681 • eISSN 2093-369X

Submitted: November 23, 2017 Revised: January 28, 2018 Accepted: January 29, 2018 Corresponding Author Ramin Ravangard, PhD

Department of Health Services Management, School of Manage- ment and Medical Information Sciences, Shiraz University of Medi- cal Sciences, Shiraz, Iran. Tel: +987132340774, E-mail: ra_ravan [email protected]

This is an Open Access article distributed under the terms of the Creative Com- mons Attribution Non-Commercial License (http://creativecommons.org/licenses/by- nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduc- tion in any medium, provided the original work is properly cited.

(2)

A patient’s length of stay in a hospital or in any of the general or specialized sections is an index that can be expressed as the days of admission, and it is usually reported as an average. This index may reflect the amount of resources used in a hospital [4]. The length of stay in a hospital is one of the most useful hospital indices that can be used for various purposes, such as hospital care management, quality control, appropriateness of using hospital services, hospital planning, determination of efficiency, and the rate of using hospital resources.

Therefore, due to the underlying role of the patients’

length of stay in ICUs, this index has been well received by researchers; hence, various studies have been conducted worldwide to predict the length of stay in ICUs. The ex- act prediction of patients’ stay in ICUs enables doctors to provide more accurate information for patient satisfaction.

In addition, doctors can more accurately and deliberately regulate patient care plans and provide more assistance to the authorities and planners to determine, prepare, and al- locate financial resources. Moreover, this prediction enables doctors to calculate the length of stay adjusted to risk, and calculate and compare it in and among organizations before and after making changes in everyday hospital management [5].

In fact, this prediction is an important tool for better ser- vice delivery and more patient satisfaction [6]. Hence, designing accurate and reliable models to predict the length of patients’ stay is not only helpful for hospital management, but also for prioritizing macro-policies in the health sector, improving the quality of healthcare services, proper priori- tization of resource allocation based on the difference in the patients’ length of stay on the one hand, and simultaneous attention to the status of the patients’ economic and social indices at a micro-level, on the other hand [6].

Machine learning (ML) is a computational method for inducing and applying knowledge in intelligent systems through particular learning mechanisms; they learn available historical data, and after mathematical model training, they are able to predict a given outcome from example data [7].

ML methods reason from externally supplied records to pro- duce general hypotheses, which then make predictions for events, such as ‘the length of stay at hospital’ in the future, as an example. There are various techniques to build a concise model of the distribution of outcome labels in terms of predictor features. The resulting predictor is then used to assign value labels to testing instances in which the values of the predictor features are known but the value of the class label is unknown [8]. ML techniques have been widely applied in

patient-centric information system development [9].

Artificial neural networks (ANNs) are biologically inspired analytical methods that are able to model extremely complex nonlinear functions. A common type of ANN architecture is the multi-layer perceptron (MLP) with learning by a back- propagation algorithm. This type of neural network is a compound of linked input/output units in which every link has an associated weight. Adjusting the weights is the core phase for predicting the correct class label of input through iterative learning. ANNs are new methods of making predictive models that are more popular than other ML algorithms due to the production of high-precision models [8]. Mean- while, neural networks, with their significant ability to find nonlinear relationships between effective factors and dependent variables as well as the relationships between independent variables through the use of the system of weighing input layer ports, achieve very good prediction performance [8]. Another advantage of this model is the ability of ANNs to learn quickly, which enables them to extract definitive and intelligent responses from fuzzy parameters and avoid com- plicated and time-consuming computations [10-12].

Besides ANN, another intelligent method can be used to determine the length of stay and has achieved good performance, namely, adaptive neuro-fuzzy inference system (AN- FIS). Using a set of input/output data, ANFIS creates a fuzzy inference system. This algorithm is widely used to study phenomena with nonlinear equations [13,14]. Therefore, the combination of fuzzy systems based on logical rules and the ANN approach, which has the ability to extract knowledge from numerical information, enables us to use the available data in addition to human knowledge to build a model.

The system’s membership function parameters are arranged through the post-propagation algorithm or its combination with the least-squares method [15,16]. Such a hybrid learning architecture can be employed to model nonlinear functions and identify nonlinear components, yielding remarkable results [13].

Several studies have used statistical methods, such as the regression method and even various data mining methods, including MLP neural networks and regression tree, to predict the length of stay of patients in hospitals. The results of some studies have demonstrated the accuracy of the methods used to predict the length of stay [3,17,18]. In some of these studies, the neural network approach has achieved better performance than other methods [19,20]. However, all these studies have focused on estimating the length of stay in the hospital. A few studies have also been conducted to determine the length of stay in ICUs. However, in these studies,

(3)

only statistical methods, such as linear regression, have been used [3,21], whereas data mining methods have been rarely used for this purpose [22].

However, since patients undergoing cardiac surgery are routinely sent to the ICU after surgery, to strategically manage ICUs, it is necessary to determine the length of stay of the patients after the surgery. Thus, order and precision in the scheduling of surgeries and the management of ICUs and operating rooms can be improved. This study aimed to use intelligent modeling methods with the capability of ex- ploring linear and nonlinear relationships between the study variables, including ANNs and ANFIS, to predict the length of stay of surgical patients hospitalized in the ICUs of three hospitals in Shiraz, Iran during the second quarter of 2016.

After designing and testing each model, the best one was selected based on a comparison of the models using mean squared error (MSE) and R criteria. Finally, the accuracy of the optimal model in this study is discussed in relation to the results of other previous reports.

II. Methods

This applied research was a cross-sectional descriptive- analytic study carried out in heart surgery ICUs of three hospitals in Shiraz, Iran during the second quarter of 2016 to examine and compare intelligent modeling techniques for prediction of the length of patients’ stay in the hospitals.

The initial data processing was done using the SPSS version 16 software (SPSS Inc., Chicago, IL, USA), and MATLB 8.4 software (MathWorks Inc., Natick, MA, USA) was then used to create and evaluate the models.

The present study was conducted in six stages.

1. Identification of the Factors Affecting the Length of Stay in Intensive Care Units

At this stage, English and Persian articles published from 1990 to 2015 in medical databases, such as Web of Science (ISI), PubMed, SCOPUS, and ProQuest, as well as domes- tic databases, such as SID, IranMedex, and Magiran, were searched and selected using the following keywords and their Persian equivalents: length of stay, hospital stay, ICU stay, neural networks, data mining, coronary artery disease, and cardiovascular diseases. For this purpose, keywords with the same meaning were placed on a common line, and during database searches, the OR operator was used for the keywords that were on the same line, while the AND operator was used to find those on different lines. After selection and review of the articles, 32 variables were extracted as determi-

nants of patients’ length of stay in ICUs after cardiac surgery, and these variables were used for the next stage of the study (Table 1).

2. Data Collection Based on Variables and Identified Factors

The data related to the 32 variables identified in the previous stage were collected by three surgical room nurses in the three hospitals for 311 cardiac surgery patients admitted to the ICUs of the hospitals, who had been selected through a non-random convenience sampling method during the second quarter of 2016. The required data were collected manu- ally from the forms available in the patients’ files, including Perfusion Data Sheets and Anesthesia Record Sheets. Other variables and data not recorded in the patients’ files were collected through interviews with the patients.

3. Data Preparation (Preprocessing) 1) Cleaning

This stage included steps to remove incorrect or incomplete items. Given that the data quality could have been low due to incorrectly registered or unregistered items, the data were reviewed and corrected before the analysis stage. Some of the nominal variables were subdivided into fewer categories, and in some cases, independent quantitative variables were changed into qualitative ones.

2) Variable reduction

To determine the most important variables affecting the dependent variable (the days of patient stay in ICUs), the CART decision tree method was used.

4. Conversion of Electronic Dataset to Learning and Test Datasets

At this stage, the collected data were divided into a learning set (80% of the data as a learning set for making models) and a test set (20% remaining as test and validation data for testing and evaluation of the models).

5. Modeling with ANN and ANFIS Methods

At this stage, the models were created using the ANN and ANFIS methods using the variables and factors determined in the previous stage.

The MLP network consisted of a feedforward network with a sigmoid function in the hidden layer as well as a logarith- mic sigmoid in the output layer. The MLP network with a graded back-propagation gradient algorithm was designed using MATLAB 8.4 software to predict the outcomes of the

(4)

length of stay. To access the best-performing network, various networks with different structures were trained, and the performance of these networks was obtained according to the test dataset (10%), validation (10%), and training data (80%) and were then evaluated. The number of neurons in the network was selected through trial and error. In this regard, the MLP network was trained with various numbers of neurons, and based on the evaluation results, the most ef- ficient network was selected.

Afterwards, the initial inputs were converted into fuzzy sets using Gaussian membership functions, and as inputs to the neural network, learning operations were performed on them.

6. Error Calculation and Comparison of Model Performance

In each of the aforementioned modeling methods, the length of a patient’s stay was determined, and the models were compared by calculating the output MSE and R.

III. Results

The results showed that most of the cases studied were male (63%) and affected by left ventricular dysfunction (81.4%), with none of following conditions, including history of heart surgery (95.8%), renal disease (94.5%), hypertension (52.7%), myocardial infarction (96.5%), respiratory disease (94.5%), peripheral vascular disease (95.8%), coronary artery bypass grafting (98.7%), and embolus (99%). Furthermore, most of the patients included in the study were non-smokers (66.6%), and showed no hypercholesterolemia (81.7%), no hypercreatinine (94.9%), no atrial fibrilliation (92%), and no pre-operation infection (98.7%). Most of the cardiac surgery patients were operated on particularly due to heart bypass (70.4%) or respiratory heart bypass (85.9%) where 85.2%

of operations were elective. Studied patients were affected by heart failure (57.6%), sinus rhythm (82.6%) and mainly needed surgery for off-pump coronary artery bypass grafting (83.6%), cardiogenic shock (99%), mitral valve pathology (69.5%), and mitral valve surgery (86.2%), as shown in Table 1.

Also, the mean age and body mass index (BMI) of the studied patients were 49.38 years and 23.44 kg/m², respectively.

The duration of aortic clamp time was 46.53 minutes and cardiopulmonary bypass (CPB) was 71.7 minutes; the mean value of patients’ hematocrit was 35.58% (Table 2).

Among 32 variables identified in the literature review, 23 influential factors were defined based on the repetition rate

Table 1. Descriptive information of factors influencing length of stay in ICU after cardiac surgery for patients considered in this study

Value Reference

Sex [5,21,23-26]

Male 195 (63)

Female 116 (37)

LVEF [21,24,27-30]

Yes 253 (81.4)

No 58 (18.6)

Previous cardiac operation [21,23,25,30]

Yes 13 (4.2)

No 258 (95.8)

History of renal disease [5,21,28,30]

Yes 17 (5.5)

No 294 (94.5)

Cardiac hypertension [5,6,23,29]

Yes 147 (47.3)

No 164 (52.7)

History of TIA or stroke [5,21,26,27]

Yes 11 (3.5)

No 300 (96.5)

History of pulmonary disease [5,26,28,31]

Yes 17 (5.5)

No 294 (95.8)

Peripheral vascular disease [6,27,29]

Yes 10 (3.2)

No 301 (96.8)

MIDCAB [5,28,31]

Yes 1 (0.3)

No 310 (96.8)

History of embolism [26,28]

Yes 3 (1)

No 308 (99)

Smoking [5,6,29,33]

Yes 104 (33.4)

No 207 (66.6)

Hypercholesterolemia [6,29]

Yes 57 (18.3)

No 254 (81.7)

COPD [27-34]

Yes 10 (3.2)

No 301 (96.8)

(5)

in decision tree nodes used for predicting the length of stay in ICUs after cardiac surgery. These variables are the following: age, gender, surgery type, hematocrit, type of operation, duration CPB, clamp time, left ventricular ejection fraction, renal disease, reoperation, hypertension, OPCAB (off-pump coronary artery bypass), CPB, sinus rhythm, myocardial infarction, mild valvulopathy, NYHA (New York Heart As- sociation), creatinine, MIDCAB (minimally invasive direct coronary artery bypass), HVS (heart valve surgery), hypercholesterolemia, preoperative infection, and BMI. The developed decision tree and variables included in the structure with a produced rule are presented in Figure 1.

Next, having applied the defined influential factors, predictive models were constructed using ANN and ANFIS algorithms. Evaluation criteria for model assessment were calculated. Table 3 presents the results of MLP with different neurons according to R indicator. The results revealed that MLP with 200 neurons in a hidden layer has more ability to Table 1. Continued 1

Creatinine ≥ 2.0 mg/dL [27]

Yes 16 (5.1)

No 395 (94.9)

Atrial fibrillation [5,28]

Yes 25 (8)

No 286 (92)

Preoperative infection [28]

Yes 4 (1.3)

No 307 (98.7)

Surgery type [24,25,27]

CABG 219 (70.4)

MVR 22 (7.1)

AVR 20 (6.4)

Other 50 (16.1)

CPR [29,30]

Yes 267 (85.9)

No 44 (14.1)

Type of operation [5,21,24,25,

28,29,31]

Elective 265 (85.2)

Urgent 28 (9)

Emergency 18 (5.8)

Reoperation/re-exploration [27,31,32]

Yes 12 (3.9)

No 299 (96.1)

NYHA [5,28]

Yes 179 (57.6)

No 132 (42.4)

Sinus rhythm [5,23,31]

Yes 257 (82.6)

No 54 (17.4)

OPCAP [5]

Yes 51 (16.4)

No 260 (83.6)

Cardiogenic shock [28]

Yes 51 (16.4)

No 260 (83.6)

Heart valve surgery [30]

Yes 43 (13.8)

No 254 (86.2)

Table 2. Quantitative variables which positively affected studied patients’ length of stay after cardiac surgery

Age (yr) 49.38 ± 1.18 [5,21,23-27,29,30,32, 34]

Aortic clamp time (min)

46.53 ± 22.95 Confirmed by experts

BMI (kg/m²) 23.44 ± 5.44 [5,29]

Hematocrit (%) 35.58 ± 7.17 [27]

Duration of CPB (min)

71.7 ± 32.46 Confirmed by experts

Values are presented as mean ± standard deviation.

BMI: body mass index, CPB: cardiopulmonary bypass.

Table 1. Continued 2

Mild valve pathology [5,30]

Yes 95 (30.5)

No 216 (69.5)

Values are presented as number (%).

ICU: intensive care unit, LVEF: left ventricular ejection fraction, TIA: transient ischemic attack, MIDCAB: minimally invasive direct coronary artery bypass, COPD: chronic obstructive pulmonary disease, CABG: coronary artery bypass graft, MVR:

mitral valve replacement, AVR: aortic valve replacement, CPB:

cardiopulmonary bypass, NYHA: New York Heart Association, OPCAB: off-pump coronary artery bypass.

(6)

determine patients’ length of stay in ICUs after cardiac surgery.

The output of ANFIS comprises rules composed of fuzzi- fied variables; in the IF part, the independent variables shown in Table 1 are used, and in the THEN part, the quantitative value of patients ICU length of stay are presented in day units.

After model development using the ANN and ANFIS algo-

rithms, assessment criteria including R and MSE were used for evaluation. As shown in Table 4, the considered evaluation criteria were higher for ANFIS than ANN. Also, Table 4 compares the MSE, which is the difference between the real and predicted values of patients’ length of stay in ICUs after cardiac surgery. Also, the value of R regression, which is the correlation between the value predicted by the developed model and the real values of length of stay in ICUs using MLP and ANFIS methods.

IV. Discussion

The methods used in this study were a MLP neural network algorithm and an ANFIS to predict patients’ length of stay (day) in ICU after cardiac surgery. In previous studies, regression methods have been used more than other methods to estimate the length of stay as a quantitative variable (in terms of hours or days). In many of those studies, the length of stay in ICU was predicted in separate classes to be

‘more than 2’, ‘4’, ‘7’, and ‘10 days’ as the outcomes; therefore, logistic regression methods, which are suitable for qualitative dependent variables have been used [3,17,18]. Clearly, quantitative determination and estimation of the length of stay (day/hour) is more useful and appropriate for manag- ing the beds of an ICU as well as policy making at hospitals

x3 < 1.5 x3 > 1.5

x2 < 1.5 x2 > 1.5

x32 < 13.57 x32 > 13.57

6.2 2 3

6

2

7

x5 < 1.5 x5 > 1.5

x25 < 1.5 x25 > 1.5

x3 < 12 x3 > 12

Figure 1. A part of the CART decision tree induced by 32 variables. In the decision tree, X = (x1, …, x32) are introduced as follows: x1 = age, x2 = gender, x3 = surgery type, x4 = hematocrit, x5 = type of operation, x6 = duration CPB, x7 = clamp time, x8 = LVEF, x10 = renal disease, x11 = reoperation, x13 = hypertension, x17 = OPCAB, x19 = CPB, x20 = sinus rhythm, x21 = myocardial infarction, x22 = mild valvulopathy, x24 = NYHA, x25 = creatinine, x27 = MIDCAB, x28 = HVS, x29 = hypercholesterolemia, x30 = preoperative infection, x32 = BMI. As an example inducted rule from decision tree: IF x8 < 3 AND x27 < 1.5 THEN length of stay = 11.2. CPB:

cardiopulmonary bypass, LVEF: left ventricular ejection fraction, OPCAB: off-pump coronary artery bypass, NYHA: New York Heart Association, MIDCAB: minimally invasive direct coronary artery bypass, HVS: heart valve surgery, BMI: body mass index.

Table 3. Results of MLP ANN evaluation based on calculated R for various numbers of neurons in hidden layers of the neural networks structures learned by training, testing, validation, and whole datasets

Whole dataset Validation dataset Testing dataset Training dataset

For 10-neurons in hidden layer 0.6 0.1 0.2 0.8

MLP: multi-layer perceptron, ANN: artificial neural network.

Table 4. Comparison of R and MSE as model assessment criteria for data learning using ANNs and ANFIS algorithms

Machine learning methods

Number of records used

for learning

MSE (%) R

Multi-layer ANN perception

311 21 0.607

ANN-fuzzy (ANFIS) 311 7 0.88

MSE: mean squared error, ANN: artificial neural network, ANFIS: adaptive neuro-fuzzy inference system.

(7)

and making optimal decisions because it provides more precise and defined values [5,6,35,36]. Although Belderrar and Hazzab [37] used the hierarchical genetic algorithm to define the main predictive factors and fuzzy radial basis function neural network to define the optimal predictive model, they estimated the length of stay in hospital rather than a specific ward, such as ICU, particularly after cardiac surgery.

Moreover, the output obtained in this study is more remarkable than other studies, including the study by Lafaro et al. [21], in which the length of patients’ stay after heart surgeries was determined using a neural network with six influencing variables of age, sex, creatinine, etc. The reason is that the value of R obtained was 0/535, which is very weak compared to the outcomes of the present study (R = 0.88).

Moreover, the number of influencing variables identified in the present study was 23, which is stronger than the set of variables used in Lafaro’s study. The value of R obtained in the study by Verburg et al. [3] entitled “A Comparison of Regression Methods for Modeling the Length of Stay in the Intensive Care Unit” conducted in 2014 was about 0.44, which is significantly less than the value obtained in this study (R = 0.88). In the present study, in addition to the high accuracy obtained by the ANFIS method (R = 0.88), another advantage of ANFIS is that it creates outputs as transparent and understandable rules for the users; in the regression algorithms used in other studies, only one numerical output is determined as the impact factor of each variable.

The results of R for training, testing, and validation sets learning using the ANN and ANFIS algorithms are compared in Table 3. To prevent overfitting, the performance of the algorithms was checked with testing and validation sets that were not used for model development. The similarity of the R values for the testing set (0.3) and the validation set (0.4) reveals the quality of the model developed by only the training set and assessed by two other sets, especially for 20 and 50 neurons in the hidden layer of the ANN structure.

ANFIS uses experts’ knowledge to define the cut-offs in defining membership functions for fuzzification of the input variables used in the ANN, and it achieves much better results due to less MSE and higher R. In this study, through applying a decision tree, more effective influential factors were found. Hence, the strength of the method developed in this study is that it uses more proper and precise inputs for mapping by a stronger algorithm (ANFIS) to find the nonlinear relationship, resulting in a valid model for this specific purpose.

These valid models can be used as part of an input-induc- tive inference engine for the knowledge base of a manage-

ment decision-making support system. It will facilitate decision making on the management of ICU beds by determining the length of stay for each patient based on his/

her independent variables influencing the length of stay in the given unit. Moreover, this proposed system may be integrated with the hospital information system as a valuable database with constant patient data collection from provided care, as suggested by Kwon et al. [38].

In this study, the duration of aortic clamp time and the duration of CPB, which are directly related indexes to cardiac patient care in ICUs, were used for the first time for modeling, and ultimately resulted in high precision with an average number of records.

Time and budget constraints prevented further prospective data collection, which could lead to higher precision in future research using additional records.

In conclusion, in this study, neural network and neuro- fuzzy (ANFIS) algorithms were used to create two models for predicting the length of stay of patients undergoing cardiac surgery in the heart surgery ICUs of three hospitals in Shiraz, Iran, and the two models were compared with each other.

The performance of the neuro-fuzzy method was superior to that of other methods. These predictors could be very helpful for hospital managers to carefully manage cardiac surgeries and ICU beds which are limited in number.

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

Acknowledgments

The present article was extracted from the thesis written by Hamid Reza Maharlou and was financially supported by Shiraz University of Medical Sciences (Grant No. 94-01-07- 10441). The authors would like to thank all hospitals managers, nurses and patients who cooperated with the researchers in gathering and analyzing the required data.

References

1. Noorizad S, Tabesh H, Mahdian M, Akbari H, Tagha- dosi M. Causes of mortality and morbidity in a neuro- surgery ICU in Kashan 1999-2001. FEYZ 2005;9(2):15- 20.

2. University of California San Francisco. ICU outcome

(8)

[Internet]. San Francisco (CA): University of Califor- nia; c2018 [cited at 2018 Apr 1]. Available from: https://

healthpolicy.ucsf.edu/icu-outcomes.

3. Verburg IW, de Keizer NF, de Jonge E, Peek N. Compar- ison of regression methods for modeling intensive care length of stay. PLoS One 2014;9(10):e109684.

4. Ghoreishi Nejad S. Agent simulation in healthcare. Ot- tawa, Canada: University of Regina; 2008.

5. Widyastuti Y, Stenseth R, Wahba A, Pleym H, Videm V.

Length of intensive care unit stay following cardiac surgery: is it impossible to find a universal prediction model? Interact Cardiovasc Thorac Surg 2012;15(5):825-32.

6. Hachesu PR, Ahmadi M, Alizadeh S, Sadoughi F. Use of data mining techniques to determine and predict length of stay of cardiac patients. Healthc Inform Res 2013;

19(2):121-9.

7. Kotsiantis SB, Zaharakis I, Pintelas P. Supervised machine learning: a review of classification techniques.

Emerg Artif Intell Appl Comput Eng 2007;160:3-24.

8. Niakan Kalhori SR, Zeng XJ. Evaluation and comparison of different machine learning methods to predict outcome of tuberculosis treatment course. J Intell Learn Syst Appl 2013;5(3):184-93.

9. Fallah M, Niakan Kalhori SR. Systematic review of data mining applications in patient-centered mobile-based information systems. Healthc Inform Res 2017;23(4):

262-70.

10. Nabovati E, Azizi AA, Abbasi E, Vakili-Arki H, Zarei J, Razavi AR. Using data mining to predict outcome in burn patients: a comparison between several algorithms.

Health Inf Manag 2014;10(6):789-99.

11. Akbarian M, Paydar K, Kalhori SR, Sheikhtaheri A.

Designing an artificial neural network for prediction of pregnancy outcomes in women with systemic lupus ery- thematosus in Iran. Tehran Univ Med J 2015;73(4):251- 9.

12. Jung E, Kim J, Choi SH, Kim M, Rhee H, Shin JM, et al.

Artificial neural network study on organ-targeting pep- tides. J Comput Aided Mol Des 2010;24(1):49-56.

13. Jang JS. ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans Syst Man Cybern 1993;23(3):665- 85.

14. Sadeghi M, Yavarmanesh M, Shahidi Nojhabi M. Pre- diction of milk components impact on recovery and extraction of enteric viruses genome using artificial neural networks and adaptive nero fuzzy inference system (ANFIS). Res Iran Sci Technol 2014;10(2):156-64.

15. Buyukbingol E, Sisman A, Akyildiz M, Alparslan FN,

Adejare A. Adaptive neuro-fuzzy inference system (AN- FIS): a new approach to predictive modeling in QSAR applications: a study of neuro-fuzzy modeling of PCP- based NMDA receptor antagonists. Bioorg Med Chem 2007;15(12):4265-82.

16. Goodarzi M, Freitas MP. MIA-QSAR coupled to princi- pal component analysis-adaptive neuro-fuzzy inference systems (PCA-ANFIS) for the modeling of the anti-HIV reverse transcriptase activities of TIBO derivatives. Eur J Med Chem 2010;45(4):1352-8.

17. Polat K, Yosunkaya S, Gunes S. Pairwise ANFIS approach to determining the disorder degree of obstructive sleep apnea syndrome. J Med Syst 2008;32(5):379- 87.

18. Riahi-Madvar H, Ayyoubzadeh SA, Khadangi E, Ebadzadeh MM. An expert system for predicting longi- tudinal dispersion coefficient in natural streams by using ANFIS. Expert Syst Appl 2009;36(4):8589-96.

19. Carter EM, Potts HW. Predicting length of stay from an electronic patient record system: a primary total knee replacement example. BMC Med Inform Decis Mak 2014;14:26.

20. Sung SF, Hsieh CY, Kao Yang YH, Lin HJ, Chen CH, Chen YW, et al. Developing a stroke severity index based on administrative data was feasible using data mining techniques. J Clin Epidemiol 2015;68(11):1292- 300.

21. LaFaro RJ, Pothula S, Kubal KP, Inchiosa ME, Pothula VM, Yuan SC, et al. Neural network prediction of ICU length of stay following cardiac surgery based on pre- incision variables. PLoS One 2015;10(12):e0145395.

22. Sharma SK. Creating knowledge-based healthcare organizations. Hershey (PA): IGI Global; 2005.

23. Atoui R, Ma F, Langlois Y, Morin JF. Risk factors for prolonged stay in the intensive care unit and on the ward after cardiac surgery. J Card Surg 2008;23(2):99- 106.

24. Gomes C, Almada-Lobo B, Borges J, Soares C. Integrat- ing data mining and optimization techniques on surgery scheduling. In: Zhou S, Zhang S, Karypis G, editors.

International conference on advanced data mining and applications. Heidelberg: Springer; 2012. p. 589-602.

25. Abrahamyan L, Demirchyan A, Thompson ME, Hovaguimian H. Determinants of morbidity and intensive care unit stay after coronary surgery. Asian Cardio- vasc Thorac Ann 2006;14(2):114-8.

26. Tu JV, Mazer CD, Levinton C, Armstrong PW, Naylor CD. A predictive index for length of stay in the intensive

(9)

care unit following cardiac surgery. CMAJ 1994;151(2):

177-85.

27. Tu JV, Mazer CD. Can clinicians predict ICU length of stay following cardiac surgery? Can J Anaesth 1996;

43(8):789-94.

28. Wong DT, Cheng DC, Kustra R, Tibshirani R, Karski J, Carroll-Munro J, et al. Risk factors of delayed extuba- tion, prolonged length of stay in the intensive care unit, and mortality in patients undergoing coronary artery bypass graft with fast-track cardiac anesthesia: a new cardiac risk score. Anesthesiology 1999;91(4):936-44.

29. Bashour CA, Yared JP, Ryan TA, Rady MY, Mascha E, Leventhal MJ, et al. Long-term survival and functional capacity in cardiac surgery patients after prolonged intensive care. Crit Care Med 2000;28(12):3847-53.

30. Bucerius J, Gummert JF, Walther T, Doll N, Falk V, Schmitt DV, et al. Predictors of prolonged ICU stay after on-pump versus off-pump coronary artery bypass grafting. Intensive Care Med 2004;30(1):88-95.

31. Ghotkar SV, Grayson AD, Fabri BM, Dihmis WC, Pul- lan DM. Preoperative calculation of risk for prolonged intensive care unit stay following coronary artery bypass grafting. J Cardiothorac Surg 2006;1:14.

32. Xu J, Ge Y, Hu S, Song Y, Sun H, Liu P. A simple predictive model of prolonged intensive care unit stay after surgery for acquired heart valve disease. J Heart Valve

Dis 2007;16(2):109-15.

33. Janssen DP, Noyez L, Wouters C, Brouwer RM. Preop- erative prediction of prolonged stay in the intensive care unit for coronary bypass surgery. Eur J Cardiothorac Surg 2004;25(2):203-7.

34. Hein OV, Birnbaum J, Wernecke K, England M, Konertz W, Spies C. Prolonged intensive care unit stay in cardiac surgery: risk factors and long-term-survival. Ann Tho- rac Surg 2006;81(3):880-5.

35. Christakis GT, Fremes SE, Naylor CD, Chen E, Rao V, Goldman BS. Impact of preoperative risk and periopera- tive morbidity on ICU stay following coronary bypass surgery. Cardiovasc Surg 1996;4(1):29-35.

36. Rosenfeld R, Smith JM, Woods SE, Engel AM. Predic- tors and outcomes of extended intensive care unit length of stay in patients undergoing coronary artery bypass graft surgery. J Card Surg 2006;21(2):146-50.

37. Belderrar A, Hazzab A. Hierarchical genetic algorithm and fuzzy radial basis function networks for factors influencing hospital length of stay outliers. Healthc In- form Res 2017;23(3):226-32.

38. Kwon YD, Yoon SS, Chang H. Knowledge extraction from hospital information system: using the integrated database of electronic medical record and patient management system. J Korean Soc Med Inform 2007;13(2):

115-22.