Participants

The Participants was 622 nurses who experienced patient safety incidents within the last 1 year were recruited from general or tertiary general hospitals nationwide

1.1 Sample size estimation

Following the study finding that the minimum required sample size for each independent variable in a multiple regression analysis is 10 participants (Lee et al., 2006), the minimum sample size for the 27 predictors in this study was calculated to be 270 participants.

1.2 Sampling Method

In this study, data collection was conducted through an online panel survey.

Nurscape (www.nurcape.net), an exclusive online community for nurses, provided access to 20,000 nurses currently working at general hospitals and tertiary general hospitals nationwide. Considering the 14-30% chance among health care providers of experiencing patient safety incidents (Scott et al., 2009), 2,807 participants were expected to meet the conditions of this study.

The sample size of the study was defined to include all 20,000 participants, considering the following three conditions: 1) prior studies conducted on the online survey platform SurveyMonkey have reported a response rate of 23.6% (An & Kim, 2013) high bias rates due to unreliable responses have been reported among online surveys. 3) some participants who experienced patient safety incidents may have reservations about sharing their personal experiences.

During data collection, the online URL link to the survey was sent to 20,000 target participants through E-mails and mobile messages. Among them, 1,222 accessed the link to the survey (16.4% response rate). 332 (3.7%) were eliminated due to incompletion, 214 (4.2%) did not meet the inclusion criteria, and data were collected from the remaining 676 participants. 54 outliers, inaccurate data and missing data were eliminated before 622 were finally examined (Figure 4).

Figure 4. Flowchart of study participants

2. Measurement Tools 2.1 Measure

2.1.1 Socio-Demographic Variables

Socio-demographic variables considered in this study were gender, age, marital status, education level, hospital work experience, work unit, turnover experience, and hospital type.

2.1.2 Variable related to Patient Safety Incident

Patient safety incident related variables examined in this study included types of patient safety incidents - medication errors, falls/slips, suicides or suicide attempts or self-harm, extravasation or phlebitis, examination-related incidents, facility or instrument related incidents, and accidental catheter removal , transfusion related incidents - severity of incidents, and duration of physical and psychological distress experienced after patient safety incidents.

2.1.3 Just Culture

The Just Culture Assessment Tool (JCAT)developed by Petschonek et al. (2013) was used to measure just culture. After receiving the approval of its original developer, the tool was translated into Korean and reviewed for reliability and validity before use.

The sub-variables of each item consisted of 27 items including 3items on feedback and communication, 5 items on openness of communication, 5 items on balance, 5items on

trust, 5items on quality of reporting process, and 4items on continuous improvement.

Every item was scored on a five-point Likert scale ranging from ‘Strongly disagree’ with 1 point to ‘Strongly agree’ with 5 points. A high score indicates a high degree of just culture in the responder’s organization. Reverse-scored items were included in the survey.

In the original tool development, Cronbach’s alpha reliability for the sub-variables were .74 for feedback and communication, .86 for openness of communication, .78 for balance, .63 for quality of event reporting process, .78 for continuous improvement, and .75 for trust. In this study, Cronbach’s alpha reliability were .65 for feedback and communication .83 for openness of communication, .63 for balance, .54 for quality of event reporting process, .86 for continuous improvement, and .71 for trust.

2.1.4 Resonant Leadership

Resonant leadership was measured by the Resonant Leadership Scale developed by Cummings et al. (2013) including 10 items. All items were scored on a five-point Likert scale ranging from ‘Strongly disagree’ with 1 point to ‘Strongly agree’ with 5 points. A high score indicates a high level of resonant leadership. Cronbach’s alpha reliability in the original study was .94 and .92 in this study.

2.1.5 Organizational Support

Organizational support was measured by the Second Victim Experience and

Support Tool (SVEST) developed by Jonathan et al (2014) including 4items on colleague support, 4items on supervisor support, 3 items on institutional support. All items were scored on a five-point Likert scale ranging from ‘Strongly disagree’ with 1 point to

‘Strongly agree’ with 5 points, and reverse-scored items were included. A high score indicates that a high level of organizational support is perceived by the responder. All items were compared on an average-score basis. Cronbach’s alpha reliability presented in Jonathan et al. were .61 for colleague support, .88 for supervisor support, .64 for institutional support. In this study, Cronbach’s alpha coefficient for organizational support was .60 -.75.

2.1.6 Employee Health

Employee health was measured by 12 items from the SVEST developed by Jonathan et al (2014) including 4items on psychological distress, 4 items on physical distress, 4items on professional self-efficacy. All items were scored on a five-point Likert scale ranging from ‘Strongly disagree’ with 1 point to ‘Strongly agree’ with 5 points, and reverse-scored items were included. A high score indicates a low level of employee health. Each item was compared on an average-score basis. Cronbach’s alpha reliability presented in Jonathan et al. (2014) were .83 for psychological distress, .87 for physical distress, and .79 for professional self-efficacy. In this study, Cronbach’s alpha coefficient for employee health was .70-.80.

2.1.7 Organizational Health

Organizational health was measured by 2 items on turnover intention and 1 on absenteeism from the Second Victim Experience and Support Tool (SVEST) developed by Jonathan et al (2014) . Every item was scored on a five-point Likert scale ranging from

‘Strongly disagree’ with 1 point to ‘Strongly agree’ with 5 points. Reverse-scored items were included. A high score indicates a low level of organizational health in the organization. The items were compared on an average-score basis. Cronbach’s alpha reliability presented in Jonathan et al (2014) were .81 for turnover intention and .88 for absenteeism. In this study, Cronbach’s alpha for organizational health measured by 2 turnover intention items and 1 absenteeism item was .80.

2.1.8 Quality of Work Life

Quality of work life was measured by the Work-Related Quality of Life (WRQoL) Scale developed by Easton and Van Lar (2013). The use of the tool was approved by its original developer before translating it into Korean. The sub-variables of each item consisted of 23 items including 6items on job and career satisfaction (JCS), 5items on general well-being (GWB), 2 items on stress at work (SAW), 3 items on control at work (CAW), 3 items on home-work interface (HWI), and 3items on working conditions (WCS), and overall satisfaction item1 .All items were scored on a five-point Likert scale ranging from ‘Strongly disagree’ with 1 point to ‘Strongly agree’ with 5 points, and a high average score indicated a high quality of work life. Cronbach’s alpha

reliability presented in Easton and Van Larr (2013) were .86 for JCS, .81 for CAW, .75 for HWI, .81 for SAW, .84 for GWB, and .75 for WCS. In this study, Cronbach’s alpha coefficients were .91 for quality of work life and .73-.84 for sub-variables (Table 4).

2.2 Translation and Development of Tools 2.2.1 Translation

Appropriate tools were selected based on findings from the Previous studies. For assessing just culture, the Just Culture Assessment Tool (JCAT) developed in Petschonek et al. (2013) was used. For assessing resonant leadership, the Resonant Leadership Scale developed in Cummings et al. (2013) was used. For assessing work-related quality of life, the Work-Related Quality of Life Scale developed in Easton and Van Lar. (2013) was used. The permission to use the tools was approved in writing by its original author. For accurate results, each tool was translated jointly by a committee of three individuals—the researcher, a nursing professor with proficiency in English and Korean, and a nurse with proficiency in English. First drafts of translated tools were reviewed for cultural relevance with the help of two nursing professors who are bilingual in English and Korean. In the JCAT, for example, “event” was modified as “patient safety incident,” “management” as

“our hospital’s management,” and “follow up team”as “patient safety team.” Also, “tattle”

was modified as “assign blame” and “my supervisor” as“the supervisor in my work unit.”

In translating the Resonant Leadership Scale, the expression “the leader in my clinical program or unit” was modified as “the supervisor in my work unit” and “feedback” as

“pay attention to my updates and give feedback.” In the WRQoL, “work” was modified as “workplace” and “facility” as “work environment.” Also, “new skill” was modified as

“new work experience.”

2.2.2 Content Validity Assessment 1) Expert panel review

Content validity of the translated JCAT, Resonant Leadership Scale, and WRQoL was assessed by an expert group to determine whether each of them is suitable for assessment of just culture, resonant leadership, and work-related quality of life (Appendix. Expert Assessment Survey).

The expert group assessing the Resonant Leadership and WRQoL consisted of 7 experts including two doctoral degree head nurses in clinical practice, two nurses pursuing a doctoral degree, and three nursing professors. The expert group assessing the JCAT consisted of 9 individuals including two doctors, a QI team leader with over ten years of patient safety experience, a patient safety expert pursing a doctoral degree, three nursing professors, and two doctoral degree head nurses working in hospitals.

The Item-level Content Validity Index (I-CVI) and the Scale-level Content Validity Index (S-CVI) were used in assessing the content validity of the tools. The validity of each tool was evaluated on a five-point scale - 1 point for “strongly disagree,”

2 points for “disagree,” 3 points for “neutral,” 4 points for “agree,” and 5 points for

“strongly agree”. Suggestions for revision or addition of information in the survey were

taken in writing. The survey was modified and finalized based on expert inputs. The I-CVI was determined by the ratio of experts who gave a score of 3 or 4. In this study, the I-CVI for the JCAT was 0.98, 0.97 for the WRQoL, and 0.96 for the Resonant Leadership Scale. The S-CVI was determined by the ratio of items that received a score of 3 or 4. In this study, the S-CVI for the JCAT was 0.98, 0.96 for the WRQoL, and 0.96 for the Resonant Leadership (Table 2).

Among the WRQoL items, the fourth item “I feel well at the moment” received a score of 2 from 3 experts, and its average I-CVI was only 0.57. Following the expert opinion, the item was removed from the survey.

Table 2. Content Validity Index of the measurements

Exp1 Exp2 Exp3 Exp4 Exp5 Exp6 Exp7 Exp 8 Exp9 JCAT 0.96 0.89 1.00 1.00 1.00 1.00 1.00 1.00 1.00

WRQoL 1.00 0.87 1.00 0.96 1.00 0.96 0.96 - -

Resonant

Leadership 1.00 1.00 1.00 0.96 0.80 1.00 1.00 - -

Mean I-CVI=JCAT 0.98 , WRQoL 0.97, Resonant Leadership 0.96 S-CVI/UA= JCAT 0.98 , WRQoL 0.96 , Resonant Leadership 0.96

JCAT, Just Culture Assessment Tool; WRQoL, Work-Related Quality of Life; I-CVI, item-level content validity index; S-CVI, scale-level content validity index, universal agreement calculation method

2) Cognitive Interviewing

Cognitive interviews with nurses were conducted to closely examine the adequacy of each tool by evaluating whether the tool developer’s purpose was sufficiently addressed in the way participants understood the translated items. One-to-one, in-depth interviews with the participants were conducted based on their answers to the survey to examine whether they understood each item in the way that the researcher had intended, whether they were confused or uncertain about the its meaning, and whether the item was phrased in a way that allowed them to answer without difficulties. In this stage, every word and sentence included in the survey was reviewed based on findings from the cognitive interviews (Jang, 2014). Five nurses each participated in an interview with the surveyor and the researcher and examined the items one-by-one while paying attention to the terms and semantics. As a result of the cognitive interviews, several modifications were made in the JCAT item “the supervisors respect employees’ suggestions”— “the supervisors” was modified as “the supervisor in my work unit” and “suggestions” as

“suggestions and ideas” to reflect the comments that the original word might imply only formal communications. Also, “uses a balanced and fair system for appraising employees” was modified as “follows fair procedures in appraising employees” based on the input that the original phrase might be obscure to nurses. “I am uncomfortable with others entering reports about events in which I was involved” was removed from the survey after reviewing reliability to reflect the comment that the item had little relevance in the context of the Korean work environment. In the Resonant Leadership Scale,

“Looks for feedback even when it is difficult to hear” was commented as obscure and modified as “The supervisor in my work unit pays attention to my updates even when it is difficult to hear and gives feedback.” In addition, “Acts on values even if it is at a personal cost” was commented as obscure, so it was modified as “The supervisor in my work unit takes action for the good of the unit (or team) even if it has personal consequences.” The nurses who assessed the WRQoL items in cognitive interviews did not report any difficulty in understanding and answering the items, and it was concluded that that they were appropriately worded.

3) Pilot- study

The survey used in the pilot-study included the following: JCAT 27 items, 10 Resonant Leadership items, 23 WRQoL items, grammar and context of which were all reviewed by scholars of Korean literature after modifications were made based on expert validity assessment and cognitive interviews, and 28 SVEST (Second Victim Experience and Support Tool) items that assess physical and psychological distress and organizational support after patient safety incidents, and items regarding general characteristics. The participants of the pilot-study were 30 nurses working in general hospitals who experienced patient safety incidents within the last one year.

2.2.3 Verification of the Tool Reliability 1) Test–retest reliability

Test-retest reliability was measured to assess consistency of each tool. The 30 participants of the pre-study were asked to take the survey again after 2 weeks, and 28 of them agreed. Data collection for the pre- study was conducted from February 22 to March 8, 2020 Using the criteria of test-retest correlation coefficient is higher than .70 (DeVon et al, 2007). The intra-class correlation coefficient for the JCAT was .81 (95% CI 0.60-0.91), 0.84 (95% CI 0.65-0.93) for the Resonant Leadership Scale, and 0.77 (95% CI

JCAT 3.49(0.35) 3.43(0.33) 0.81(0.60~0.91) <0.001

Resonant leadership 3.73(0.47) 3.59(0.47) 0.84 0.65~0.93) <0.001

WRQoL 3.11(0.37) 3.00(0.47) 0.77(0.49~0.89) <0.001

JCAT, Just Culture Assessment Tool; WRQoL, Work-Related Quality of Life; CI, confidence interval.

2) Internal consistency reliability

To assess the reliability of the tools, the internal consistency within each tool and each of its sub-factors were measured. In general, a Cronbach’ alpha coefficient, which measures internal consistency, larger than .60 indicates reliability (Kellar & Kelvin, 2013). In this study, the Cronbach’s alpha of the JCAT was .90, Resonant Leadership .92, and WRQoL 0.91. Within the JCAT, feedback and communication showed a Cronbach’s alpha of .65, openness of communication .83, continuous improvement .86, Trust .71. and balance .63. Despite the relatively poor internal consistency of balance, this dimension is considered theoretically essential to the understanding of just culture and warrants inclusion in the JCAT.

The internal consistency reliability coefficient of the sub-variable ‘quality of event reporting process’ was .48, but it was re-evaluated as .54 after removing the item

“I’m given time to enter event reports during work hours.” ‘Quality of event reporting process’ is related to the quality of the reporting system, which includes the accessibility and convenience of reporting. In detail, the sub-variable represents the level of perception among individual workers toward the reporting system by assessing whether they are given appropriate time to report errors or incidents and whether the reports are being appropriately monitored. But, the item “I’m given time to enter event reports during work hours” was deemed an overly general and one-dimensional question revealing little more than whether patient safety incidents were reported. It was considered inappropriate for assessing the quality of the reporting process, so it was removed (Table 4).

Table 4. Reliability of the measurements interface; SAW, Stress at work; CAW, Control at work; WCS, Working conditions

3. Data Collection

Data collection for this study was conducted through online surveys after the Institutional Review Board of the CHAMC (IRB No. 2019-11-060-003). Data were collected from 10 to March 18, 2020.

For the survey panel, nurses working in general hospitals and tertiary general hospitals were recruited through Nurscape, an online membership community for nursing professionals. The purpose and methods of the study were disclosed to the authorized staff members of Nurscape before their approval for conducting the survey. Afterward, emails containing a recruitment notice and the web address for the survey questionnaire (ko.surveymonkey.com/r/3FH6FH8) were sent to the panel members.

The survey questionnaire was developed using SurveyMonkey, an online survey tool, and distributed to a internet panel at Nurscape (http://www.nurscape.net). This online community is exclusive to registered nurses, and its membership is authorized to those who can provide a nurse license or certificate. Submitted documents are checked for forgery or alteration, and license or certificate numbers are validated by the Ministry of Health and Welfare before the administrator approves membership. Names, cell phone numbers, and IP addresses are verified to prevent duplicate subscriptions.

Among the community members, 20,000 currently working at general hospitals or tertiary general hospitals were selected as target participants of this study. They received recruitment notices through e-mails disclosing: information on the study including its title, purpose, and scope; the length of the survey; and a reward

(10,000giftcone) for participation. They also received through e-mails or SMS messages the online URL link to information such as the description of the study, its purpose and method, risks and benefits, terms of survey participation, withdrawal of participation, guarantee of anonymity, and restricted use of survey data to research purposes. Only after having consented to participate in the study by checking the “I Agree” button, the participants were prompted to start the survey. After the completion of the survey, the participants were asked to enter their phone numbers, which were used for self-identification and rewarding Gifticons equivalent to 10,000 Won and this personal information was destroyed shortly after. Additionally, the survey template was programmed to prevent duplicate participation, in which case access to the survey was restricted with a message that the link had expired. The length of the survey was about 15 minutes. To those who completed the survey.

4. Data Analysis

Data analysis in this study was conducted using SPSS 26.0 . The detailed steps of analysis are as follows.

1) General characteristics of the participants were analyzed through descriptive statistics including real numbers, percentiles, averages, and standard deviations.

2) Employee health, organizational health, organizational support, and quality of work life according to the participants’ general characteristics were analyzed using t-test and ANOVA. Significant variables from the ANOVA results were further examined by the Scheffé test.

3) Pearson’s correlation coefficient were analyzed was used to identify the relationship amongst variables

4) The multiple regression method was used to identify factors affecting quality of work life, employee health, and organizational health, and the multicollinearity of measured variables was detected by examining tolerances and variation inflation factors (VIFs).

5) The validity and reliability of the instruments used was verified using the I-CVI (Item-level Content Validity Index) and the S-CVI (Scale-(Item-level Content Validity Index), and their internal consistency was verified by examining Cronbach’s alpha coefficients.

C. Qualitative Study

Suitable participants of the qualitative study were selected before their personal experiences of patient safety incidents were investigated in in-depth interviews to find meaningful insights, with the use of semi-structured questionnaire.

1. Participants

Participants of the qualitative study were practicing nurses who had experienced a patient safety incident within the past 1 year, were willing to disclose personal experiences and symptoms related to patient safety incidents, and had agreed to participate in the study after being informed of its purpose and objectives. 8 participants were selected through purposive sampling and snowball sampling—they were recruited from among the acquaintances of either the researcher or existing study participants.

For data saturation, data were collected through in-depth interviews with the participants

For data saturation, data were collected through in-depth interviews with the participants