Quality in Primary Care Open Access

Keywords

Team training; Human factors; Crew Resource Management; Health care quality improvement; Teams

Introduction

Health care is increasingly becoming a complex endeavor, requiring expertise and adequate actions from all health care professionals including various medicals specialists. In order to achieve a high standard of patient care multidisciplinary and inter professional teams are formed every day in both routine and emergency settings.

Even with exceptionally high level of individual expertise of team members, team activities may be compromised because of ineffective teamwork. Such a situation is known to be vulnerable for medical errors. Since the publication of the NIH report “To err is human” in 2000 increased awareness has developed [1]. In the US, additional medical costs due to these errors are estimated as high as $17 billion per year [2].

Analysis of errors in health care reveals that a major negative contribution is made by ineffective “non-technical skills’ amongst health professionals like communication, situational awareness, decision making, leadership and assertiveness. In the Netherlands, this analysis revealed that 80% of errors in a surgical setting could be attributed to various forms of these human factors [3].

Creating and maintaining high standards of team performances is therefore mandatory and requires many competencies. Essential quality competencies of health care workers are summarized in the “medical expert” framework by the Royal College of Physicians and Surgeons of Canada (CanMEDS Competencies) [4,5]. Raising awareness about the influence of these competencies and training them may improve the performances of teams in daily patient care. Strategies focusing on reducing errors, need to be developed and implemented [6-8].

At different levels reducing human error could be tackled: a) At the system-design level: Reducing errors by reducing error-triggering situations; e.g. improved displays and alarm philosophies, b) at the process level establishing measures and protocols that shield patients and health-care professionals from errors that may occur; e.g. improved medication labels, double-checking of procedures, c) education and training constitute another level to help health-care professionals recognize, avoid and recover from errors when they occurred [9].

Around 2010 various simulation team training projects were initiated by physicians of various departments in VU University Medical Center, Amsterdam (VUmc) to improve team performance of resuscitation, trauma, ICU and maternity teams, since it is known that team training improves patient outcome [6]. This project was based on voluntary participation. Because of its voluntary origin, mainly residents and nurses attended the training sessions. After a high impact incident in patient care in our hospital, the Board of Directors and Medical Staff Board stated that non-technical skills have a substantially high impact on Patient Safety and need to be trained for all health care professionals including all medical specialists. An advisory board (with participation of the Patient Council) was consulted in the process of designing this program. Finally it was chosen to implement a training system, designed by healthcare professionals for healthcare professionals.

TeAMS (Training and Assessment of Medical Specialists) started in 2014 as a pilot project to investigate whether it was possible and feasible to start a hospital-wide program. In 2015, after completion of a successful pilot, the Board of Directors of VUmc decided that the program should be mandatory for all medical specialists working in the hospital, as they are the key players in most of the multidisciplinary-interprofessional teams.

Goal of the Program

The long term goal of the TeAMS program of VUmc is to improve non-technical skills necessary for working in multidisciplinary and interprofessional teams, like leadership, decision-making, situational awareness and communication. Improving team performances is expected to improve the level of expertise, efficiency and safety of patient care [6,10,11]. Subsequently this program strives to support a change in hospital culture where addressing difficult issues becomes normal for team members in all settings.

The short term goal was to implement the TeAMS program effectively and ensure high quality training by following a Plan-Do-Check-Act cycle. In order to meet this goal concrete criteria were specified:

• Train at least 80% of the medical specialists( i.e., physicians that completed a specialty training)at VUmc

• The participants should rate the training at least 7.5 on a 10 point scale.

• Investigate the logistical elements in implementing a hospital-wide program.

In this paper we report the outcome and challenges of the first two years of running this program in an academic center.

Method, What We Did

Approximately 400 medical specialists work in our hospital. Based on experience in former team training settings, we wanted at least 4 medical specialists from 3 different disciplines to participate in a training to optimize interaction. The other trainees would be the health professionals that in daily practise would be part of the team. We therefore scheduled 70 training sessions for: 25 acute situations, 20 complex situations and 25 regular situations per year.

Theoretical background and basic principles of the training

The attendants of the training would mainly be skilled professionals. Therefore, in designing the training, we applied theoretical concepts for adult-learning. A professional needs to see the relevance of the subject that is presented before he starts learning [12]. Realizing that a lack in competence leads to sub-optimal functioning, is known to be a powerful driver for learning [13]. This need to learn nearly always stems from experience in daily practice [14]. These concepts were chosen with a group of psychologist, sociologists, medical practitioners and team trainers, all with experience in behavioral teaching, in order to achieve a high learning efficiency.

In designing this training program we therefore decided to create authentic and relevant settings for authentic teams leading to the following main principles:

• Train the teams that actually work together

• Train only clinical situations that are relevant for those teams (by defining specific learning objectives for specific teams based on interviews of all participating departments for identifying their training needs).

• Train skills that can be applied immediately.

Participants of the training

The target group consisted of all medical specialists working in VUmc (n=400) and training was mandatory. Following our main principles other healthcare professionals would participate in the training as well (residents, nurses, etc.). We expected that half of the participants would be medical specialists and the other half would be other healthcare professionals.

Design of the training

Since the TeAMS-program of VUmc aimed to improve non-technical skills necessary for working in multidisciplinaryinterprofessional teams, we mainly focused on communication, teamwork and professionalism. The various training programs were designed by local medical specialists and educational and behavioral specialists using the above-mentioned the basic principles.

Every training module consists of four elements: 1) Education on team-processes; 2) practice 3) analysis of and reflection on clinical scenarios and 4) assessment for performance (formative). The theoretical models that were used as basis for the design were Kolb’s learning cycle and Crew Resource Management (CRM) principles [9,15,16].

To be able to reach all medical specialists that are involved in patient care, 3 different settings were defined in which medical teams work together: a) Acute situations, b) complex situations and c) multidisciplinary patient meetings.

Specific training modalities were developed for these 3 situations:

• Simulation training of acute clinical situations (SIM)

• Training of complex situations (CST)

• Observation training of multidisciplinaryinterprofessional patient meetings (MDM)

Each training session lasted for a maximum of 4 h. Participants had to come from at least three different disciplines and at least 4 medical specialists had to attend.

All different medical staffs in the hospital were contacted by the program organization to identify possible training needs in acute, complex or regular situations.

Trainers

All training sessions were facilitated by two trainers. For the acute situation training and the complex situation training, at least one of the trainers was a medical specialist. The other trainer was either a psychologist or a behavioral specialist with special expertise in addressing and teaching non-technical skills.

Organizational embedding

To ensure a lasting structure, the program was firmly implemented in the organization of the hospital (Figure 1). The Board of Directors had the final responsibility of the program. Furthermore, they decided that participation of medical specialists was a key performance indicator, making it a mandatory training for the medical specialists.

Figure 1: Organizational embedding of the program. Acute situation training (SIM), complex situation training (CST), regular situation training (MDM).

The program organization consisted of a medical program leader (medical specialist) together with an organizational program leader. They chaired the program committee, responsible for organization, finances and quality and were held accountable for the results by the Board of Directors and the Medical Staff Board. The program committee consisted of stakeholders and external advisors, experts in training and patient safety. Every training modality had a medical specialist as program coordinator and a case manager. Together they were responsible for the content and the logistics of the trainings.

Quality assurance of the training

A robust system, using a Plan-Do-Check-Act cycle, was created to assure a high quality of the TeAMS program in VUmc [17].

To organize the program, first a pilot was conducted to evaluate the three different training modalities. The content and logistics were adjusted based on the feedback of the participants. This resulted in the above-described design.

From 2015 the program was run to meet the goals as were set at the start of the program. Data collected included the number of participants, disciplines and professions.

To be able to adjust and improve the trainings, all participants were asked to fill out a post-training questionnaire. We focused on the first 2 levels of Kirkpatrick’s model for evaluation of the training interventions [18]. The topics of the questionnaire are listed in Table 1. Every trimester the program committee met with the program coordinators and case managers. The result of the evaluations, fact, figures and finances, quality of the trainers, publicity and other subjects were discussed and if necessary, adjustments were made.

Element	Questions
Content of training	Was the goal of the training clear? Was the training form adequate? Were the various components of the training instructive?
Trainers	Was the training guidance agreeable? Was the training environment safe? Was the trainer capable? Was the feedback useful?
Training form	Was the level of the training sufficient? Was the pace of the training sufficient? Was the duration of the training sufficient? Did it simulate daily practice?
Effect of training	Did you acquire new knowledge and understanding concerning collaboration? Did you acquire new knowledge and understanding concerning personal style? Did you acquire new knowledge and understanding concerning collaboration styles? Will you implement this knowledge in daily practice?
Logistics	Were the organization, location and catering sufficient?
Rating	Please rate the training on a scale from 0-10 Would you recommend this training to your colleague?
Suggestions or remarks

Table 1: Evaluation questionnaire (translated from Dutch).

In addition we defined a “faculty development and maintenance” program to ensure a high level of long-lasting expertise. This program included a follow-up refresher training day and a feedback and reflection session of at least one video-taped debriefing for all instructors (once a year). Also workshops on specific themes were organized [19].

Results and Evaluation

Number of training sessions

In 2015 19 SIM-training sessions, 15 CST-training sessions and seven MDM-training sessions were realized. In 2016, 19 SIM-training sessions, seven CST-training sessions and eight MDM-training sessions respectively, meaning we achieved to organize approximately 60% of our goal.

Participants

Thirty-seven different medical disciplines and sub-disciplines participated in one of the training sessions, implying that the TeAMS program was indeed hospital broad.

A total of 182 (45%) medical specialists and 155 other health professionals participated in one of the trainings in 2015 and in 2016 122 (30%) medical specialists and 134 other health professionals. Numbers and professions are listed in Table 2.

	SIM	CST	MDM	Total
Medical Specialist (n: 2015/2016)	63/57	79/29	40/36	182/122
Resident (n: 2015/2016)	44/52	38/19	6/3	88/74
Doctor (non-resident) (n: 2015/2016)	4/4	0/0	0/0	4/4
Nurse (n: 2015/2016)	21/28	6/4	2/4	29/36
Anaesthesia nurses (n: 2015/2016)	12/5	0/0	0/0	12/5
Other professions (n: 2015/2016)	7/10	2/3	1/2	10/15
Unknown (n: 2015/2016)	7/0	5/0	0/0	12/0
Total (n: 2015/2016)	158/156	130/55	49/45	337/256

SIM: Acute Situation Training (simulation training); CST: Complex Situation Training; MDM: Multidisciplinary Meeting (regular situation training)

Table 2: Demographic characteristics of the participants.

As expected, half of the participants were medical specialists. The relative low participation of nurses and other health professionals was because the design of the training was focused on the medical specialists. Less than 5 persons participated twice in one year. In both 2015 and 2016 we didn’t meet our goal to train 80% of the medical specialists.

Reasons for not achieving the goal were:

• Inability to organize a moment where all participants as well as trainers were available for the training;

• Cancellation by individual participants for various reasons. If too many cancelations occurred or if the cancelation was related to a core member of the team, the training session had to be rescheduled or cancelled;

• Inability to participate, because patient care, other obligations or mandatory issues prevailed;

• In the beginning 2016 an electronic patient record system was introduced. Because of the impact this would give on the hospital, it was decided only to organize training sessions the second half of the year.

• Heads of departments didn’t realize that their staff members had to participate yearly.

Training evaluation and quality assurance

Two hundred and thirty six participants filled in the evaluation questionnaires (70% response rate) in 2015. To improve the response rate, trainers were asked to hand out and collect the questionnaires at the end of a training session, leading to a response of 224 (87%) participants in 2016. All 3 training types were appreciated by nearly all participants (Figure 2). The mean rating of the trainings was above 7.5. The SIM training was appreciated the most.

Figure 2: Percentage of participants filling out the questionnaire, with “yes” or “no” to the question: “Would you recommend this training to a colleague” for the acute situation training (SIM), the complex situation training (CST) and the regular situation training (MDM).

Program Committee meetings took place 4 times a year. Analysing the evaluation forms during the meetings, led to adjustments of the trainings. This led to an improvement in the rating of the training with a mean >8 in 2016 (Figure 3).

Figure 3: Number of participants and their rating of the training with a number between 1 and 10, overall and for of the acute situation training (SIM), the complex situation training (CST) and the regular situation training (MDM).

Needs assessment and feedback of the participants, also led to development of a variant of the MDM training: A multidisciplinary outpatient consultation session, with patients present during consultation.

Trainers

Twenty three trainers were selected for the program (11 medical specialists). Seven intervision and evaluation meetings have taken place. Trainers reflected on their personal evaluation by participants, feedback of colleague’s and personal trainer skills.

Three workshops have been organized for the trainers, addressing encountered themes like resistance, team dynamics and debriefing.

Organizational embedding

The leaders of the TeAMS program met with the Board of Directors and Medical Staff Board once a year to give account of the various aspects of the program in an annual report. Possible actions on the strategic level including finances were discussed.

The cost of running the program was approximately €250,000-Euros. This included the overhead costs of the organization and the logistics, the fee of the trainers, the faculty development program and the maintenance of the quality system.

Discussion and Lessons Learned

This hospital based team-training program for all medical specialists to the best of our knowledge, is novel. It is also unique because it is mandatory for all medical specialists involved in patient care. Acute situation training using simulation is a known and established form of team-training. The TeAMS program also uses other modalities to train complex situations and regular situations. These two training modalities for medical teams in hospitals are new.

Our results indicate that the implementation of the program can be regarded as successful. Various factors contributed to this success. The first factor was the use of authentic training scenarios for authentic teams. Second, the use of a training form, based on audit and feedback and CRM principles [20,21] the third factor was the expertise of the trainers. Selection, training and maintenance of expertise of all trainers (faculty development program) had a high priority within the TeAMS program [19].

At the strategic level, the Board of Directors rated the training as a key performance indicator. This underlined the importance of the program and helped to overcome a part of the reluctance of the specialists to participate.

The establishment of a Program Committee, chaired by a medical specialist, with various stakeholders, contributed to the quality, finances and organization, at a tactical level. At the operational level, every training was coordinated by a medical specialist together with a case manager. This combination provided for adequate logistics, ambassadorship, structured training evaluation and customization of training scenarios.

With this organization structure the TeAMS program was run by medical staff and focused on medical staff. This hospital-wide approach directed at all hierarchical levels of medical staff, along with a good to excellent rating of the training, formed a good basis for a sustainable effect [22].

However, quantifying the effect on patientcare at a hospital level remains difficult [23].

As expected several challenges were encountered at different levels in the organization [24]. One of the factors was the reluctance of the staff to participate. This was noticed in the remarks the coordinators and case managers involved in the program, received during the year. The fact that participation in the program was a key performance indicator helped to overcome this reluctance.

Factors considering logistics, planning and organization played an important role in not achieving the goal of the total number of trainings. There was tension between daily clinical tasks, patient care and presence off all (10-15) team members at the training. Long term planning seemed to propose a solution. Three to six months advance notice seemed to be acceptable and gave clinicians a better opportunity to arrange their clinical tasks.

In order to diminish cancellation, we organized a system for reminding the participants that they were expected at training. They received three e-mails at various points in time before the training session. If they were unable to attend, there was still time to find another participant.

Two consecutive years we were not able to reach our goal of training 80% of the medical specialists. The Board of Directors will address all Heads of Departments to stress the key performance indicator. We promoted the program by updating the website, emailing and publishing our results in hospital website and news.

In order to customize the clinical scenarios, the medical program coordinators conducted personal interviews with the participants. Based on the interviews, new clinical scenarios were created and existing scenarios were adjusted. This was very time consuming.

Because of logistic difficulties the cost of this program was relatively high. If all training sessions do take place, we expect to train 900 health professionals per year. The cost will than decrease to approximately €275 per participant per training.

Conclusion

The TeAMS program of VUmc, is a unique multidisciplinary-interprofessional team training program for non-technical skills, mandatory for all medical specialists. The program reached all different medical specialisms, throughout the hospital. Embedding the program in the organizational structure of the hospital assured stability and continuity. A robust system of quality maintenance helped in achieving these results. The 3 training-types that have been designed used relevant clinical settings and scenarios. This tailor-made approach was highly appreciated by the participants. Logistics and finances were challenging.

Funding

Running the program was possible by funding with the grant “regime quality-impulse” (subsidieregeling kwaliteitsimpuls)

Previous Presentation

Part of this work has been presented previously at the International Forum of Quality and Safety in Health Care, Gothenburg, 2016

Declaration

The departments of all authors except Rashmi Kusurkar receive reimbursement for the hours they work for the TeAMS program. Since this is not a study involving patients, no patient consent was necessary. All authors declare that this manuscript is not submitted for publication elsewhere.

References

1. Kohn LTC JM, Donaldson MS. To Err is human: Building a safer health care system. Institute of Medicine, Washington DC 2000.
2. Andel C, Davidow SL, Hollander M. The economics of health care quality and medical errors. J Health Care Finance 2012; 39: 39-50.
3. Wagner CSM, van Wagtendonk I, Zwaan L, Lubberding S, Merten H, et al. Oorzaken van incidenten en onbedoelde schade in ziekenhuizen. Een systematische analyse met PRISMA op afdelingen Spoedeisende Hulp (SEH), chirurgie en interne geneeskunde. EMGO Instituut en NIVEL 2008.
4. Wong BM, Ackroyd-Stolarz S, Bukowskyj M. The CanMEDS 2015 patient safety and quality improvement expert working group report. Ottawa 2014.
5. Frank JR. The CanMEDS 2005 physician competency framework: Better standards, better physicians, better care. The royal college of physicians and surgeons of Canada, Ottawa, Ontarion, Canada 2005.
6. Salas E, DiazGranados D, Klein C. Does team training improve team performance? A meta-analysis. Hum Factors 2008; 50: 903-933.
7. Neily J, Mills PD, Young-Xu Y. Association between implementation of a medical team training program and surgical mortality. JAMA 2010; 304: 1693-1700.
8. Stocker M, Pilgrim SB, Burmester M. Interprofessional team management in pediatric critical care: Some challenges and possible solutions. J Multidiscip Health 2016; 9: 47-58.
9. Ostergaard D, Dieckmann P, Lippert A. Simulation and CRM. Best Pract Res Clin Anaesthesiol 2011; 25: 239-249.
10. Zeltser MV, Nash DB. Approaching the evidence basis for aviation-derived teamwork training in medicine. Am J Med Qual 2010; 25: 13-23.
11. Clay-Williams R, Greenfield D, Stone J. On a wing and a prayer: An assessment of modularized crew resource management training for health care professionals. J Contin Educ Health Prof 2014; 34: 56-67.
12. Knowles MSHI EF, Swanson RA. The adult learner. Routledge, New York, USA 2015.
13. Hattie J. The applicability of visible learning to higher education. Scholarship of Teaching and Learning in Psychology 2015; 1: 79-91.
14. Lake FR, Ryan G. Teaching on the run tips 2: Educational guides for teaching in a clinical setting. Med J Aust 2004; 180: 527-528.
15. Kolb D. Experiental learning: Experience as the source of learning and development. Eaglewood Cliffs (NJ): Prentice-Hall Inc., 1984.
16. Helmreich RL, Merritt AC, Wilhelm JA. The evolution of crew resource management training in commercial aviation. Int J Aviat Psychol 1999; 9: 19-32.
17. Kilo CM. A framework for collaborative improvement: Lessons from the Institute for Healthcare Improvement's breakthrough series. Qual Manag Health Care 1998; 6: 1-13.
18. Kirkpatrick DL. Evaluating training programs: The four levels. Berrett-Koehler, San Francisco 1998.
19. Steinert Y, Mann K, Anderson B. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update. Med Teach 2016; 2016: 1-18.
20. Salas E, DiazGranados D, Weaver SJ. Does team training work? Principles for health care. Acad Emerg Med 2008; 15: 1002-1009.
21. Ivers N, Jamtvedt G, Flottorp S. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; 6: CD000259.
22. Shrader S, Hodgkins R, Laverentz D. Interprofessional education and practice guide no. 7: Development, implementation and evaluation of a large-scale required interprofessional education foundational programme. J Interpr of Care 2016; 2016: 1-5.
23. Clay-Williams R, Nosrati H, Cunningham FC. Do large-scale hospital- and system-wide interventions improve patient outcomes: A systematic review. BMC Health Serv Res 2014; 14: 369.
24. Grol R, Grimshaw J. From best evidence to best practice: Effective implementation of change in patients' care. Lancet 2003; 362: 1225-1230.