Quality in Primary Care Open Access

Keywords

general practice, quality assessment, quality improvement, qualitative research

Introduction

Quality improvement (QI) in general practice is defined as the combined and unceasing efforts of healthcare professionals, patients and their families, researchers, payers, planners and educators to make changes that will lead to better patient outcomes (health), better system performance (care) and better professional development (learning).[1] It is important because the size and complexity of primary care units are growing, thereby introducing more staff and multidisciplinary teamwork.[2–4] Also, there is some evidence that including staff in decision making is positively correlated with higher quality of care,[5] because their influence and shared knowledge bring encouragement and work satisfaction.[6,7]

Due to these developments, organisational assess-ment as a method for QI is becoming an increasingly common and accepted feature in general practice.[8–11]

Some types of QI focus on summative assessment and accreditation, such as the Visit in Practice method, proposed by van den Hombergh and involving external visitors,[12] whereas others, such as the Primary Care Assessment tool, focus on externally guided self-assess-ment by the practice team.[13] Yet other, more forma-tive development methods – which do not rely on the assessment itself – include the Multi-method Assess-ment process,[14] and the Clinical Micro-systems Survey.[15]

The Maturity Matrix (MM) is a formative instru-ment for internally driven development work, but also includes an element of (self-)assessment. For a detailed explanation on the theoretical background and the differences between the MM and other assess-ment tools see References.[3,16–18]

The MM allows general practices to assess their level of organisational development with the help of a trained facilitator, and aims at stimulating QI following the assessment. The MM instrument is based on the principle of the ‘balanced scorecard’,[19] and covers 12 dimensions of organisation including data manage-ment, QI activity, patient involvement and staff man-agement (see Box 1). Each dimension is broken down into eight stages describing a typical development pathway from a basic to a mature practice organis-ation. For further information see Appendix 1.

The aim of the MM is to stimulate practices to identify areas for improvement through a facilitated group process. A central aspect of this process is involvement of both GPs and staff in the MM process, as well as in the subsequent activities aimed at quality improvement.[16,18] Earlier studies of the MM show that participants find it a useful method to achieve insight and prioritise aspects in need of improvement, and find that it has high face validity,[18] and that it can be a starter level for QI before taking on a higher level such as European Practice Assessment.[20]

What remains unknown about the use of the MM and other similar formative self-assessment tools is how participants use the results of their assessment to achieve improvement, whether participants find the assessment useful as a starting point for processes of organisational change, and what the subsequent effects on teamwork and organisational development may be.

The objectives of this study were to examine overall satisfaction with the MM process one year after first exposure, investigate how the practice teams worked on the goals set at the MM meeting, and map barriers and drivers for reaching the agreed changes with regard to practice organisation and aspects related to the MM.

Developing and testing the Maturity Matrix in a Danish context

The Danish version of the MM shares dimensions, method and aims with the original UK version.[21] However, the three counties that financed the Danish version added a focus on improvement in organisational aspects identified via the MM and knowledge on how participants used the MM after the initial facilitation. Therefore, the facilitator in the Danish version aimed at getting participants to appoint a responsible ‘anchor person’ within the practice for each chosen develop-ment area, and to set their specific goal for the develop-ment process in the following year. After one year, the facilitated meeting was repeated, allowing the partici-pants to re-evaluate their organisational development and set new development goals for the coming year.

Method

Setting

The MM was introduced and adapted in Denmark in the years 2004 to 2006 using a pilot in four practices. Final adjustments were made and the MM was tested on a larger scale in 60 practices in 2007 to 2008. Par-ticipating practices were located in the counties of Aarhus, Copenhagen and Frederiksborg. The facili-tators, who visited participating practices, were GPs with prior experience from facilitating QI in general practice and specific training using the MM.

Participants

An invitation to the MM project was sent to all general practices in the participating counties. The first 60 practices to come forward were included in the pro-ject. These included single, small and group practices, and were found to be representative in terms of variations in size and geography. In all, 163 GPs and 157 staff members from the 60 practices participated in the first round of MM visits. Altogether, 117 GPs and 111 staff members from 48 practices also partici-pated in the second round of visits.

Interviews

Of the 48 practices that completed first and second MM visits, seven were purposefully chosen for inter-view in order to ensure variation in terms of geography, practice size and facilitators. Seventeen representatives were chosen for interview and equal representation between GPs and staff was obtained as presented in Table 1.

Data

A list of research areas (see Box 2) was developed on the basis of the following:

1   the investigator’s participation in management meetings in the Danish MM organisation

2 15 qualitative interviews that were conducted fol-lowing the first round of MM visits by an external investigator (Anna Viola Hammer, ethnologist; unpublished data)

3  the investigator’s observation of two facilitated practice meetings. One meeting was held in a group practice with five GPs, two secretaries, two practice nurses and one laboratory assistant, and one meet-ing was held in a small group practice with three GPs, two secretaries and two practice nurses.

The above preparations, and discussions with repre-sentatives of the MM project, gave a clear indication that the research areas presented in Box 2 were essen-tial to explore in the evaluation.

The areas listed in Box 2 were explored in semi-structured interviews lasting between 30 and 65 min-utes with each of the 17 participants. To further map the barriers for working with the MM, short telephone interviews or mail communications were conducted with representatives from 11 of the 12 practices that did not participate in the second visit (see Table 2).

Analysis

All interviews were recorded digitally and transcribed by a student assistant. In order to ensure room for new research themes to emerge, the interviews were openly coded by the interviewer and double checked by the student assistant. The open codes were subsequently coded selectively and clustered around the initial research areas (see Box 2). The coding process did not result in essential new analytical themes, but interesting subcategories such as power structure within the practice, motivation for participation, the influence of practice size, patient turnover and the practice’s need for support did emerge.

 

Results

Participants’ evaluation of the outcome from the Maturity Matrix

The data suggested that participants primarily associated the MM with the facilitated meetings and shared dialogue between GPs and staff members that took place and this was highly valued. Having a shared two-hour meeting about co-operation and organisation was described as a unique event by many participants, because it was usually the GPs alone that handledmatters of organisation and quality and because, according to the participants, the busy working day tended to crowd out both intentions and processes of QI.

 

The majority of the participants interviewed also found that the facilitation process provided a useful combination of overview and insight into the daily working routines of their practice that enabled them to assess their maturity in each dimension. The newly gained knowledge of their organisational maturity again enabled most practices to choose areas in need of organisational development in a much more systematic way than they were used to.

Several informants (mostly GPs) expressed that the shared dialogue between GPs and staff was useful in bringing forth new common knowledge, and stated that participation in the MM was a good starting point for improved communication and co-operation within the practice team. Some GPs added that the combination of a shared dialogue and an overview of their organ-isation was the MM’s main strong point compared to other projects concerning QI, which usually only concerned the GPs.

However, there were nuances to the findings above. Although most participants praised the shared dia-logue between GPs and staff, no participants found that participation had changed their daily communi-cation and co-operation. The main reason given by the informants was that the MM was considered too small an intervention to change the established group dynamic between GPs and staff or the standing daily procedures in their practice. Also, some of the staff members stated that the facilitation process had caused them stress because the GPs apparently found it much easier– and were much quicker – to understand and use the MM tool, resulting in episodes where staff members did not have time to finish their evaluation before the meeting went on to the ‘consensus discussion’.

A minority of the interviewed GPs criticised the MM tool for being incongruent with the stages of development in their organisation and noted that the individual steps in many dimensions were incoherent and poorly explained. Also GPs from two of the inter-viewed practices (the largest group practice and one of the small practices) found that the MM was ‘quite superficial’ and ‘harmless’ because participation was not based on a perceived need for improvement, introduced no cost, and entailed no formal obligation that went beyond participation in the facilitated meet-ings. Finally, the interviews revealed only a moderate wish to participate in the MM and similar QI initiat-ives if participation was free and practices were com-pensated financially for lost time, and there was no willingness to participate if participation entailed any financial cost.

Organisational factors within the practice that influence implementation

Informants from the four smallest practices found it more diffcult to organise and find time to implement the changes decided upon via the MM than inform-ants from the three group practices in the sample. The barriers experienced were a high patient-related work-load that kept participants in their daily routines, and no tradition for formal and shared meetings concerning QI because of small size and vulnerability to sickness and turnover among staff and GPs.

In contrast, most informants from the three group practices stated that one of the advantages of being large was an organisation with shared meetings and divided responsibility for different aspects of their organisation that made it easier to structure and find time to work with agreed changes. The downside to being large was, however, that processes of change were time consuming and dependent on a local anchor person who was really committed to implementing the intended change. In other words, having the time and a supporting structure was felt to be a necessary but not suffcient prerequisite for succeeding with the intended change.

Furthermore, informants from the two largest group practices did not express that turnover among GPs and staff or a high patient-related workload represented serious barriers in their work with the MM. When asked about these barriers, the informants agreed that they posed a challenge for QI, but also felt that the size of their practice made them relatively flexible when covering for sick colleagues, and meant that they were used to handling introduction of new GPs and staff using formal procedures.

Maturity Matrix-related factors that influence implementation

The interviews contained questions concerning the role of the external facilitator and the impact of naming an anchor person for each area in need of improve-ment.

Facilitators and follow-up activity

All participants were extraordinarily satisfied with their facilitator. The combination of a respected peer with detailed understanding of the everyday work and dilemmas of general practice, and the objective stan-dards from the MM tool provided a secure setting for the facilitation process, which enabled the participants to lay all facts on the table – including areas where the practice really needed improvement. Even when asked directly, no one suggested any changes to the role of the facilitator during the consensus meetings.

However, when asked, almost all participants ex-pressed a wish for more follow-up activities by their external facilitator as an integrated part of the MM. This was partly as an extra support for the responsible anchor persons in their work with specific target areas, and partly as a reminder for the practice team of their intentions and to motivate the team. Participants from one of the small practices in Aarhus county explained that their facilitator had contacted and helped them a couple of times in the year between the first and second visit. This relatively simple and small inter-vention was seen as big help.

Anchor persons

In six of the seven investigated practices one or two responsible persons were appointed for each chosen area of improvement. According to the participants, the anchor persons were responsible for keeping the implementation process active but not necessarily responsible for doing the actual work. According to most informants, the intended course of action was that the individual anchor person (in most cases a GP) prepared and presented a plan for implementation for the rest of the practice as part of one or more shared practice meetings. Informants from five of the six practices with anchor persons stated that at least one of the appointed anchor persons had made an actual effort by producing written material or by looking up ways to solve specific problems. Examples of the achievements of the anchor persons were to:

•    develop a form to register unintended events (un-toward or critical incidents)

•    make a list of the equipment that needed to be in an emergency box

•    write a guideline for the staff (concerning manage-ment of chronic conditions, e.g. diabetes or chronic obstructive pulmonary disease)

•    write a guideline for the entire practice (for example concerning emergency procedures or registration of untoward incidents)

•    find out how to do diagnosis coding

•    make an agenda and/or summary of a practice meeting

•    plan a weekend seminar for GPs and staff.

As intended, the decisions on how to implement agreed changes were usually taken as part of shared practice meetings, which most of the investigated practices had singled out as a prioritised area for improvement following their participation in the MM. The meetings were described as: a time where the anchor persons initiated the others in their work, an occasion to hold each other accountable with respect to the agreed work, and a place to gather up and agree on how to use the material that the anchor persons have made.

In contrast to the above, the GPs from the two single practices stated that they had not worked with their intended changes at all. In the Copenhagen practice, the GP reported that this was because ‘the nurse and secretary left the practice just after the first facilitator meeting’. In the practice from Aarhus the GP explained that:

‘At first I and my nurse agreed that it wasn’t necessary to appoint anchor persons [...] but now it is clear to me that we haven’t succeeded with anything because we didn’t share any responsibility between us – there were no dead-lines and no meetings [...] to me it is clear that even single practices have to plan and meet around intended changes.’

Personal responsibility was a common denominator of the positive statements above, but approximately half of the informants (most of them staff members) added that the personal responsibility that was intro-duced via anchor persons had posed some diffculties

– here exemplified by a nurse from the largest group practice:

‘In general we are not very good at keeping each other responsible, especially when it is a GP, the staff do not follow up on lack of initiative.’

Also the interviews were unable to shed light on the extent to which the initiatives developed by the anchor persons themselves were implemented as actual com-mon routines in daily practice.

Reasons for non-participation in second Maturity Matrix visits

There were 12 (20%) practices that did not participate in the second MM session. Those practices were con-tacted by mail or telephone and asked for the reasons why they chose not to go through with the second visit. The resulting answers are listed in Table 2.

As seen in the table, lack of perceived benefit, turnover and lack of time constituted the three main reasons for withdrawal from the MM. The reasons given were consistent with the statement from the conducted interviews, where turnover and lack of time constituted the commonest barrier for translating the results of the MM into action.

Discussion

Principal findings

Most informants found that the facilitation process and the MM tool provided a useful way to evaluate their organisation and to choose areas in need of improvement. In five of the seven participating prac-tices, the MM was used as a starting point for devel-opment, and progress was reported in four of these. All participants were very satisfied with the role of the facilitators during the facilitated meetings, but many requested greater input from facilitators.

In terms of barriers and actual processes of change, the results indicated that small practice teams found it harder to adhere to the intention of implementing QI than larger practice teams. Staff turnover was com-mon, but it particularly affected the smallest practices and crowded out the work on QI. Similarly, small practice teams were characterised by an informal team and meeting structure that made it diffcult to operate with responsible anchor persons, deadlines and formal-ised meetings – aspects that larger practices in the study saw as a clear support for their work with QI. Also the MM currently seems to be conceived by participants as an event rather than a sustained pro-cess – as they appeared to view the main outcome as the meeting and the shared dialogue taking place – rather than following the process of implementation or changes in daily communication and co-operation.

Finally, the results indicate that the individual aspects of the facilitation process favoured the GPs over the staff, most likely because GPs were more academically trained and had a higher degree of previous knowledge of the concepts addressed via the MM. If this head start allows the GPs to dominate the dialogue and the consensus scoring – thereby maintaining the established power structure – it may also act as a barrier to inclusion of the staff in shared decision making and the process of change.

Strengths and limitations of the study

The main strength of this study was that it is among the first to explore in depth the connection between change initiated via facilitated quality assessment and the methods used for implementation. It also assessed experiences of practice development from both GP and staff perspectives. However, the sample was small and derived from one country only. Healthcare con-text will vary, particularly in terms of variations in size of organisations, financial incentives, and tradition for locally driven QI; different participant experiences and evaluations are likely in different contexts.

Comparison with existing literature

The generally positive evaluation of the facilitation process and the MM tool by participants in this study was consistent with other quantitative evaluations of the MM.[21,22] The reported organisational develop-ment achieved in this small sample was also consistent with the progress made among the wider sample of 60 participating practices in which practices generally scored themselves higher on the second than on first MM sessions, showing both change in global score and across all dimensions.[23]

The relationship between effective organisation and good-quality patient care is widely accepted, resulting in a growing interest for organisational QI tools.[24–26] Although the connection between participation in quality assessment and resulting change is not well documented, other investigations have shown similar findings to our study. A qualitative study using semi-structured interviews to identify barriers and facilitators for implementation of changes in practice according to guidelines in the Netherlands, found an excess of arriers that could provide explanation for resistance to change.[27] Barriers included pressure from patients and, consistent with our study, lack of time and personal routines. Facilitators for change included strategies to prevent relapse into old routines, and structured QI activity with repeated educational meeting and follow-up.[27] Other studies have found similar barriers and drivers of QI.[28,29]

The experiences of participants with the MM were generally positive. This formative self-assessment and development method can be compared favourably with summative methods such as practice visits or complex accreditation methods such as Quality Team Development.[30] Also the MM has the potential to satisfy both participants’ (practices) and strategic pri-mary care organisations’ needs for structure and tangible output, which other formative development methods such as the Clinical Micro-systems Survey and Multi-method Assessment do not address.

Implications for practice

The results raise a key question concerning internally versus externally driven motivation. This arises from participants giving a general impression that the MM represented a risk-free opportunity more than a per-ceived necessity and was attractive because it imposed no immediate cost or obligations on participating practice teams. There also appeared to be low willingness to participate if participants were not compensated for costs or lost time. These issues indicate barriers that need to be addressed for meaningful participation to achieve a likelihood of sustained change. However, they may also be helpful in identifying opportunities in terms of how motivation for QI, and thereby trans-lation of agreed changes into practice, can be enhanced via more intensive support and a more binding com-mitment when participating.[27]

Further research

The feasibility of the MM as an intervention to promote QI and achieve development requires evalu-ation in other healthcare systems. Also, there is a need for attention to how the introduction to QI and supporting activities can be enhanced, particularly in the facilitation of meetings, and the strengthening of support that the facilitator can provide to practices during the development phase.

Conclusions

Participation in the MM constitutes a useful starting point for QI. However, for facilitated quality assess-ment and improvement to function more effectively as a lever for actual change, participation should include:

(a) greater awareness of the importance of common meeting structures, organisational support and lead-ership, (b) help to organise and support an active interaction between anchor persons and the rest of the staff and GPs, (c) systematic follow-up activity and support to participating practices, (d) awareness of the barriers for QI that are connected with small practice size, and (e) consideration of how to create incentives for commitment among participating prac-tices that stem from a combination of external, formal and internally driven motivation.

References

1. Batalden P and Davidof F. What is ‘‘quality improve-ment’’ and how can it transform healthcare? Quality and Safety in Health Care 2007;16:2–3.
2. Hombergh P. Practice Visits. Assessing and improving management in general practice. PhD Thesis, Nijmegen, Netherlands, 1998.
3. Rhydderch M, Edwards A, Elwyn G et al. Organizational assessment in general practice: a systematic review and implications for quality improvement. Journal of Evalu- ation in Clinical Practice 2005;11:366–78.
4. Moss F, Garside P and Dawson S. Organisational change: the key to quality improvement. Quality in Health Care 1998;7(suppl.):1–2.
5. Rhydderch M, Elwyn G, Marshall M and Grol R. Organisational change theory and the use of indicators in general practice. Quality and Safety in Health Care 2004;13(3):213–7.
6. Westrum R. A typology of organisational cultures.Quality and Safety in Health Care 2004;13 (suppl 2): ii22–7.
7. Tapp L, Edwards A, Braspenning J, Eriksson T, Kuch C and Elwyn G. Developing organisational maturity: con-sidering the role of culture typologies for primary care practices. Education for Primary Care 2008;19(1):1–12. 8 Walshe K and Walsh N. Accreditation in primary care.
8. In: Walshe K, Walsh N, Schofield T and Blakeway-Phillips C (eds) Accreditation in Primary Care: towards clinical governance. Oxford: Radcliff Publishing, 2000.
9. Huntington J, Gillam S and Rosen R. Clinical govern-ance in primary care: organisational development for clinical governance. BMJ 2000;321:679–82.
10. Campbell SM, Braspenning J, Hutchinson A and Marshall M. Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care 2002;11(4):358–64.
11. Buetow SA and Wellingham J. Accreditation of general practices: challenges and lessons. Quality and Safety in Health Care 2003;12(2):129–35.
12. Hombergh P, Grol R, Hoogen Hvd and Bosch W. Assessment of management I. general practice: validation of a practice visit method. British Journal of General Practice 1998;48(436):1743–50.
13. Starfield B, Cassady C, Nanda J, Forrest CB and Berk R. Consumer experiences and provider perceptions. Journal of Evaluation in Clinical Practice 1998;11(4):366–78.
14. Crabtree BF, Miller WL and Stange KC. Understanding practice form the ground up. Journal of Family Practice 2001;50(10):881–7.
15. Mohr J and Batalden P. Improving safety on the front lines: the role of clinical microsystems. Quality and Safety in Health Care 2002;11(1):45–50.
16. Rhydderch A, Edwards A, Elwyn G, Marshall M and Grol R. Developing a facilitation model to promote organ-isational development in primary care practices. BMC Family Practice 2006;(38).
17. Rhydderch M, Elwyn G, Marshall M and Grol R. Organisational change theory and the use of indicators in general practice. Quality and Safety in Health Care 2004;13(3):213–7.
18. Elwyn G, Rhydderch A, Edwards A et al. Assessing organisational development in primary medical care using a group based assessment: the Maturity Matrix. Quality and Safety in Health Care 2004;13:287–94.
19. Kaplan RS and Norton DP. The Balanced Scorecard: Translating Strategy Into Action. Harvard Business Press, 1996.
20. Edwards A, Rhydderch M, Engels Y et al. Assessing organisational development in European primary care using a group based method: A feasibility study of the Maturity Matrix. International Journal for Quality in Health Care 2009; in press.
21. Rhydderch M. Assessing organisational development in family practice – The Maturity Matrix. PhD Thesis, Nijmegen, Netherlands, 2006.
22. Løgstrup L, Edwards A, Waldorff F, Siersma V, Eriksson T and Buch M. GP and staff evaluation of the Maturity Matrix as a tool to assess and improve organizational development in primary care. International Journal of Health Care Quality Assurance 2009; in press.
23. Eriksson T, Siersma V, Løgstrup L, Buch MS, Elwyn G and Edwards A. Documenting organisational develop-ment in general practice using a group based assessment method – the Maturity Matrix. Quality and Safety in Health Care 2009; submitted.
24. Rhydderch M, Edwards A, Marshall M et al. Maturity Matrix: a criterion validity study of an instrument to assess organisational development in European general practice. Quality in Primary Care 2006;14(3):133–43.
25. Campbell SM, Cantrill JA and Roberts D. Prescribing indicators for UK general practice: Delphi consultation study. BMJ 2000;321(7258):425–8.
26. Buetow SA and Wellingham J. Accreditation of general practices: challenges and lessons. Quality and Safety in Health Care 2003;12(2):129–35.
27. van Bokhoven M, Koch H, Dinant G-J, Bindels P, Grol R and van der Weijden T. Exploring the black box of change in improving test ordering routines. Family Practice 2008;25(3):139–45.
28. Geboers H, van der Horst M, Mokkink H et al. Setting up improvement projects in small scale primary care prac-tices: feasibility of a model for continuous quality improvement. Quality in Health Care 1999;(8):36–42.
29. Wallin L, Bostro¨m A-M, Harvey G, Wikblad K and Ewald U. Progress of unit based quality improvement: an evaluation of a support strategy. Quality and Safety in Health Care 2002;(11):308–14.
30. Royal College of General Practitioners. Quality Team Development – an accredited approach to quality improve-ment for primary health care teams. London: Royal College of General Practitioners, 2002.

Peer Review

Not commissioned; externally Peer Reviewed.

Conflicts of Interest

None.