Research Paper - (2007) Volume 15, Issue 3
1Consultant in Public Health Medicine, Gateshead Primary Care Trust, UK
2GP Principal and Clinical Governance Lead, South Tyneside Primary Care Trust, UK On behalf of the Gateshead and South Tyneside PCTs’ Joint Assessment Advisory Group
Received date: 19 December 2006; Accepted date: 13 March 2007
Introduction Primary care trusts (PCTs) in England are required to set in place local arrangements to identify and deal with concerns about general practitioners’ (GPs’) performance. Assessing GP performance at a local level can be challengingand there is little published information available to describe the methods PCTs use. This paper describes the local assessment methods developed by two PCTs.Methods Gateshead and South Tyneside PCTs have jointly developed methods to locally assess whether GPs are underperforming. The methods involve lay, clinical and management representatives and employ a variety of tools including casebased assessment and a questionnaire to colleagues.Most of these tools measure performance against the standards set out in Good Medical Practice or collate data derived from nationally validated surveys. The methods have been developed to promote transparency, objectivity and consistency while making the most of scarce local expertise.Results In our experience, case-based assessment and questionnaires to colleagues provide the most helpful information. Our local assessments enable practitioners to continue their work and do not incur travel or accommodation costs for the assessedor the assessors. GPs and their defence organisations find the methods acceptable.Conclusions It would be helpful for other PCTs to publicise their assessment methods so that best practice can be developed and standardised, thus ensuring that all patients and GPs receive the same levels of protection and support at a local level.
assessment methods, professional regulation, quality assurance
How this fits in with primary care
What do we know?
National changes to the regulation of general practitioners highlight the need for more local assessment of general practitioners. However, little is known about the local methods that primary care organisations use to assess concerns about general practitioners’ performance.
What does this paper add?
This study provides an overview of the methods used in two English primary care organisations to locally assess GPs who may be underperforming.
Measures to protect patients and assure the quality of general medical practice are current priorities for the NHS and the medical profession.[1,2] At a local level, primary care trusts (PCTs) are already required to set in place local arrangements to identify and deal with concerns about general practitioners’ (GPs’) performance.[3]
The arrangements for, and the activities of, the local performance procedures (LPPs) in Gateshead and South Tyneside, two PCTs in the north east of England, have previously been described.[4] These procedures have existed since 1997 and have evolved into the South of Tyne Assessment Advisory Group (AAG), established in 2003, which currently leads the process and comprises amultidisciplinary group of lay, managerial and clinical representatives. The AAG is responsible for investigating all concerns about local GPs, with the aim of identifying doctors who are underperforming and recommending how the PCT should manage the underperformance.
Since 1997, there have been few changes in the core membership of the supervisory group, resulting in 10 years of experience of locally assessing and managing practitioners whose performance gives cause for concern. The AAG investigates concerns about approximately 25 GPs annually. If cases are deemed serious or if the GP will not comply with LPPs then they are referred to either the General Medical Council (GMC) or the National Clinical Assessment Service (NCAS). We have averaged three such referrals per year since 2003.
The methods that the AAG uses to assess performance have evolved over time in response to local and national experience and guidance. The current methods were adopted in 2004. We see this structure and its processes fitting well with local GMC affiliates described in the recent White Paper.[5] Given the numbers involved, regional GMC affiliates will need to liaise with and co-ordinate local assessment bodies in primary and secondary care, as they are unlikely to be in a position to undertake all assessments in house and are encouraged to seek ‘more effective engagement with local services’.[5]
The best way of assessing GP performance is the subject of significant debate.[1,2,6–8] The aim of this paper is to describe the local methods the South of TyneAAG4 currently uses to investigate local concerns about GP performance and make a judgement about performance. Our own experience is that doctors, managers and patients are poorly informed about the methods used to assess GP performance at a local level, and there may be considerable variation in the methods that are used. We have not identified any published information about the assessment methods used locally in other areas.
Identifying GPs who give concern
The AAG invites and considers concerns about contractors’ performance from a wide range of voluntary, statutory and professional groups, organisations and individuals. All concerns are considered by the AAGat monthly meetings according to the two-stage assessment process outlined in Figure 1.
Figure 1: Gateshead and South Tyneside AAG: assessment process
In Stage 1 the AAG members act as a peer review panel considering concerns raised, supporting information about the concern, and then collectively determining whether there is cause for concern. In many instances the concern requires further investigation before the AAG can come to a decision. Such investigations are undertaken by the chair (as lay representative) and one of the clinical members, by undertaking a structured interview with the practitioner concerned. The interviewees develop their questions and record their findings on a standard proforma based around the standards set out in Good Medical Practice (GMP).[9,10]
Once AAG members feel that they have sufficient information to make a decision about the concern, there are three possible outcomes to Stage 1. If the panel agree there is no cause for concern (for example if this was a nuisance complaint or the result of an unforeseeable event), the findings are noted and we thank the clinician for their co-operation. If there are only minor concerns regarding underperformance, the practitioner is informed, appropriate action is undertaken which may involve the PCT clinical governance or other support groups, and although there is no further assessment we do request follow-up to ensure the action has happened. If the panel agrees that there is significant cause for concern, then the practitioner moves to Stage 2 of the assessment process and the panel considers how to manage the concerns regarding performance. If the panel determines that further assessment is required before appropriate actions can be identified, then they will either recommend a full local or NCAS assessment. This decision is often made after consulting NCAS advisors.
Assessment tools used in Stage 2 to generate evidence about GPs’ performance.
Principles
Arange of assessment tools have been developed to meet the AAG’s principles of assessment set out in Box 1.
Box 1: Principles underpinning the local assessment tools
Contents
The tools aim to measure the dimensions of performance against the standards set out in GMP using local information which is easily accessible. These dimensions are summarised in Table 1 together with brief details.
Table 1: Tools used by Gateshead and South Tyneside Assessment Advisory Group to measure a GP’s performance against the standards in GMP
Samples of two of the tools are provided in Appendix 1 (the questionnaire to colleagues) and 2 (the casebased assessment proforma). Both of these tools are based on the standards set out in GMP.[9,10] Colleagues are asked to rate practice and provide comments on strengths and weaknesses. If more than one colleague scores low in the same area then the pattern can point to areas of concern to follow up at the practice-based assessment. Further details of any of the tools can be provided on request.
For cased-based assessment the visiting clinicians examine 30 sets of notes drawn at random from a large sample for which patients have given their consent, to reflect the following groups: recent deaths and referrals; patients with chronic diseases such as ischaemic heart disease (IHD), diabetes etc, and any clinical group in which there has been reported cause for concern. For example if the concern is failure to visit patients we may request a sample of records of patients whowere visited at home, as well as those who requested and failed to get a home visit. Details from the cases are then used in discussion with the clinician.
Applying the assessment tools
The AAG sends relevant GPs a list of the areas of practice that will be assessed.
Anew assessment team is convened to undertake each local assessment. The team comprises four assessors – the lay AAG chair to provide lay representation, the public health lead acting as a clinician focusing on the quality agenda, a primary care liaison officer from the PCT holding the contract and providing all the administrative support, and a clinical governance lead from the neighbouring PCT to undertake the clinical assessments without having any bias arising from previous knowledge in their local clinical governance role.
The team meets to plan the assessment, agree individual roles and responsibilities, and specify the criteria for identifying and retrieving case notes to be used for the case-based assessment. It also reviews the findings from assessments that have been completed in advance of the practice-based assessment as indicated in Table 1.
The practice-based assessment lasts a whole day, during which a range of assessments are completed (see Table 1) including seeking the views of the practice manager and members of the primary care team.
Using the evidence to assess performance
All of the information is recorded using standard proformas and then used to form a chain of evidence to support any conclusions about performance.Toavoid bias arising from unsubstantiated evidence, concerns about performance are only recorded when there is evidence from more than one source (triangulation).
The assessment team considers each of the areas of GMP and concludes a level of concern – low, medium or high. On the basis of these conclusions, the team makes recommendations about a remedial action plan or referral to a regulatory body. The evidence, conclusions and recommendations are reported to the AAG and the GP, who is given the opportunity to comment within the body of the written report.
These methods have been used to plan and undertake four GP assessments so far. The assessments have resulted in detailed action plans which have led to improvements in performance that have been demonstrable at the follow-up visit.
Experience of the effectiveness of the various techniques
Experience of the assessments has shown us that although every element of the assessment process can provide helpful information, two techniques are especially revealing. These are the questionnaires to staff and colleagues and the case-based assessments.
Colleagues’ questionnaires are most valuable in identifying behavioural and management performance problems. However, colleagues are often reluctant to reveal what they know if it is critical of a fellow clinician, and we sometimes have to remind individuals that their clinical governance responsibilities and patient safety must outweigh any sense of loyalty.
The case-based assessments are most discriminatory in identifying poor clinical performance. Examples of areas that have been identified and used in subsequent discussions include:
fla visit where a patient was given antibiotics for a ‘chest infection’ and also a flu vaccination at the same time
fla request from a hospital consultant to titrate the dose of an angiotensin-converting enzyme (ACE) inhibitor for a patient with diabetes, hypertension and microalbuminuria where the starting dose remained unchanged over 15 months
fla patient who had been noted as starting a reducing course of diazepam but for which there were no further records regarding dosage, and prescriptions for the original dose were still being issued two years later.
The financial burden of local assessments is largely staff and administrative costs. The AAGmeets monthly, and this is regarded as part of the job of all involved from the PCTs whether clinical or administrative. The exception is the lay chair for whom this is their job. However, the assessment visits fall outside these commitments. The four members of the team spend about two days participating in planning, assessing and formulating a report and recommendations on an assessment. There are few financial implications for the assessed GP who can continue to practice and does not have to travel to a different location. There are no significant travel or accommodation costs for any party.
Local assessments may be quick to complete but can still be slow to organise. We have found that most GPs delay the assessment as long as possible. It is possible that closer links with the local GMC affiliates may minimise this.[5]
Despite the delays, GPs have bought into the process and they welcome the objectivity and transparency of the assessment process.OneassessedGPwrote: ‘I think it would be beneficial for all doctors to periodically go though a less formal version of such an assessment’.
All GPs assessed so far have involved their representatives from defence organisations in the assessment interviews and in responding to the assessment report. These representatives have provided verbal support for our methods and we have not received any critical feedback from them.
We appreciate that assessing performance at a local level is challenging for all parties. Assessed GPs face an emotionally demanding ordeal and are justifiably concerned about the objectivity, confidentiality, legitimacy and validity of LPPs. Assessors are aware of the gravity of their recommendations, and local relationships and existing knowledge may influence their ability to be objective and adequately protect patients or support doctors.
Our aim is to protect patients and support doctors by developing processes and tools which are consistent with our principles (see Box 1) and enable the local assessment of clinical performance (what the doctor actually does in the workplace) against the standards set out in GMP.[10]
All of the evidence and judgements are available for external scrutiny, thus promoting a transparent process which is open to challenge. Our use of standardised tools and triangulation of evidence promotes objectivity because assessors make recommendations based upon weight of evidence rather than subjective impressions. Furthermore, because a significant part of the evaluation is based on the clinician’s own records, we are able to consider examples of actual practice rather than practice in simulated or atypical settings. Our case-based assessment is a context-rich assessment which explores application of knowledge to practice, not just the underlying knowledge base.[11]
Our methods are acceptable to doctors and their defence organisation representatives, and can be undertaken quickly although the speed with which they can be organised depends on the co-operation of the doctor being assessed. The main costs of the process relate to employing the assessment team over two days. There are no financial costs to the clinician, who can continue to practice.
The recent White Paper, Trust Assurance and Safety – The Regulation of Health Professionals in the 21st Century recommends an extension of regulatory powers to a regional level, working closely with local services.5 Our experience and findings can be used to inform debate on that issue as well as allowing other PCTs to compare their practice with ours.
In the context of LPPs, more widespread information about local assessment methods would be helpful to enable PCTs to identify and agree best practice to ensure that all patients and GPs can expect the same levels of protection and support at a local level, and to link such procedures into a future regulatory structure.
The members of the Gateshead and South Tyneside PCTs’ Joint Assessment Advisory Group provided advice and support in developing the methods described in this paper.
None.
Appendix
Appendix
