Research Paper - (2005) Volume 13, Issue 1
Honorary Senior Lecturer
Ian Harvey MB PhD FFPH
Professor of Epidemiology and Public Health
Amanda Howe MA MD MEd FRCGP
Professor of Primary Care
Gillian Horrocks PhD
Research Associate
School of Medicine, Health Policy and Practice, University of East Anglia, Norwich, UK
Received date: 9 November 2004; Accepted date: 24 November 2004
Aims: To understand the aids and barriers to the progress of clinical effectiveness within primary care organisations (PCOs). Design: A qualitative study based on interview data from a two-stage survey employing postal questionnaires and telephone interviews. Key informants in PCOs were nominated by their chief executives. Data collection started on 1 October 2000 and ended on 4 April 2002. Setting: PCOs in England. Results: Barriers to clinical effectiveness as perceived by the participants were mostly non-specific: under-funding, excessive central guidance, and constant reorganisation. Specific barriers included negative attitudes to clinical effectiveness and a limited understanding of its nature. Leadership and the building of personal relationships were cited as aids. Involvement by public health doctors was seen as an aid by some, though some participants would have welcomed more involvement. Conclusions: Two options for PCOs in enhancing clinical effectiveness are suggested by these results. They could promote training to improve knowledge and understanding of clinical effectiveness within their organisations. Key figures in PCOs could raise the profile of clinical effectiveness through leadership and example.
aids, barriers, effectiveness, primary care organisations, qualitative
Clinical effectiveness is closely related to evidence based health care. Both terms refer to systematic at-tempts to improve the quality of health care by closing the research–practice gap.[1] Clinical effectiveness has a health service perspective with the emphasis on en-suring that research findings are disseminated to those who should apply them.[2] Evidence-based health is rooted in clinical practice with the emphasis on equipping clinicians with the skills needed to utilise research findings.[3] The term clinical effectiveness is used in this report because the focus of the study is at the health service level. Clinical governance is a systems approach for quality improvement which includes audit and education, as well as evidence-based care.[4]
The transfer of knowledge from research to practice is problematic.[3] The rigours of evidence-based prac-tice pose several obstacles: the time needed to find and evaluate relevant evidence, the lack of relevance of evidence to some types of clinical problems, and information overload.[5–10] It has been claimed that general practitioners (GPs) lack the skills to access
electronic sources of information and an understand-ing of evidence-based practice.[11,12] It has been sug-
gested that GPs may not share the assumptions of
evidence-based practice but other studies have found that some GPs welcome evidence-based medicine.[8,13,14]
There has been less research on the attitudes of allied health professionals and policy makers towards evidence-based care, but what has been done suggests
that they too see a skills gap, lack of time and irrele-vance of available information as the main barriers.[15–18]
The internal market created by the National Health Service and Community Care Act, 1990 introduced purchasing as part of a wider process of commis-sioning, which encouraged managers to evaluate the effectiveness of management options.[19] The incoming Labour government of 1997 announced a greater role for clinical effectiveness in the White Paper, A First Class Service: quality in the NHS.[4] Primary care groups and primary care trusts, collectively known as primary care organisations (PCOs), were established in 1999 and given the task of organising primary care and commissioning secondary care. The Department of Health required PCOs to implement nationally de-rived guidance in the form of guidelines from the National Institute for Clinical Excellence (NICE) and National Service Frameworks (NSFs). It also expected that effectiveness should be considered in all decisions:
Clinical decisions should be based on the best possible evidence of effectiveness, and all staff should be up to date with the latest developments in their field.[4]
The aim of the current study is to investigate aids and barriers to the progress of clinical effectiveness in PCOs as perceived by those entrusted to promote it within the organisations.
A two-stage survey of PCOs in England was under-taken between 1 October 2000 and 4 April 2002. A one in two sample of primary care groups (PCGs) (n = 217), all 22 primary care trusts (PCTs), and all 24 PCGs that were scheduled to become PCTs in the study period were included. Chief executive officers nominated a key informant to participate.
The aim of the first stage was to describe the extent of clinical effectiveness activity in PCOs. This was con-ducted by postal questionnaire which included work-ing definitions of clinical effectiveness and clinical governance. Respondents were invited to participate in the second stage, a semi-structured telephone inter-view. The interviews provided the data for this report.
Public health doctors were given the responsibility for clinical effectiveness within health authorities.[19] Therefore, questions were asked specifically about the role of public health doctors if interviewees did not spontaneously mention it.
Field notes were taken during interviews which were also recorded and transcribed. A grounded theory ap-proach was used. Credibility was improved through investigator triangulation: the field notes were analysed by KH and the transcripts by GH independently before comparing emerging codes, concepts, and themes. Disagreements were resolved through negotiation.
One-hundred and sixty questionnaires (61%) were returned. Ninety-four of the 160 respondents (59%) agreed to be interviewed. The professional backgrounds and positions of the participants are shown in Table 1.
Table 1: Characteristics of respondents and interviewees
Seven themes emerged. They did not fall into two discrete groups, aids or barriers. What seemed an aid to some was a barrier to others and sometimes both to the same people. We report each theme and explore its status as aid or barrier. Some themes were not specific but related more generally to the core tasks of PCOs.
Shortage of resources
The most common theme was the shortage of re-sources, mentioned by 90 interviewees. Two terms recurred in different guises: time and money (Table 2, comment 1).
Table 2: Expressed perceptions of aids and barriers to clinical effectiveness
Some saw clinical effectiveness as desirable but unaffordable. Others thought it vital to efficient ex-penditure (comment 2). Three types of resource shortage were identified. First, administrative infra-structure: a shortage of staff within the PCOs to support clinical effectiveness. Second, clinicians’ availability: insufficient time to attend activities to promote clin-ical effectiveness. Third, a budget deficit: resources were insufficient to implement effective interventions, such as statin therapy. Several interviewees stated im-mediate needs left little time for reflection and plan-ning, such as when considering effective prescribing (comment 3). One interviewee (from a PCT with an advanced system of quality improvement) dismissed financial constraints as an excuse, and stated forcefully that ‘you just have to get down and do it’.
Tension between central directives and locally derived goals
The influence of central directives was mentioned by 24 interviewees in all. A few stated that they had used them to improve attitudes to evidence-based practice or as a useful lever against professional defences (comment 4). Several thought that they provided easily accessible information.
However, most interviewees were ambivalent, stating that central directives were also a hindrance. The view that too many central directives were swamping local initiatives spanned all professional categories (com-ment 5). They regretted the passing of the original notion of the Health Improvement Plan, a strategic programme led by local health authorities aimed at the
needs of their populations, as envisaged in The New NHS.[20]
Rapid and continual organisational change were major distractions from the core work of PCOs
Many interviewees reported that frequent health ser-vice reorganisations were distracting them from their core tasks (comment 6). Recurring phrases were ‘change fatigue’ and ‘continuous reorganisation’. One comment epitomised the theme: ‘The agenda is ever changing and rapidly’.
Attitudes to clinical effectiveness and the relationships between practitioners and PCO personnel
Attitudes of practitioners
Thirty interviewees reported that practitioners’ atti-tudes to clinical effectiveness were not receptive and sometimes hostile. Some interviewees focused on the attributes of practitioners while others focused on practitioners’ perceptions of clinical effectiveness. Sev-eral attributes were felt to be important:
• a reluctance to change: ‘GPs are anti-change’, ‘conservative outlook of nurses’, ‘GPs focused on short term and sickness’
a lack of skills or commitment to quality improve-ment: the persistence of a ‘largely anecdotal style’ of medicine, ‘GPs are not academic so they are not interested in measurement’.
However, some interviewees reported a hostile atti-tude to clinical governance in general rather than specifically to clinical effectiveness. Perceived atti-tudes varied from indifference (‘apathy’ or ‘waste of time’) to outright suspicion (comment 7). Some GPs were reported to fear that it was either a cover for cost cutting or an exercise in ‘policing’.
Several interviewees reported more positive atti-tudes. Nineteen reported that GPs were favourably disposed. Thirteen believed that nurses and allied health professionals were more favourably disposed than GPs, offering various reasons: nurses are accus-tomed to guidelines, they are more reflective, and they have something to gain from the expanded role offered by clinical effectiveness initiatives. Thirteen interviewees reported that they felt that attitudes to clinical effec-tiveness were improving with time (comment 8).
Attitudes of senior PCO members
Levers for change included leadership from key figures and personal relationships. The views of key figures such as chief executives were seen to be important in influencing the culture of the organisation and the allocation of resources (comment 9).
Several interviewees believed that they had won over practitioners to the PCO in general, and hence overcome negative attitudes to clinical governance and clinical effectiveness by building amicable relation-ships with practitioners. These interviewees were either managers or prescribing advisers. Building relation-ships was recognised to take longer but considered to give better results (comment 10).
A lack of familiarity with the concept of clinical effectiveness and the skills to practise it
Several interviewees stated that practitioners or PCO personnel lacked skill and competence in clinical effectiveness. Individuals cited as having and applying the requisite skills included public health doctors, clinical governance leads, or clinical effectiveness facil-itators but the most frequently cited person was the prescribing adviser. However, a few managers assumed that all practitioners were practising effectively (com-ment 11).
A striking feature of many interviews was that the answers given by interviewees to direct questions on clinical effectiveness had little to do with clinical effectiveness, but rather with other related clinical governance matters. This occurred even though all intervieweeshad been provided with working definitions of clinical effectiveness and clinical governance. This implies a widespread under-appreciation of clinical effectiveness. Other activities often mentioned as if they were the same as clinical effectiveness included audit, education, and the equalisation of access (see Box 1).
Box 1
Only a few interviewees commented on the diffi-culties of practising clinical effectiveness as reported in the other surveys. The relative paucity of References to difficulties with the search for and the appraisal of evidence could be because these activities were in-frequently undertaken. However, one group reported frequently undertaking search and appraisal of evi-dence as part of their work, and these were the prescribing advisers.
In all, 30 interviewees believed that the traces of fundholding continued to influence the work of the PCO. Four thought this was beneficial, believing that ex-fundholders were more receptive to change. Others disagreed. Three stated explicitly that fundholding had left a legacy of inequity and the strategies for primary care development reported by another 26 respondents revealed that the inequity of services between former fundholders and non-fundholders was an important issue in shaping their strategies. The commonest ex-amples were counselling and physiotherapy (com-ment 12).
Secondary care was seen as still beyond the influ-ence of commissioners even though increasing such influence was the stated aim of so many NHS re-organisations (comment 13). One commissioning manager reported receiving a hostile response after asking for the evidence base for new business pro-posals made by the acute Trust.
The role of public health doctors
Involvement by public health doctors in the work of PCOs varied greatly in the degree of involvement and the content of their work.
Degree of involvement
The degree varied from virtually none to close in-volvement. Poor relationships between the PCO and the health authority were reported as the cause of poor relationships with its public health department by several interviewees. Some interviewees reported a specific apparent lack of interest in clinical effective-ness by some public health doctors (comment 14).
In contrast, some interviewees reported good rela-tions with the health authority and its public health department from the start. Some authorities had already played a leading role in clinical effectiveness through such programmes as the PACE programme (Promoting Action on Clinical Effectiveness) before the creation of PCOs.[21]
Other interviewees identified lack of resources as a barrier to public health doctors’ involvement (com-ment 15). Involvement improved when public health doctors were appointed to PCOs, a situation which occurred as PCGs became PCTs.
Several interviewees realised during the interview that they had not sought as much help as they could have, raising the issue of who should be taking the initiative. Several interviewees reported that public health doctors based at their health authority tended to take a reactive approach to the work of the PCO, providing information only in response to requests. Several interviewees stated that they would have pre-ferred a more proactive approach.
Content of work
Public health doctors have many skills and access to information that is potentially useful in the work of PCOs: local epidemiological data, needs assessment, education, performance indicators, audit and evalu-ation, and searching and appraisal. Those interviewees who stated that the public health department had supported them in their pursuit of clinical effective-ness were asked to elaborate. More frequently than not, the examples given had little to do with search and appraisal but more to do with epidemiological data, needs assessment, and audit.
Some public health doctors, including those based at health authorities, went beyond providing information and took a leading role in clinical effec-tiveness in the work of PCOs, for example, running clinical effectiveness units. Some PCOs pooled re-sources with others to participate in shared clinical effectiveness units. All had evolved out of units that had existed prior to the creation of PCOs, such as former medical audit advisory groups. In a few cases, such initiatives which had existed before 1999 disap-peared with the reorganisations following 1997 and were not replaced. Three interviewees were public health doctors. They would have liked to contribute more if allowed to (comment 16).
A potential weakness of a survey of individuals is that the views expressed may not be representative of the organisation as a whole. However, the perceptions of these individuals are both credible and significant since they were key informants in their organisation on the subject under study. A wide range of professional back-grounds was represented, so the views are not con-fined to any particular group.
The main barriers to clinical effectiveness perceived by the interviewees were similar to the barriers to the work of PCOs in general. These barriers, which have been reported in several studies, are: shortage of re-sources, excessive organisational change, and tensions in delivering national directives. The shortage of re-sources was identified as a barrier in the National
Evaluation of Primary Care Groups and Trusts and a qualitative study of PCGs and PCTs.[22,23] The Tracker Survey reported that chief executive officers of PCOs perceived shortages to be the main obstacles to pro-gress including quality improvement.[24] The compounding effect of the disruption caused by organisational change has been noted in several studies.[23–25] The hindrance posed to local initiatives by a large volume of guidance from the centre, and the priority commanded by national targets over local ones has been noted in other studies.[23,24]
These three barriers to the work of PCOs in general appeared to be of greater importance than any specific barriers to clinical effectiveness itself. Negative atti-tudes of practitioners were mentioned by only a third of interviewees. This was less than might be expected from studies of practitioners themselves, and the reasons for it are open to conjecture. It is unlikely that the practitioners belonging to the PCOs we sur-veyed were different from practitioners as a whole, since nearly a quarter of PCOs in existence at the time were included. A more plausible explanation is that few negative attitudes were encountered because PCOs on the whole did not promote clinical effectiveness in their early years, when the emphasis was on launching themselves as organisations.[26] Whatever valuable con-tributions public health doctors were making to PCOs, their potential to support clinical effectiveness appears to have been less fully realised than their other conventional public health roles.
Although the main barriers are beyond the power of PCTs to change, there are two areas where they can make a difference. First, they could promote training to improve knowledge and understanding of clinical effectiveness within their organisations. Second, key figures in PCTs could raise the profile of clinical effectiveness through leadership and example. The perception of some interviewees that attitudes among practitioners were improving might mean that such moves would find more favour amongst practitioners than has hitherto been expected.
We would like to thank: Professor Mildred Blaxter, UEA, for advice on the design of the project, Dr Barbara Richardson, UEA, for comments on an earlier draft, and Dr Karen Blades, freelance GP, for assistance with analysing the examples of clinical effectiveness.
None.
