Research Paper - (2003) Volume 11, Issue 4
Assistant Psychologist
Rachael Noble BSc
Assistant Psychologist
Chris Leach PhD MSc FBPsS CPsychol*
Research Consultant/Consultant Clinical Psychologist
Karen Holland MA Dip Speech Therapy DMS
QED Department Manager
Virginia Minogue PhD MA BSc CQSW Dip SW
R&D Manager
Quality, Evaluation & Development (QED) Department, South West Yorkshire Mental Health NHS Trust, Wake®eld, UK
Accepted date: August 2003
Following The Culyer Report into research and development (R&D) activities in the NHS, the Quality, Evaluation and Development (QED) Department of WakeŽ eld and Pontefract CommunityHealth NHS Trust (WPCHT; now part of the South West Yorkshire Mental Health NHS Trust – SWYT) started offering advice surgeries to trust staff in 1995. These surgeries provide guidance on many aspects of research (and audit)and have proved increasingly relevant, with clinical governance requiring NHS trusts to evaluate services.Similar surgeries have also been provided to NHS trusts by university departments.The focus of this article is R&D advice surgeries provided to WPCHT staff, reporting on an evaluation completed in March 2002. We examined records of the surgeries in the previous Ž financial year and conducted telephone interviews with 28 staff who had attended advice surgeries related to research projects in this period. Our main Žfindings are that attendees valued advice surgeries,and that they catered to a wide range of professionals,on a wide range of research topics. The evaluation also identiŽ ed areas for improvement,namely increased advertising, and ? exibility of time and location. We conclude that advice surgeries provide an effective way of increasingresearch awareness in trusts and contribute well to the clinical governance agenda.Comparisons are also made with similar surgeries partially funded by the NHSE and offered to NHS Trusts in the North East of England by the Nuffield Institute (University of Leeds) and the University of Teesside.
advice surgeries, clinical governance, research awareness, research governance
For some time, the need to increase research awareness and research skills among health professionals has been recognised.[1,2] Some health professionals, such as clinical psychologists, have high levels of research skills from their training, whereas others, such as nurses and allied professionals, may have very limited research skills. More recently, clinical governance has put quality high up on the clinical agenda and brings with it the need to evaluate services using research and audit methods, making both research awareness and some level of research skill a requirement of everyday clinical practice across the professions. [3,4] To help address the need to improve research awareness and skills, the Wakefi eld and Pontefract Community Health NHS Trust (WPCHT) set up a Quality, Evaluation and Development (QED) department in 1995 to support quality, audit, and research initiatives and to provide a one-stop source of advice and project management in these areas. In-house advice surgeries were set up on a weekly basis and have continued ever since. These surgeries were partly a response to The Culyer Report into research activities in the NHS, with the trust board making a commitment to research, and partly building on previous quality initiatives within the trust.[5] Advice surgeries are open to both trust staff members and to primary care trust (PCT) staff previously employed by the trust. Staff can book a session for up to one hour, and advice may be sought in all areas of the QED department’s expertise (including audit, quality initiatives, service evaluation and research). These advice surgeries have continued in the newly merged South West Yorkshire Mental Health NHS Trust (SWYT). This trust is unusual in having a high level of research skill available on site, including fi ve professorial posts at the University of Huddersfi eld (four visiting professors and one jointly funded professor), one of whom is employed in the QED department. Advice surgeries have been provided by QED staff with high-level expertise in research design, statistics and qualitative research.
To examine the effectiveness of these advice surgeries, an evaluation was commissioned by the NHS executive’s former Northern and Yorkshire Regional Offce (NHSE NYRO).[6] Attendance records were audited to examine the proportion of advice surgeries devoted to R&D issues and the professional background of staff attending the surgeries. Staff who had attended advice surgeries relating to research projects were approached to take part in a telephone interview on their experience of the surgeries. Although the focus was on R&D, many of the fi ndings reported here are relevant to advice surgeries offered on other topics, such as audit and service evaluation.
In addition to the in-house funded advice surgeries, similar surgeries have been offered to acute and community trusts by university departments. Use of university R&D expertise and infrastructure to help increase NHS R&D awareness and capacity at all levels has been increasingly encouraged in government initiatives (e.g. Building a Research Conscious Workforce 7). This study includes a brief evaluation of two such initiatives offered by the University of Leeds’ Nuffield Institute for Health and the University of Teesside’s Centre for Health & Medical Research. Both initiatives were partially funded by the NHS Executive. Since the Funding came to an end, the Nuffield Institute service has been discontinued, although telephone advice is still offered to NHS staff as required, and the University of Teesside service has continued in a different form, funded by the trusts that made most use of the service. In the discussion section, we compare these two services with our own, drawing on reports prepared by the two universities and interviews by two of us (EW and RN) with some of the staff who offered the services.[8,9]
The Trust’s QED department advice surgeries were evaluated in three ways.
1 Advice surgery diary and attendance forms allowed basic information about the sessions to be collated. Advice surgery appointments are generally made by telephone and are booked in a diary. In addition, after each appointment the advice giver routinely completes an attendance form detailing the content of the session. These were examined in our evaluation, with 53 members of staff identifi ed as having attended 96 sessions relating to R&D between 1 April 2000 and 31 March 2001. The diary and attendance forms were also used to collate information on the type of advice sought, area of work and profession of the people seeking advice.
2 Telephone interviews with a sample of staff who had attended R&D advice surgeries allowed a fuller evaluation. The interviews used a semi-structured questionnaire to assess the effectiveness of advice surgeries and to ask for suggestions on ways they could be improved. Of the possible sample of 53, no telephone number or other contact details were located for 11 attendees, but 28 (67%) of the remaining 42 were successfully contacted to complete the questionnaire.
3 Six case vignettes were constructed from completed projects that benefi ted from advice surgery input to illustrate the range of projects and the levels of help offered, as well as the impact on the career development of both the attendees and the advice surgery providers.
Analysis of diary and attendance forms
Our evaluation showed that the advice surgeries are used to provide R&D advice across a range of topics. The advice surgery content is led by the attendee’s needs, and during each session advice can be sought on multiple topics. For the 53 attendees in 2000–2001, the most frequent topics for advice were research design (74%), statistics (39%), dissertation advice (38%) and gaining Ethical Approval (34%) (see Figure 1). Advice surgeries were also found to cater to a wide range of professionals. Figure 2 shows that the majority of attendees (51%) were nursing staff, reflecting their proportion (57%) in the overall workforce. Clinical psychologists and medical staff attended proportionally more advice surgeries than other professionals, perhaps due to the high emphasis on evidence-based practice in their training. Figure 2 also shows that our telephone sample reflected the proportions of staff from the different groups, with professions allied to medicine slightly over-represented in the sample.
Figure 1: Topics on which advice was sought (n = 53)
Figure 2: Advice surgery attendance by profession
Telephone sample results
In general, we found that the surgeries are well attended and are valued by those attending. From our 28 respondents, 39% had attended an advice surgery on more than one occasion, one as many as nine times. Nearly all the respondents (96%) found the advice helpful. In fact, only one person (4%) said that the advice had been unhelpful, although every one of the respondents (including this individual) said they would be happy to use an advice surgery again. Respondents were also asked to specify the ways in which they found the advice helpful: 93% agreed that it provided clarifi cation, 81% agreed that it helped with the focus and direction of their project, and 67% agreed that it provided explanation, information, and specialist knowledge.
We were also interested in fi nding out what attendees do following the advice surgery. Only four of the 28 (14%) decided either not to go ahead with their original idea or to do something else. The rest carried out their project either on their own or with help from QED, the vast majority saying that the surgery had helped them continue with their research. Three people in our sample said they had submitted Funding bids, two of which were accepted (with one pending). In addition, two people had given a presentation at a conference, and two had presented a poster at a conference. Although there had been no publications in our sample, two respondents said they intended to publish their fi ndings, and two others said they were considering it.
Respondents were also asked how they had found out about the advice surgeries. Sources varied, the majority saying they had found out about them via a QED-related source, or a work colleague. This perhaps highlights the need formore explicit advertising, and indeed more than two-thirds (68%) of respondents felt that advertising could be improved. Suggested improvements included increased use of posters, leaflets and flyers, as well as having a frequently asked questions (FAQ) section on the trust intranet. Since completing the evaluation, the department has produced a leaflet and posters, and further methods of advertising are being considered.
Our evaluation also assessed respondents’ views of the accessibility of advice surgeries. Advice surgeries are currently provided on the main site, usually at a fi xed time in the week (Wednesday afternoon). Thirty-two per cent and 21% of respondents requested greater flexibility of time and location respectively. Many of these requests were from staff based off the main site.
The six case studies in Table 1 illustrate the range and level of research carried out in the trust that has been supported by advice surgeries. These have been selected fromcompleted research that relied on advice surgery attendance at some stage. For this reason, the sample is drawn retrospectively, so includes people attending prior to the year 2000–2001, reflecting the time taken to complete and publish research. We emphasise here the impact of advice surgery attendance on the careers of both those initiating the research and the researchers giving advice.
Table 1: Case vignettes of South West Yorkshire Mental Health Trust research helped by R&D advice surgeries
The examples given here are all small-scale projects carried out by clinical and research staff without external Funding. They all arise from clinicians questioning and wishing to evaluate day-to-day clinical practice. This is one of the core requirements of clinical governance, using research and audit activities to evaluate and improve the quality of care. The advice surgeries have facilitated this by providing on-site help at short notice from staff with research expertise who also have local knowledge of the clinical services.
In 1995, when the R&D advice surgeries were set up in the then Wakefi eld and Pontefract Community Health NHS Trust, sources of advice provided on site by members of the trust’s staff with research expertise were rare, particularly in areas outside teaching hospitals. The surgeries have stood the test of time and have continued into the recently merged mental health trust. The results presented here for a single year are typical of the less formal feedback from staff received on a regular basis. Staff value the surgeries, which help the clinical governance agenda by helping clinical staff evaluate their services in a structured way. They also help involve staff in R&D and can have a major impact on career development for staff at all levels, as illustrated in Table 1. The major criticisms have been about advertising and flexibility of timing of the surgeries. Advertising has been improved, but the fact that it continues to be raised as an issue on a regular basis partly reflects the turnover of staff in a large trust, despite presentations at induction days on the services offered by the QED department. Flexibility is, again, something that we try to address, but it is always difficult given a small infrastructure of research support and the need for research staff to deliver in other areas, including grant income and publications. SWYT, however, is unusual in having a core of very skilled research-active staff, so the degree to which the service offered could be repeated in other trusts would need careful consideration.
University-provided R&D surgeries
An alternative, but complementary, model for R&D support is to bring expertise from university departments into the NHS. This is recommended in the Building a Research Conscious Workforce (BRCW) initiative,7 and we briefly compare here two examples from the North East of England partially funded by the former NHS Northern & Yorkshire Regional Office.
The Nuffield Institute offered a package of advice surgeries and workshops to fi ve local NHS trusts, with a reasonably even split between acute and primary care professionals using the advice surgeries. The trust making most use of the surgeries was a combined acute and community trust, while the teaching hospitals were reported not to be big users of the service. Nuffield staff were available on site at allocated times and trust staff could turn up and knock on the door. Since the external Funding ceased, the Nuffield Institute stopped providing on-site advice surgeries to trusts. However, because of the Institute’s previous involvement, telephone and email enquiries are still received, with initial advice given free of charge but sustained advice charged to the researcher’s trust.
The majority of professionals seeking advice were doctors and nurses. The time slots in the funded surgeries were reported not to be used efficiently, with often only one person attending during the whole surgery. Clinicians said they found the allocated times unhelpful, as they could not guarantee to be free at those times. Consequently, most advice was sought by telephone. In addition, many people asked R&D questions at the end of workshop sessions held in the trust. The majority of the advice sought was handled by a senior lecturer at Nuffield with expertise in both qualitative and quantitative research methods, together with about six other key advice givers. An advantage was that the Nuffield Institute could draw on a large base of academics at the university to provide advice as necessary. The advice given was generally at the intermediate level (training level 2) of the BRCW model, with occasional provision of advanced advice.[7] The most frequently sought advice was about questionnaire design and statistics, with additional regular queries about sample size and power, research Funding, research design and data collection. Occasionally, queries were received about setting up databases and writing for publication. People generally sought advice more than once for the same research topic. Surgeries were reported not always to be used in the best way, with researchers sometimes attending at too late a stage, with badly designed questionnaires already distributed, so that little could be salvaged. However, others were able to use the surgeries appropriately to work through preliminary ideas before the design and instruments had been fi xed. In one evaluation of a 12-month period of advice surgeries, three Funding bids were submitted (and awarded) following attendance, and one article was submitted (although the outcome for this was unknown).
University of Teesside’s Centre for Health & Medical Research’s R&D support clinics
A full evaluation of the R&D support clinics offered by the University of Teesside and funded by the NHSE NYRO is available in their fi nal report and will be summarised only briefly here.8 As with the Nuffield scheme, support clinics were offered on-site to fi ve local trusts, with a mix of acute and community trusts. Teesside University was also able to draw on a wide variety of experts to provide advice, including medical statisticians, qualitative researchers and ethics experts. Advice was reported to cover the whole spectrum of the BRCW model, from basic to advanced.[7] The most frequently sought advice was about statistics, data collection, questionnaire design, Funding, formulating the question, data entry, fi nancial management and audio transcriptions, with about two-thirds of advice classifi ed as quantitative, 10% as mixed methods and 4% as qualitative. It was common for staff to attend more than once, with 78% of appointments being fi rst appointments and 22% follow ups. The largest professional group attending the clinics was medical staff (45% overall, but 77% in one acute trust), with nurses attending 28% of clinics overall. Although the acute/community mix was not recorded systematically, it was thought that most attendees came from acute areas. As with the Nuffield Institute, there were examples of good and poor use of clinic time, with poor use exemplifi ed by researchers attending too late in the process and good use exemplifi ed by professionals coming early in the research process with good ideas but no direction. In the 12-month period evaluated for the fi nal report, approximately 15 research Funding bids (predominantly to external funders, but with some for internal university Funding) were submitted on the back of advice provided at clinics, with half of these being successful. It was unknown how many publications had been submitted following advice from the clinics, but there had been some successes, with university staff able to put their names on articles that came about in this way.
A key problem identifi ed by the Teesside staff and also echoed by the Nuffield Institute was poor advertising of the clinics within the trusts. Teesside has responsibility for advertising its own internal clinics, with reminder emails sent to all staff a week before each clinic and clinics also advertised in delegate packs for their annual postgraduate conference and on the School’s intranet. For the clinics provided to trusts, the trusts took responsibility for advertising and there was little evidence of effective advertising.
When the Funding ceased, two of the trusts continued to use the service, paying for one clinic a month, with half a day providing surgery advice and the other half spent carrying out work requested by trust staff (e.g. data analysis). One other trust makes occasional use of clinics, paying by the session. Internal clinics are currently offered to all staff from Teesside University’s School of Health and Social Care, and the university also offers a free one-hour appointment to NHS or social care staff.
Comparison of the three approaches
Although the three services target slightly different professional groups, with most of the SWYT advice surgeries focusing on the needs of community and particularly mental health staff and the two university examples serving a larger proportion of medical staff in acute settings, it is clear that the services provided and the aims are very similar. Key similarities and differences are highlighted in Table 2.
Table 2: Comparison of R&D support o¡ ered to NHS sta¡ in selected trusts
The evaluations make it clear that R&D advice surgeries are valued by the trusts hosting them, which help increase R&D activities, including publications in peer-reviewed journals and Funding bids. Trusts value on-site advice, freely available from advisors with a range of R&D expertise. Advertising on-site is often limited and would benefi t from regular attention. Flexibility of advice surgeries is also valued.
A good idea from the University of Teesside is the recently introduced proposal development meeting. Mainly available to university staff, but occasionally used for external clients, these meetings consist of a team of advisors meeting with the professionals seeking advice for Funding bids to help them improve their proposals and complete the costings. Bids are now reported to be of higher quality and more likely to gain Funding.
• Advice surgeries offered by experienced researchers on the trust site appear to be an effective way of increasing research activity in NHS trusts, an important component of clinical governance.
• The vast majority of the people interviewed were happy with the R&D advice they received and found it helpful in a number of ways.
• Advice surgeries are an important component of Building a Research Conscious Workforce and help motivate staff to undertake and continue research.[7]
• Advice surgeries are a key element of Research Governance, improving the quality of research.
• Although flexibility was identifi ed as an area for improvement, it may be difficult to improve due to limited resources.
• This evaluation highlights the value of advice surgeries, and hence suggests the need for similar services to be established within other NHS trusts. The South West Yorkshire Mental Health NHS Trust solution has been to draw on the high levels of research skill available in the trust. Other trusts with less expertise available on-sitemay need to use external (e.g. university) resources for such purposes and the two services reviewed here (Nuffield Institute and University of Teesside) have been valued by trusts.[8,9]
The evaluation reported here was partly funded by a grant from the NHS Executive Northern and Yorkshire Regional Office. We would like to thank Keith Hurst of the Nuffield Institute, University of Leeds, and Professor Tracey Howe,Vicki Ashton and Andrea Cook of the Centre for Health and Medical Research, University of Teesside for agreeing to be interviewed. We would also like to thank Lesley Mackay and Annette Lankshear of the National Health Service Executive Northern and Yorkshire Regional Office for continued support and advice throughout the project. Thanks are also due to the staff who took part in the telephone survey.
