Research Paper - (2010) Volume 18, Issue 4
Professor of Health Innovation, Curtin Health Innovation Research Institute, Curtin University of Technology, Perth, Australia
Carolyn O’Shea MBBS (Melb) MMed (Melb) Grad Cert Health Prof Ed (Monash) FRACGP
Senior Lecturer, Department of General Practice, Monash University, Melbourne, Australia
Gareth Merriman PhD
Program Leader (Sexology), Senior Lecturer (Sexology), School of Public Health
Georgia Halkett PhD FIR
Senior Research Fellow, WA Centre for Cancer and Palliative Care
Katrina Spilsbury PhD MBioStats BSc
Senior Research Fellow, Centre for Population Health Research, School of Public Health Curtin Health Innovation Research Institute, Curtin University of Technology, Perth, Australia
Received date: 25 January 2010; Accepted date: 2 June 2010
BackgroundPsychosexual problems are a common presentation in general practice. Given that the cornerstone of assessment is excellent consultations skills, it may be assumed that general practitioners (GPs) will perform skilfully for such presentations.AimTo determine if there is a significant difference in consultation skills assessed using a generic test of consultation performance compared to one which has been specifically developed for experts in psychosexual care, albeit modified for general practice consultations. MethodsSix GPs were video recorded consulting six standardised patients at their respective practices.Two assessors independently rated the consultation performance using the Leicester Assessment Package (LAP), a generic tool to assess GP consultation performance. Four sexologists, blind to the review by the LAP assessors, assessed the same consultations deploying the Permission, Limited Information, Specific Suggestion, Intensive Therapy (PLISSIT) framework. The PLISSIT is routinely used to teach health professionals communication skills when consulting people with psychosexual problems.ResultsThirty-four consultations were successfully recorded. The mean duration of consultations was 12 minutes and 10 seconds (range 7 m. 54 s. to 16 m. 54 s.). Three GPs differed significantly in core competencies as measured by the LAP. Similarly, three GPs differed in competencies as measured by the PLISSIT. There were significant differences in mean LAP scores and PLISSIT scores observed for different doctors. Mean LAP scores varied by actor–scenario after adjusting for doctor clustering, whereas PLISSIT scoring did not vary significantly by actor–scenario in this small study with limited power. There was no evidence that mean LAP scores were associated with PLISSIT scores. ConclusionsTwo measures of consultation competence revealed different outcomes when applied to the same consultations. We found evidence that general practitioners vary significantly on different measures of consultation competence when consulting patients with psychosexual problems in the context of a cancer diagnosis.
cancer, consultation skills, psychosexual problems
One in four Australians will develop cancer before the age of 75 years.[1] Most patients will survive and eventually die of a condition unrelated to cancer. The trajectory through the illness will be punctuated by surgery, chemotherapy and radiotherapy which in some cases will impact on sexual function. Many people will be supported during their illness by an intimate partner. During treatment those couples who have stopped being sexually active may be ambivalent about the prospects of a return to sexual relationships once treatment is complete. Previous studies have reported that cancer patients are often fearful about resuming intimate relationships because they are unsure whether they will be able to perform, if there will be pain and whether they are still attractive to their partner because of the effect of the cancer and/or treatment.[2,3] It has been suggested that people with cancer should be encouraged to consult their doctors about prob-lems with sexual function.[4] However, people report difficulties communicating their sexual problems to doctors especially within the context of a cancer diag-nosis.[5–7] To our knowledge we are the first to inves-tigate how doctors score on two different measures of consultation competence when consulted by patients with sexual problems.
There are an estimated 100 million consultations with GPs in Australia annually.[8] Most consultations last no more than 15 minutes and may involve dealing with multiple problems. Therefore it is important that the doctors have the expertise to communicate effect-ively and determine how the patient can be best sup-ported or treated. Viewing the consultation through Watson’s ‘caring lens’, a critical element for the outcome of the consultation is the quality of the interaction between the patient and the care giver, in this case the GP. The doctor must exhibit empathy and a willingness to consider the problem from the patient’s perspec-tive.[9–11] Second, when doctors are distracted by mul-tiple problems presented at a single consultation, especially when one of these is a potentially life-limiting condition, they may fail to effectively address other issues which are also a priority for the patient.[12] The aim of this study was to determine if there is a significant difference in consultation skills assessed using a generic test of consultation performance and one which has been specifically developed for experts in psychosexual care, albeit modified for general practice consultations.
Design
In this study we are building on lessons learned from conducting recorded standardised patient consulta-tions, as reported previously in this journa1.[13] The study involved video recorded consultations with gen-eral practitioners, in which five out of six actors portrayed people with cancer. One case did not involve a cancer diagnosis in order to introduce variety among the cases presented. The consultations took place on the premises of the participating general practitioners in Perth, Western Australia. Patients presented in the same order to each practitioner.
Actor patients
The scenarios were developed by the members of the team and are outlined in Box 1. Physical signs, presented as descriptions, were available if the GP proposed relevant physical examination. Therefore no actor was subjected to a physical examination during the study. Patients were amateur actors trained to portray the relevant case. A brief medical record with the relevant past medical history was prepared for each patient and was available to the GP. The GPs were aware that the ‘patient’ presenting to their clinic was an actor.
Six GPs were invited to consult with the actor-patients as though the person had previously visited the prac-tice for one or two ongoing medical problems – brief medical records were provided. The practitioners were allowed up to 15 minutes per consultation as per routine practice in Australia. The scenarios were pre-sented to the practitioners as consecutive cases. GPs were asked to make clinical notes and outline any management plan in as much detail as they would in their practice. GPs were informed that the study involved patients presenting ‘intimate relationship problems’.
Quality of consultation
The Leicester Assessment Package (LAP) is an estab-lished measure of quality in general practice consul-tations. With training, general practitioners can use the LAP framework to assess a doctor in various categories of competence in communicating with patients and offering a management plan for the problems presented. Its validity has been confirmed for general practice consultations.[14] Five of the seven LAP categories of consultation competence (inter-viewing and history taking, problem solving and patient management, anticipatory care and behaviour/rela-tionship with patients) were assessed in this study. The recordings were independently reviewed by two general practitioners (COS and MJ). The team had previously assessed consultations using the LAP.[15,16] The scores were then compared and the final scores represented the consensus view on the quality of the consultations. This method is consistent with how con-sultations are assessed during professional examin-ations. The competencies of the six GPs in this study were compared to those of practitioners participating in a similar study with standardised patients, which was also scored using the LAP.[16]
Annon introduced the PLISSIT framework to guide practitioners on the most effective strategies when discussing sexual issues and to facilitate communi-cation when there is a need to refer for more spe-cialised treatment/assessment.[17] The application of the PLISSIT framework has previously been shown to be beneficial to patients.[18] Several authors have extended or adapted the framework of PLISSIT to more accurately assist in the assessment and treatment of specific difficulties.[19,20] For the purposes of this study four trained sexual health counsellors, including one who was a GP, assessed the consultations using the PLISSIT framework. The four counsellors reviewed the consultations independently and compared their scores with colleagues. Discrepancies were discussed and a final score was agreed as a measure of the performance of the doctor in each of the four do-mains, weighted for their opinion about the import-ance of each domain. The four components of the framework are summarised as follows.
1 Permission
Many people experiencing sexual dysfunction may be reluctant to discuss details of their sexuality in a medical consultation. It is helpful to communicate that the practitioner is open to discussion of what may be perceived to be an embarrassing problem. The coun-sellor is encouraged to demonstrate active listening and provide acknowledgement, support and reassur-ance. This domain of the PLISSIT is consistent with history taking as assessed by the LAP. The assessors in this study weighted this element of competence highest on the PLISSIT scoring schedule (Table 1).
2 Limited Information
Patients with sexual health problems may be har-bouring misconceptions about the extent to which the problems they are experiencing are common to others. This may inhibit disclosure about the nature of the problem. Those advising patients are encouraged to give ‘limited information’ in order to encourage further disclosure of what the patient may consider embarrassing details. This information serves to pro-vide a level of normalisation, dispel misconceptions and to lead to the sharing of accurate information. This is also consistent with history taking and problem solving as assessed in the LAP.
The practitioner is encouraged to provide specific suggestions to address the problems. These suggestions range from what the patient can do for themselves and may include pharmacological and/or psychological interventions (e.g. prescriptions, exercise, multimedia aids). This is consistent with the problem solving domain assessed in the LAP.
The practitioner must demonstrate skill in offering specialised assessment or treatment. This element of the framework requires particular expertise in the management of human sexual dysfunction. The psycho-sexual counsellors deemed this component to be much less important in the general practice consultation as it was considered inappropriate for a general practi-tioner to be engaged in ‘Intensive Therapy’ within the 15 minutes available and on the first occasion that the patient presented the problem. This element was weighted least on the overall assessment, as shown in Table 1.
Statistical analysis
The mean LAP and PLISSIT scores were estimated by the two sets of assessors for each doctor. The differ-ences in mean scores between individual doctors were estimated in a standard unadjusted linear regression model. Intraclass correlation coefficients were used to indicate whether there was a high degree of correlation of doctor LAP and PLISSIT scores. In order to deter-mine whether the mean LAP and PLISSIT scores varied according to each of the actor-patient scen-arios, generalised estimating equations were used to fit a linear model actor-patient scenario as the single independent variable. Robust standard errors with a small sample adjustment and clustered on the doctor were estimated. The small sample size of this study resulted in sufficient power to test for only large differences in the mean LAP and PLISSIT scores. Power calculations for correlated data, using simulations to avoid an underlying assumption of normality, showed there was 90% power with an alpha level of 0.05 to detect a mean difference in LAP scores between doctors of 9, assuming a standard deviation of 7 and ICC of 0.75. Similarly, for the more variable PLISSIT scores, there was 90% power to detect a mean difference as small as 12, assuming a standard deviation of 10 and ICC of 0.75.
To determine if the LAP scoring methodology was comparable to PLISSIT scores a Bland–Altman plot of the standard deviation against the mean was gen-erated; that is, LAP and PLISSIT scores were plotted against mean scores (LAP + PLISSIT)/2. Pitman tests of difference in variance were performed. Similar comparisons were made with the four domains of the PLISSIT scores.
Thirty-four consultations were successfully recorded. The mean duration of the consultations was 12 min-utes and 10 seconds. One doctor’s recordings failed in two cases. The overall mean LAP score was 59.8 (SD 6.1). There was no evidence that the GPs in this study scored significantly differently from six other GPs in a similar actor-patient consultation study in which the mean score was 62.6 (SD 13.6, p=0.5716).Agreement by assessors on GP LAP scores was generally good. The assessors were from similar practice backgrounds (the UK and Australia) and had similar experience in conducting assessment, but different seniority as practitioners (5 vs 15 years). Close attention was paid to calibration of the assessors, with each assessor scoring the consultations independently and then comparing scores to focus on areas of disagreement in order to arrive at a consensus score. The mean PLISSIT score was 64.3 (SD 13.8). There were significant differences in both LAP and PLISSIT scores between the doctors (Table 2). Although the highest and lowest LAP and PLISSIT scores were obtained by the same doctors, there was variation in relative scores amongst the remaining doctors.
Strong within-doctor (using intraclass correlation coefficients) correlations for LAP scores (0.79) and for PLLISSIT scores (0.76) were observed. In order to determine whether the mean scores varied for each of the actor scenarios, generalised estimating equations were used to fit a linear model that took this within-doctor correlation into account (Table 3). Individual actor-patient scenarios were observed to be predictors of mean LAP scores but not mean PLISSIT scores in this small sample, although the study was only suf-ficiently powered to detect large differences in scores.
A Bland–Altman plot of the difference between LAP and PLISSIT scores plotted against the mean scores shows that there was poor agreement between LAP and PLISSIT scores in this small study (Figure 1). The limits of agreement (mean 2SD) are wide and range from -27.4 to 17.9, reflecting the small sample size and the large variation in differences between the two scales. There is also evidence of an asymmetric bias (Pitman test P=0.002) and as indicated by the slope of the regression line of the difference between scores and mean scores. At lower mean scores the LAP scores are higher than the PLISSIT scores, whereas this reverses as the mean score increases, with PLISSIT scores higher than LAP scores.
Figure 1: Bland–Altman plot of the difference between LAP and PLISSIT scores plotted against mean scores
It was observed that total LAP scores showed better agreement with two domains of the PLISSIT scores compared to the overall PLISSIT scores. The mean difference in LAP score and the ‘Permission’ domain of PLISSIT was 33.9, with narrower limits of agree-ment of 24.3 to 43.4 and a Pitman test of difference in variance p-value of 0.372. For the ‘Limited Infor-mation’ domain, the mean difference in scores was 36.8 with limits of agreement ranging from 26.7 to 46.8 and a Pitman test of difference in variance p-value of 0.301.
There were significant differences in LAP scores and PLISSIT scores when we analysed the data by doctor and by scenario, suggesting that the participating doctors had different levels of performance and that the two different measures deployed assessed different things. We noted that doctor 1 ranked highly on PLISSIT but did not perform as well on LAP. If such variability is confirmed in a larger study then one might conclude that generic measures of GP consul-tation performance are not necessarily appropriate in some circumstances as they may not detect significant deviations from the mean. In our data the LAP assess-ment appeared to have some association with the type of actor–scenario, whereas the PLISSIT scores were more homogenous, although this study was only sufficiently powered to detect large differences between the scores. There was evidence that the ‘Permission’ and ‘Limited Information’ domains of PLISSIT showed better agreement with LAP scores than did other domains. This is entirely consistent with the fact that the LAP domains assessed in this study included history taking skills.
The approach we took to exploring the issue of management of psychosexual problems has several strengths; we were able to replicate conditions that might be difficult to control or observe in clinical practice and the practitioners all saw the same patients in the same sequence.[16] In many ways the method-ology involving actor-patient consultations mimics the formal assessment or examination of candidates seeking membership to many professional colleges and has been previously demonstrated.[21] As in pre-vious studies, the participating GPs were volunteers and perhaps unrepresentative. That alone was not con-sidered a major limitation in the design of this study, which was intended to compare competencies meas-ured on two different measures. There was no evi-dence that the GPs in this study performed better or worse than practitioners in a similar recorded, stand-ardised patient study but we have no measures of how these practitioners perform in routine practice outside the study using these or other measures of compe-tence.[16] We are therefore unable to confirm how well their performance here reflects their competencies when ‘real’ patients consult them. We also acknowl-edge that the PLISSIT assessors were mostly experts in psychosexual care, although one assessor was also a GP. Assessment by a predominantly specialist team may have introduced some bias into the analysis of the consultations. However, it should also be noted that the PLISSIT scoring schedule was weighted to take account of the fact that GPs do not generally provide psychosexual therapy in the course of a routine con-sultation.
The GPs were also aware that the ‘patient’ pre-senting to the clinic was an actor and that a formal examination would not be required. While incognito presentations are sometimes recommended for stand-ardised patient studies, primarily because they reduce the perceived loss of ‘reality’ for the GPs, such pres-entations when focusing on psychosexual problems were deemed unethical.[22] We were unable to assess the impact of observation on the GPs’ performance although the literature on video recording for the purposes of assessment suggests that it has no signifi-cant adverse effect.[23] Finally, as investigators in the study, the assessors could not be blinded to the aims of the study. We do not believe this had an impact on the scores; however, it would be prudent to deploy as-sessors who could successfully be blinded at the time of reviewing the consultations.
Psychosexual problems are a common presentation in general practice. Consulting patients with psycho-sexual problems requires excellent consultations skills. Our data suggest that it cannot be assumed that GPs will display the same level of skills on different measures. When the subject of variable performance relates to the management of a physical symptom such as rectal bleeding or chest infection then experts other than GPs may have a role in the education or assess-ment of GP competence.[24,25] We propose that appro-priate management of psychosexual problems requires similar multidisciplinary input. We also believe it is important and interesting that measures of com-petence in psychosexual care demonstrate variety in the skill set. These data are a prelude to interventions that may be helpful. The PLISSIT framework has demonstrated variable performance, and like the LAP offers a framework for elements of the consultation that may be the focus of training.
We report evidence that the GPs in this study were deemed to have varying skills when assessed using two different measures. It is unsafe to assume that GPs who display satisfactory consultation skills by one generic measure would be rated equally well when assessed using another specialised schedule.
Dr Moira O’Connor, Dr Marthe Smith and Ms Hayley Arnet for support with the management of this study.
Western Australian Cancer and Palliative Care Net-work.
This study received Ethical Approval from HREC at Curtin University of Technology (RD-57–07).
Not commissioned; externally Peer Reviewed.
None.
