Research Paper - (2014) Volume 22, Issue 2
PhD Candidate
Marjan J Faber PhD
Senior Researcher
Gert P Westert PhD
Professor and Director
Scientific Institute for Quality of Health Care, Radboud University Medical Centre, Nijmegen, the Netherlands.
Glyn Elwyn BA MB BCh MSc FRCGP PhD
Professor, The Dartmouth Center for Health Care Delivery Science, and The Dartmouth Institute for Health Policy and Clinical Practice, Hanover, New Hampshire, USA
Joze´ C Braspenning PhD
Associate Professor, Scientific Institute for Quality of Health Care, Radboud University Medical Center, Nijmegen, the Netherlands
Received date: 1 December 2013 Accepted date: 31 January 2014
Background In many countries, market orientation in healthcare has resulted in the publication of comparative performance information (CPI). Most of the research in this field is oriented towards the content and the presentation format of CPI while little is known about how consumers value CPI and the use of this information. Aim The aim of this study was to clarify the perceived value that CPI brings for consumers of healthcare.Methods Qualitative research using six focus group interviews. Twenty-seven healthcare consumers were recruited using a mailing list and by personal invitation. Data from focus group interviews were transcribed and thematic analysis undertaken. Results Most participants were unaware of CPI, and valued alternative sources of information more than CPI. Through discussion with other consumers and by means of examples of CPI, respondents were able to express the values and perceived effects of CPI. Numerous underlying values hindered consumers’ use of CPI, and therefore clarification of consumer values gave insights into the current nonusage of CPI. Conclusions CPI is marginally valued, partly because of conflicting values expressed by consumers and, as such, it does not yet provide a useful information source on hospital choice beyond consumers’ current selection routines in healthcare. Future research should be more focused on the values of consumers and their impact on the use of CPI.
comparative performance information, consumer values, choice behaviour, qualitative re-search
Provider choice for healthcare consumers has emerged as a key policy focus in modern Western healthcare systems. The architects of the policy outlined several reasons to promote choice among healthcare con-sumers. First, to encourage competition between pro-viders which was expected to improve the efficiency and quality of healthcare;[1,2] and second, to increase patient empowerment and improve the position of consumers in healthcare.[3,4] It is based on the expec-tation that patients will choose the best healthcare provider once they are informed.
Systematically collected, publically available infor-mation about the performance of healthcare pro-viders, called comparative performance information (CPI), can be used as a tool to inform healthcare consumers to enable them to make informed provider choices. The information consists of, for example, patient volumes, treatment methods, waiting lists and patient experiences, and can be found in leaflets, magazines, books and on the internet.
Despite the efforts and resources that go into the collection, production and dissemination of infor-mation for the public,[5] there is little evidence that CPI affects consumer choices. Many people do not understand the information, or do not view the information as useful.[6] British and American studies have shown that as few as 5–7% of consumers use CPI to choose a hospital[7,8] and only 12% use CPI to select a primary care physician,[9] although awareness of the right to choose a provider has increased. The overall conclusion is that consumer usage of CPI to make an informed choice is still limited. Instead, consumers rely on information from family and friends or base their choice of hospital on recommendations from their referring primary care physician.[10,11] The choice is largely driven by familiarity with a certain hospi-tal[12,13] or the distance between home and hospital.[12,14 ]Using CPI is also a difficult and complex task for consumers, which limits its impact on consumer choice even more.[14,15]
Because provider choice by healthcare consumers is positively associated with autonomy and self-deter-mination, both important principles of patient acti-vation,[16] the need to explain the limited exercise of choice by healthcare consumers might improve our understanding of active patient participation in healthcare.
Previous studies have questioned whether this lim-ited usage of CPI is caused by the absence of the right information content[17] or whether CPI presentation formats do not support decision-making tasks.[7] Evi-dence for both assumptions has been found. Damman and colleagues conclude that the presentation of CPI facilitates consumers’ correct interpretation as well as effective use, meaning the ability to choose the best performing provider.[18] Although these are without doubt crucial elements in facilitating consumers in using such information, fundamental questions over how CPI is perceived, valued and appraised by healthcare consumers for their choice of hospital are overlooked. Some theory and evidence suggest that healthcare consumers are insufficiently informed about what is important for them.[5,19] Current shared decision-mak-ing models for clinical treatment acknowledge the need to clarify patients’ values to promote active patient participation in decision-making.[20,21] The aim of this study was to identify consumers’ values, ap-praisal, understanding, opinions and judgement of CPI. We conducted a series of focus group interviews.
The current study was a qualitative focus group study in 2009. The method allows researchers to capitalise on communication between participants to generate pertinent information.[22] The focus group method was chosen because of its flexibility to explore unanticipated issues, and to make use of the interaction between group members. We encouraged the participants to discuss the subject of CPI, ask each other questions, exchange opinions and views, and share experiences.[23] We presented the ‘values’ of CPI initially using terms such as ‘meaning’, ‘sense’, ‘ideals’ and ‘principles’ to explain their meaning for healthcare consumers. We introduced three real-world CPI examples to align the mindset of the participants in the discussion (see Table 1). The example shown in Table 1 combined stars with percentages, the second used different kinds of bullets, and the third one showed the availability of a service using ‘yes’ or ‘no’.
Our initial idea was to approach patients with a specific condition. However, a commonly cited con-cern is that patients predominantly focus on their personal context and return to the hospital where they were treated previously.[24] Also, those who are most dependent on care services and who could benefit most from a ‘good choice’ are also those who tend to have more problems associated with the capacity to manage informed choices about care providers.[25] We therefore decided to recruit a cross-section of health-care consumers from the general population. By ‘healthcare consumer’ we mean anybody who is a user or potential user of healthcare. In order to increase the potential use of healthcare for themselves, family or friends, we selected respondents aged over 35 years. Our participants responded to an invitation letter sent to a random sample of 480 people listed at a large health insurance company. We used a stratification procedure to include a representative age distribution. We sent 160 letters to people aged 35–50 years and 320 to people aged 50–65 years (ratio: 1/3 to 2/3). Because the response rate was low, an additional approach was undertaken, namely a personal invitation, made by members of the research team, to healthcare con-sumers, including people sitting in a general practi-tioner’s (GP’s) waiting room. Participants received a small incentive in exchange for their efforts. No medical information was asked for, so approval by an ethics committee was not needed.
First, an interview guide was developed by members of the research team (NK, MF, JB) and influenced by the available literature about this topic. The interview guide was used to guarantee consistency among groups. The interview guide is available in an online appendix.
Before beginning each session, the aims and methods to be used were explained by the moderator (NK). Participants were asked to give their permission for audiotaping and transcribing. At the beginning of the focus group session, the moderator attempted to create a thoughtful, non-threatening atmosphere and set the tone for the discussion. Participants were encouraged to respond to all the issues raised by the moderator or other participants. Every effort was made to create an environment that encouraged individuals to partici-pate fully in the discussion. After the introduction and the first items of the interview guide, real-world examples were explained by the moderator to encour-age involvement and to support respondents in their discussion about big themes as values. Participants were asked to look at the examples and to answer the following questions:
• Do you understand what the performance infor-mation is showing you?
• Could this information benefit you in any way when you would be in a situation to make a comparison between different providers?
• What benefit could you gain from this infor-mation?
All focus group meetings lasted about 1.5 hours and were audiotaped. In addition, an assistant made notes during each session. Data saturation, the point at which the participants provided no new information to the researchers, was reached after focus group five.
All audiotaped sessions were transcribed verbatim into a Microsoft Word[1] file. Analysis of the tran-scripts was facilitated by the use of a qualitative software program Atlas.ti.5.2 to manage the data.[25] Thematic analysis was undertaken to reveal core consistencies and meanings in the data by identifying and analysing themes, which are abstract categories of meaningful data fragments. In themes, several frag-ments, known as codes are connected to each other and recur in a patterned way.[26]
Data were collected and transcripts read thoroughly to estimate data saturation. The transcripts were read several times by two investigators (NK, MF) and the main themes were extracted. Participants’ statements referring to a particular theme were grouped by similar codes, and further explored. The analysis followed an inductive approach, which was emergent and strongly linked to the data because assumptions were data-driven. This means that the process of coding occur without trying to fit the data into a pre-existing model or frame.[27]
Statements about how many people have said something can leave readers unsure how to interpret quantitative numbers in a qualitative study. However, a relatively high frequency may also signify the im-portance of a finding. In describing something in between, we avoid actual concrete numbers and used terms such as, for example, ‘many’, ‘most’ or ‘a minority’. We present the focus groups discussion in sufficient detail supported by quotes which can be read in the boxes after every theme, to allow readers check the interpretation made during the analysis.[26] Every quotation used in the boxes is followed by the number of the focus group (FG), the gender and age of the participant.
We took several measures to ensure the trustworthi-ness of this study, including multiple methods of recruitment; multiple researches to reflect on the analysis process; multiple rounds in which data were read, analysed, compared and contrasted; project team meetings to review and explore scientific and organ-isational aspects of the project.
Thirty-seven consumers agreed to participate in the focus group interviews. Twenty people responded to the invitation letter. Seventeen consumers were recruited having been approached personally; two in the GP’s waiting room. Of the 37 participants, 27 finally participated in one of the six pre-planned focus group interviews (7 men and 20 women). The mean age of the participants was 59 years. We describe four themes that emerged from the analysis.
Most participants were not familiar with CPI as a tool to guide informed choice or to compare hospitals. Only a minority said they had seen CPI before. Although they believed themselves to be skilled internet users, most participants did not know where to find CPI. They said, ‘somewhere on the internet’, but they could not specify. Some were aware of the national website presenting CPI for hospitals in the Netherlands, a site the Dutch government initiated in 2005 (www. kiesbeter.nl).
Despite a lack of awareness of CPI as a tool to compare hospitals, most participants reported that they knew about the possibility of comparing hospitals and were aware of their free choice of providers. Nevertheless, they mostly seemed to have a low level of interest in CPI. They said that, being in good health, they were not interested in hospital choice, and they did not feel a sense of urgency to look for CPI. The participants also agreed that if they became ill, they probably would not have the time and energy to look for CPI. During the focus group meetings, the par-ticipants realised that, because of a low level of awareness, there was a vicious circle in which they continued to be unaware of the potential value of CPI to help them choosing high-quality care.
Most of the participants said that their hospital choice was fixed: essentially, they always went to the same hospital. This hospital was usually the closest one, as distance was an important choice attribute. They did not have any reason to change this routine, and questioned the added value of CPI. There were other significant sources they currently used for their de-cision making. Consumers said that their own pre-vious experiences were important and outweighed the impact of CPI. They also highly valued the advice of family and friends. The consumers trusted their own GP as a source of information. There was some disagreement about whether to follow the GP’s advice and the GP’s role in providing information. In one session, some said they would not argue with their GP’s advice because they felt it could damage their relationship with their GP.
Most participants had never seen CPI before attending the focus group interviews. They had some doubts and felt a little confused after being confronted with the CPI examples. The examples caused a variety of reactions among the participants. The inventory of these reactions, including start-up questions, was supportive for participants in the discussion that followed. At first, they gave a reaction to the examples and their usefulness in general. Some participants immediately tried to interpret what they saw, others were primarily looking to see if the examples con-firmed their ideas or their own experiences. Some consumers questioned the added value of such infor-mation.
During the discussion, reactions evolved to more specific goals and values by using the examples. Participants reported that having the ability to select a healthcare provider might involve a significant effort to find and compare information. They argued that choosing a hospital introduced a new responsibility for patients, including feelings of distrust and anxiety, since they could not foresee the consequences that they might be held responsible for. Not using CPI was sometimes explained as a strategy to prevent regret for a wrong decision.
The participants pinpointed an important paradox: the more you know, the more uncertain you become, especially if information from different sources is inconsistent. The participants were afraid of losing their trust in certain aspects of healthcare that they had previously assumed to be good.
An important theme was the importance con-sumers assigned to the reliability and trustworthiness of the information. Respondents complemented each other in listing conditions CPI must fulfil before it became of value for them and before they would start using this information. The discussion included the objectivity of the information, how the information was collected, which groups of patients were com-pared, and at what level the CPI was published. Most participants stressed the importance of the reliability of CPI, but at the same time they found it difficult to determine on what grounds CPI could be considered reliable and trustworthy. Our participants wished to see that sources presenting CPI would include dis-claimers about reliability aspects and declare any conflicts of interest. Also, up-to-date information was of much value for consumers. The remarks regarding to this can be summarised with the question: ‘How do I know if what was good then is good for me right now?’ Some participants wanted a single composite indicator of overall quality to compare hospitals rapidly and easily. Other participants stated that the level of aggregation was too general in the real-world examples. The examples referred to an entire hospital, whereas participants wanted to have information at the level of departments, e.g. cardiology and ortho-paedics, or at the level of individual doctors.
The values identified stimulated the participants to mention possible effects of CPI: patient empowerment, waste, freedom of choice, benchmarking, changed perceptions, and a dichotomy in society. Patient empowerment was mentioned in several sessions, so was the counterpart that CPI is a waste. Some respon-dents found that using CPI to choose providers went beyond what is necessary in healthcare in terms of good care. Increasing freedom of choice was noted as a positive effect, as well as an attitude among some participants who felt their perceptions of quality of care had changed. They declared raising more aware-ness about the quality of care after the recent intro-duction of choice for consumers. Consumers supported the effect of benchmarking, so that professionals and organisations could compare each other in terms of quality of care. Finally, some participants had con-cerns about whether using this information might cause a further dichotomy in society by increasing inequity in healthcare. They foresaw that the ability to use information required skills of healthcare con-sumers for which some would be better than others.
This study explored the values, thoughts, understand-ing and evaluations of CPI for hospital choice among healthcare consumers in the Netherlands. By means of real-world examples, healthcare consumers were able to express their views about CPI. The four themes (awareness, current sources of decision-making, value of CPI and perceived effect of CPI) from the data suggest that there are numerous underlying but con-flicting values, which are important for healthcare consumers relating to their use of CPI. The CPI was only marginally valued, due to consumers’ values during their processing of CPI, and wider principles that limited consumers even considering the use of the information.
Most participants were unaware of CPI, and did not use this kind of information. Therefore, participants could not give a direct answer when we asked them what kind of information they would like to have or what matters most to them at the beginning of the focus group sessions. It seemed that the participants had never reflected on the information they would like to have while comparing hospital care. This is a general finding in judgement and decision-making research because decision-makers often do not know their own values.[28,29] The examples were needed to clarify information preferences and led to active debates during the focus group session and to a deeper understanding of the consumers’ values around CPI. In Moser and colleagues’ study, the use of concrete examples led to similar positive results.[30] Hibbard et al,[5] stressed the need for consumers to develop a better understanding of quality of care and current measures of quality. A recent study in clinical deci-sion-making[31] focuses on the social influence of interaction in decision-making. The concept of ‘shared mind’ as an underlying process for clarifying individ-ual values between two or more people, in which new ideas and perspectives emerge, corresponds with our findings using focus groups. Epstein also underlines the power of multiple perspectives of patients, family, physicians or other members of the healthcare team.[31]
Our study showed that consumers expressed values on several levels and that these values sometimes conflicted with each other. Some values were in favour of the use of CPI, while other doubts, concerns and principles negatively affected consumers’ views of CPI. Participants relied on previous routines such as consulting GP, family, friends, and personal experi-ences as the basis for choosing a hospital.[32,3]Nonetheless, they were also keen on having a choice. Respondents also appreciated the increased trans-parency in care, and the effect on patient empower-ment.[34,35] However, the responsibility was difficult, and participants felt that choosing a hospital was a bit of a burden. Fear of disrupting existing relationships was another consideration that prevented people from using CPI to choose a hospital. The relationship with their GP was of much more value to them, which corresponds with UK and Dutch studies.[35–37] A new perspective that was raised in our study is that not using CPI was explained as a strategy to prevent regret for a wrong decision as participants could not foresee the consequences of using CPI. Trusted others can help them to clarify the possible consequences and compare this with personal values. Finally, as in other studies our participants placed much value on reli-ability and distrust the current CPI.[24,30]
The use of CPI by healthcare consumers is a complex process in which values, rather than ration-ality, play an important role. Our findings show that consumers need help from others to solve conflicting values, to develop a firmer understanding of the quality of care concept, and to move forward in making active and informed hospital choices.
Our study adds to the understanding of the role of values, appraisal and judgement of CPI among Dutch consumers. However, the study has a number of limitations that limit the generalisability. First, the small sample size. We intended to include more par-ticipants, but recruitment was problematic, despite the use of various strategies. Owing to this difficult recruitment, we were not able to achieve our planned age distribution for the study. Moreover, 25% of the recruited participants did not attend the focus group session, underlining our finding that current CPI is valued marginally. Second, the data collection and analysis took place simultaneously, as the time frame of three months for focus group meetings was short. A strength of this study is the use of real-world examples, which was very helpful for the discussion.
Our results make clear that simply providing infor-mation is not enough and will not enhance the usage of CPI. CPI is valued only marginally, and as such it is not yet used by service users as an additional infor-mation source, nor does it challenge healthcare con-sumers’ current selection of hospitals. Our findings also show that more focus is needed on eliciting the underlying values of consumers. Several studies, in-cluding ours, have stressed the need for an agent that can support healthcare consumers in choosing and can coordinate on their behalf when bringing the choice into practice.[38,39] Such an agent might be able to elicit consumers’ preferences, clarify the values of CPI and preferably integrate this into the referral processes of primary care providers.
We would like to thank the participating healthcare consumers, who shared their time and thoughts with us.
The Dutch healthcare insurance company CZ funded this study.
Ethical approval is not necessary for focus groups meetings with healthcare consumers.
Not commissioned; externally peer reviewed.
None.
