Journal of the Pancreas Open Access

Keywords

Pancreatectomy; Pancreatic neoplasms; Psychometrics; Quality of life; Surveys and questionnaires; Validation Studies

Abbreviations

C30 core questionnaire; EORTC European Organization for Research and Treatment of Cancer; PAN26 pancreatic cancer module

INTRODUCTION

Pancreatic cancer and its treatment can have severe negative effects on patients’ health related quality of life (QOL), some of which are perceived differently by doctors and patients and are best assessed by patients themselves [1]. The prognosis for pancreatic cancer patients is generally poor; less than 20% of patients are eligible for potentially curative pancreatic resection, and this treatment is associated with substantial morbidity and measurable mortality. More research on the effects of pancreatic cancer and its treatment on health related QOL and how these effects can be accurately measured is urgently needed in order to provide the best care for these patients.

In response to rising demand, in the European Organization for Research and Treatment of Cancer (EORTC) in 1999 developed the PAN26 instrument, which is a patient-reported measure of heath related QOL issues specifically arising in pancreatic cancer patients and is intended to supplement the EORTC Core Quality of Life Questionnaire [2]. The PAN26 has completed Phase III of EORTC module development, meaning that it can be used in clinical trials with permission. Phase IV of development, which requires psychometric testing in a large international group of patients, is ongoing [3, 4]. The tool has been translated using robust EORTC translation processes, ensuring its use is valid across multiple nations [5, 6]. To our knowledge, only a limited amount of previous work describing data collected with the PAN26, and specifically validating it, has been published. We recently described the trajectory of PAN26 and QLQ-C30 scores in the 60 days following surgery but the psychometric validation presented in that paper was not comprehensive and consisted only of confirming that the PAN26 scales had adequate internal consistency [7]. It is critical that this instrument be validated to ensure its appropriateness to measure the most important concerns arising for pancreatic cancer patients in a way that is reliable, interpretable, and sufficiently sensitive to detect health related QOL differences that are meaningful to patients. In addition, the usefulness of the instrument in the pancreatic resection setting needs to be confirmed. Previous validation studies have been according to EORTC procedures for module development [3], on small populations and in the setting of chronic pancreatitis.

This paper describes a psychometric validation study assessing the reliability and validity of the PAN26 carried out using a high-quality dataset from a Phase III clinical trial of 300 patients undergoing surgical resection for pancreatic and peri-pancreatic neoplasms. We also aim to confirm the presence of expected health related QOL differences between clinically different groups and to provide reference data (here and in related publications) to inform future sample size calculations and for use in interpretation [7, 8, 9]. Only by understanding the psychometric properties of the PAN26 will we be able to make the most accurate interpretations of the data collected using this instrument and ensure that clinical decisions are based on solid data rather than solely on expectations and expert opinions.

METHODS

Patients and Study Design

Three hundred patients undergoing pancreatic resection were enrolled in a Phase III, single-center, randomized, double-blind, placebo-controlled trial of preoperative pasireotide to reduce postoperative pancreatic fistula. Primary clinical results have been reported previously. [9] Patients completed the EORTC core cancer module (QLQ-C30) and pancreatic cancer module (QLQPAN26) at three time points: prior to surgery (baseline), 14 and 60 days after surgery [10]. The numbers of questionnaires returned at each time point were 299, 273 and 265 respectively. Surveys were completed in person at scheduled clinic visits with the assistance of research study assistants if needed.

Questionnaire

The QLQ-C30 consists of 28 four-level Likert items, and two seven-level Likert items which are scored according to the EORTC scoring guidelines into the 15 domains. Dyspnea, insomnia, appetite loss, constipation and financial difficulties are single-item scales while the remaining scales are composed of two to five items [10]. The QLQ-PAN26 consists of 26 four-level Likert items which are scored according to draft scoring procedures supplied by EORTC to obtain seven multi-item scale scores consisting of two to four items and nine single-item scores (Supplemental Table 1).

Domain scoring can be done when less than half the items within a domain are missing and is done by taking the mean of the component items and scaling so that zero corresponds to the lowest possible score and 100 corresponds to the highest possible score [11, 12, 13]. The QLQ-C30 pain scale items ask about general pain while the PAN26 pancreatic pain scale items refer specifically to abdominal discomfort, back pain, pain during the night, and discomfort in certain positions. To distinguish between the pain scales, we will refer to them as QLQ-C30 pain and pancreatic pain. For most scales, higher scores indicate worse symptoms and worse health related QOL. The QLQ-C30 physical, role, emotional, cognitive and social functioning scales, the QLQ-C30 global health status scale, and the PAN26 satisfaction with health care scale are scored as functional scales where higher scores indicate better function and better health related QOL.

Ethics

This study was approved by the institutional review board at Memorial Sloan Kettering Cancer Center and all subjects provided written consent.

Statistics

Validity and reliability of the QLQ-PAN26 scales were assessed separately for the three time points due to the trial design and because a previous report demonstrated that scores changed across time points [7]. Correlations between four-level ordinal items were estimated using polychoric correlation and correlations between items and multi-item scale scores were estimated using polyserial correlation. Multi-trait scaling analysis was performed to confirm appropriate scale aggregation, that is, that multiitem scales consisted of similar, inter-correlated items and that items were more correlated with their own scale than with other scales. Item-total correlations were corrected for overlap by leaving the item of interest out of the total. Internal consistency of multi-item scales was assessed using Cronbach’s alpha and coefficient omega, a measure of internal consistency that relies on fewer assumptions than alpha, with values >0.7 considered acceptable [14, 15, 16]. For scales with only two items, reliability measures were based on polychoric correlation, which leads to alpha and omega being equivalent because the two factor loadings are equal [17].

Convergent and discriminant validity were assessed using a correlation matrix containing Pearson’s correlations between all PAN26 domains versus all PAN26 and QLQ-C30 domains. To assess known-groups validity, we compared mean domain scores by final diagnosis and gender using t-tests (p<0.05 considered significant). Statistical analysis was performed in SAS 9.3 (SAS Institute, Cary, NC) and R 3.1.1 (R Foundation, Vienna, Austria) with the polycor, psych, and GPArotation packages [18, 19, 20]. This study is registered as ClinicalTrials.gov number NCT00994110.

RESULTS

Characteristics of the 300 evaluable patients and means and standard deviations of domain scores at each time point have been presented previously [7, 9]. Average age was 65 years and 45.0% of patients were female. Eighty patients (27%) had a distal pancreatectomy and 220 patients (73%) had a pancreaticoduodenectomy. Fortyfive patients (15.0%) developed pancreatic complications (grade three or higher postoperative pancreatic fistula, leak or abscess within 60 days). A drain was utilized to manage pancreatic complications in the 45 patients who experienced complications and the median duration of drainage was seven days (range 3-152 days). One hundred and sixty patients had a final diagnosis of pancreatic adenocarcinoma and 84 had a final diagnosis of benign pancreatic neoplasm.

Response rates for the baseline, 14 day and 60 day questionnaires respectively were 99.7%(299/300), 91.0% (273/300) and 88.3% (265/300). On the baseline questionnaire, non-completion rates by item ranged from 0-14.0% depending on the item and non-completion rates by domain ranged from 0-11.0% (Supplemental Table 2). The least-completed question was item 56, “Have you felt less sexual enjoyment during the past week?” and leastcompleted domain was sexuality.

Multi-trait scaling analysis revealed that all items were more correlated with their own scale than with other scales except for the hepatic scale items at 14 days (Table 1). Hepatic item-total correlations were low at 14 days (0.05) and 60 days (0.28). Correlations between PAN26 single-item scales and PAN26 multi-item scales were low to moderate (Supplemental Table 3). The correlation between Q32, “Did you have a bloated feeling in your abdomen?” and pancreatic pain was fairly high: 0.70, 0.49 and 0.68 at baseline, 14 and 60 days. This relationship was driven by high correlations between item 32 and item 31 (“Have you had abdominal discomfort?”): 0.78, 0.63 and 0.77 at the three time points respectively.

Cronbach’s alpha and omega values for baseline measurements ranged from 0.78 to 0.96 (Table 2). Internal consistency remained good at the second and third time points with the exception of the hepatic domain, which had values of 0.09 and 0.43 at 14 and 60 days. The hepatic domain consists of two items: “Have you had itching?” and “To what extent was your skin yellow?” At 14 and 60 days, few patients reported any extent of yellow skin (9.8% and 2.8% respectively), but more than one out of four patients had itching and 2-3% of patients had “very much” itching.

Table 3 shows correlations between each of the PAN26 multi-item scales and the other PAN26 scales, and the QLQ-C30 scales. Correlation between PAN26 domains was generally low (<0.5). As expected, pancreatic pain and QLQ-C30 pain had a high correlation (>0.7). Correlations between QLQ-C30 and PAN26 symptom scales were positive or very close to zero with one exception (correlation between constipation and altered bowel habit at 14 days = -0.21), while correlations between QLQ-C30 function scales and PAN26 symptom scales were negative. Correlations between the PAN26 single-item scales were positive and low to moderate in magnitude (Supplemental Table 4). As expected, and similarly to the PAN26 multiitem symptom scales, the PAN26 single-item scales were negatively correlated with QLQ-C30 function scales and (with two exceptions of correlations close to zero) positively correlated with QLQ-C30 symptom scales. Correlations of high magnitude were observed between item 52 (“Were you limited in planning activities in advance (e.g. meeting friends)?”) and social functioning and global health status, and between item 42 (“Did your arms and legs feel weak?”) and fatigue. In general, correlations between the health care satisfaction scale and all other scales were close to zero (range: -0.17 to 0.17). Correlations between scales generally had a smaller magnitude at 14 days compared to pre-op and 60 days.

At baseline, female gender was significantly associated with worse outcomes on the pancreatic pain and body image scales and diagnosis of pancreatic adenocarcinoma was associated with worse outcomes on the digestive, altered bowel habit, hepatic and sexuality scales (Table 4, all p<.05).

DISCUSSION

This paper reports on the psychometric properties of the PAN26 pancreatic cancer module of the EORTC Quality of Life Group in patients undergoing pancreatic resection. To our knowledge, only a limited amount of research has been done assessing the reliability and validity of the PAN26. A previous development study by Fitzsimmons et al. assessed the appropriateness of the PAN26 instrument in a cross-cultural population of 66 patients with chronic pancreatitis, 36 of whom underwent resection [11]. They found that the instrument had adequate internal consistency, that correlations between conceptually related scales were high, and that the instrument detected differences in health related QOL based on both performance status and the requirement for opiate analgesia. Shaw et al. assessed quality of life in 40 patients who had undergone pancreaticoduodenectomy at a median of 42 months after surgery, demonstrating that the instrument can distinguish between long term survivors and matched controls on some problems associated with exocrine insufficiency (upper gastrointestinal symptoms, weight loss, muscular weakness) [21]. The levels of other pancreatic-specific symptoms were not significantly different in the two groups, probably due to the long time period between surgery and assessment. Recently, the PAN26 has been shown to be reliable in patients receiving chemoradiotherapy [22]. As in the present study, Cronbach’s alpha was >0.7 for all scales except hepatic. The instrument was able to detect changes in health related QOL after radiotherapy, which recovered at later assessment. Comparison of known groups identified clinically significant differences; however, numbers in that study were insufficient for full psychometric assessment of scale structure The PAN26 has also been reported after total pancreatectomy [23, 24].

During Phase I-III of development of the PAN26 module, the instrument was translated into 10 European languages and it is now available in over 30 languages other than English [2]. The PAN26 was translated into Lithuanian, following EORTC translation procedures, by Vanagas et al. who also performed a preliminary assessment of the internal consistency of the scales based on limited validation data from 13 patients with pancreatic cancer [6, 25]. Although Cronbach’s alpha was quite low for the digestive and health care satisfaction scales, they concluded that these deviations were insignificant due to the small number of patients and the small number of questions (two) comprising the scales.

In the largest psychometric analysis of the PAN26 to date, we found that the instrument was generally valid and sensitive to clinically relevant differences in health related QOL including impairment on multiple scales prior to surgery inpatients with cancerous versus benign diagnoses. Previous analyses have shown that the instrument can detect the short term morbidity associated with pancreatic resection, with significantly lower scores on all scales except hepatic at 14 days post resection, compared to baseline, as well as the negative impact of pancreatic complications on quality of life [7].

Multi-trait scaling analysis generally supported appropriate aggregation of items as almost all items were more correlated with their own scale than any other scale. (Note that the scales used here are hypothesized scales; confirmed scales will be released when Phase IV of validation is complete.) The one exception to this was the hepatic scale, where item-total correlation was only 0.05 at 14 days and remained low at 60 days. The hepatic scale also had low internal consistency at postoperative time points. Similar findings in patients treated with chemoradiation therapy suggest that the hepatic scale is probably most relevant at the time of presentation and for those treated by palliative stenting, in whom obstructive jaundice is more common [22]. Fitzsimmons et al. also observed low internal validity for the hepatic scale (Cronbach’s alpha=0.18) [11]. The hepatic scale encompasses two symptoms, itching and yellow skin. The low Incidence of yellow skin (which may be expected to resolve following surgery) and other causes of itching (opioid analgesia, time spent in bed, wound healing) postoperatively are likely the main reasons why this scale has poor internal consistency in postoperative patients. Nevertheless, the hepatic scale was able to identify significant differences at baseline between benign and malignant tumors (which are more likely to cause jaundice), supporting the construct validity of this scale. The reliability problem may be limited to the post-surgery and chronic pancreatitis settings. Internal consistency was good for all other scales.

Single items that were not part of a scale generally had low correlations with existing scales and thus there is no evidence that they should be merged into the existing scales. A possible exception is question 32, “Did you have a bloated feeling in your abdomen?” which had a fairly high correlation with the pancreatic pain scale (0.70 at baseline, 0.68 at 60 days), and specifically, with item 31 (“Have you had abdominal discomfort?”), indicating that some patients may interpret items 31 and 32 as asking about very similar symptoms. The preliminary analysis in Phase III of development of the PAN26 showed that item 32 was moderately correlated with both pancreatic pain and digestive symptom, and this item was ultimately not included in any scale [2].

Correlations between the PAN26 scales and across PAN26 and QLQ-C30 scales supported the construct validity of the PAN26, with high correlation between conceptually related scales (pancreatic pain and QLQ-C30 pain), generally positive correlations between symptom scales, and negative correlations between symptom severity and function. Interestingly, correlation between health care satisfaction and symptom and function scales was generally of low magnitude, possibly indicating that satisfaction is not related to outcome but rather to parts of the disease/treatment process not measured by these instruments. Administering the instrument was feasible as we were able to achieve high survey completion rates, likely partially due to the short time needed to complete the survey [25]. Similar to others, we observed that questions related to sexuality were the most likely to be skipped; many factors can contribute to this [10, 22, 26, 27].

Our population of interest was patients undergoing pancreatic resection. Though this population is heterogenous in terms of pathologic diagnosis and includes patients with benign as well as malignant neoplasms, all resected patients have similar symptoms and short-term quality of life regardless of their diagnoses, thus this is an appropriate population for validation of the PAN26. Our analysis had several limitations. Only 20% of pancreatic cancer patients are suitable to undergo resection, so other studies are required to assess the instrument across the full range of stage and treatments. We could not assess the reproducibility of the PAN26 since we did not have access to test-retest data, and our trial did not include a debriefing questionnaire, which examines patients’ reasons for non-compliance and whether items are upsetting or distressing and records the time to complete the tools. An additional limitation of this study is the lack of long-term follow-up data. Lastly, the generalizability of our data may be limited as it came from a single center; we await ongoing EORTC validation studies for additional insight into psychometric issues and use of the instrument in international, crosscultural settings. Nevertheless, the EORTC have used clinical trial data in the past to validate tools, as have other investigators reporting EORTC module validation data, and our validation study represents the largest and most thorough psychometric analysis to date of the PAN26 [26, 27].

Overall, we found that the PAN26 is valid, reliable and able to detect clinically important differences in patients undergoing pancreatic resection. The hepatic scale identified impaired health related QOL in patients with adenocarcinoma (versus benign conditions) at baseline but may not be valid in the post-operative setting; additional research is needed to determine in which situations its use is appropriate. With the exception of QLQ-C30 pain and pancreatic pain, we did not find high correlations between QLQ-C30 and PAN26 scales, indicating that the PAN26 supplements the QLQ-C30 core questionnaire by measuring aspects of health related QOL not already captured in the QLQ-C30. We look forward to a full EORTC Quality of Life Group validation study of the PAN26 module, which we anticipate will confirm our findings. In light of the extreme importance of patient-reported quality of life, especially in the setting of short life expectancy and treatments that may severely impair quality of life, reliable assessment of health related QOL in patients with pancreatic cancer is crucial. We have shown that the use of the PAN26 in studies of surgical treatment will yield reliable results, allowing patients to compare treatment options based on empirical data.

Acknowledgements - Sources of Support

This work was supported in part by the NCI Cancer Center core grant P30 CA008748 and Novartis.

Conflict of Interest Statement

Dr Allen has worked in a consulting/advisory role for Sanofi and received research funding from Novartis. Dr Bottomley is an author of the EORTC QLG measurement system. EORTC tools are provided free to academics; however, a user fee is applicable for use in industrysponsored research. These profits cover costs of future psychometric validation, testing and translation of EORTC tools.

References

1. Fitzsimmons D, George S, Payne S, Johnson CD. Differences in perception of quality of life issues between health professionals and patients with pancreatic cancer. Psychooncology 1999; 8:135-43. [PMID: 10335557]
2. Fitzsimmons D, Johnson CD, George S, Payne S, Sandberg AA, Bassi C, Beger HG, et al. Development of a disease specific quality of life (QoL) questionnaire module to supplement the EORTC core cancer QoL questionnaire, the QLQ-C30 in patients with pancreatic cancer. EORTC Study Group on Quality of Life. Eur J Cancer 1999; 35:939-41. [PMID: 10533475]
3. Johnson C, Aaronson N, Blazeby JM, Bottomley A, Fayers P, Koller M, Kuliś D, et al. Guidelines for developing questionnaires modules. 3rd ed. ed. 2011, Brussels: EORTC Publications.
4. EORTC. Modules in development and available for use.  [cited 2015 June 1]; Available from: https://groups.eortc.be/qol/modules-development-and-available-use.
5. Kuliś D1, Arnott M, Greimel ER, Bottomley A, Koller M. European Organisation for Research and Treatment of Cancer Quality of Life Group. Trends in translation requests and arising issues regarding cultural adaptation. Expert Rev Pharmacoecon Outcomes Res 2011; 11:307-14. [PMID: 21671700]
6. Dewolf L, Koller M, Velikova G, Scott N, Bottomley A. 2009 EORTC Quality of Life Group Translation Procedure 3rd edn. EORTC Quality of Life Group.
7. Eaton AA, Gonen M, Karanicolas P, Jarnagin WR, D'Angelica MI, DeMatteo R, Kingham TP, et al. Health-related quality of life after pancreatectomy: results from a randomized controlled trial. Ann Surg Oncol 2016; 23:2137-45.[PMID: 26786091]
8. Scott NW, Fayers PM, Aaronson NK, Bottomley A, de Graeff A, Groenvold M, Gundy C, et al.Sprangers MAG on behalf of the EORTC Quality of Life Group. EORTC QLQ-C30 Reference Values. EORTC Quality of Life Group Publications, Brussels, 2008.
9. Allen PJ, Gönen M, Brennan MF, Bucknor AA, Robinson LM, Pappas MM, Carlucci KE, D'Angelica MI, et al. Pasireotide for Postoperative Pancreatic Fistula. N Engl J Med 2014; 370:2014-2022. [PMID: 24849084]
10. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993; 85:365-76. [PMID: 8433390]
11. Fitzsimmons D, Kahl S, Butturini G, van Wyk M, Bornman P, Bassi C, Malfertheiner P, et al. Symptoms and quality of life in chronic pancreatitis assessed by structured interview and the EORTC QLQ-C30 and QLQ-PAN26. Am J Gastroenterol 2005; 100:918-26. [PMID: 15784041]
12. EORTC Quality of Life Group, Draft scoring procedure for the EORTC-PAN26. 2007.
13. Fayers PM, Aaronson NK, Bjordal K, Groenvold M, Curran D, Bottomley A. The EORTC QLQ-C30 Scoring Manual 3rd Edition ed. 2001, Brussels: European Organisation for Research and Treatment of Cancer.
14. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol 2013; 105:399-412. [PMID: 24844115]
15. Peters GJY. The alpha and the omega of scale reliability and validity: Why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality. European Health Psychologist 2014; 16:56-69.
16. Nunnaly J. Psychometric Theory. 1978, New York: McGraw-Hill.
17. Zinbarg RE, Revelle W, Yovel I, Li W.Cronbach's α, Revelle's β, and McDonald's ωH: their relationships with each other and two alternative conceptualizations of reliability. Psychometrika 2005; 70:123-133.
18. Fox J.Polycor: Polychoric and Polyserial Correlations. 2010.
19. Revelle W.Psych: Procedures for Personality and Psychological Research. 2015.
20. Bernaards CA, Jennrich RI.Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement 2005; 65:676-696.
21. Shaw CM, O'Hanlon DM, McEntee GP. Long-term quality of life following pancreaticoduodenectomy. Hepatogastroenterology 2005; 52:927-32. [PMID: 15966234]
22. Hurt CN, Mukherjee S, Bridgewater J, Falk S, Crosby T, McDonald A, Joseph G, et al. Health-related quality of life in scalop, a randomized phase 2 trial comparing chemoradiation therapy regimens in locally advanced pancreatic cancer. Int J Radiat Oncol Biol Phys 2015; 93:810-8. [PMID: 26530749]
23. Epelboym I,Winner M, DiNorcia J, Lee MK, Lee JA, Schrope B, Chab ot JA, et al. Quality of life in patients after total pancreatectomy is comparable with quality of life in patients who undergo a partial pancreatic resection. J Surg Res 2014; 187:189-96. [PMID: 24411300]
24. Billings BJ, Christein JD, Harmsen WS, Harrington JR, Chari ST, Que FG, Farnell MB, et al. Quality-of-life after total pancreatectomy: is it really that bad on long-term follow-up? J Gastrointest Surg 2005; 9:1059-66; discussion 1066-7. [PMID: 16269376]
25. Vanagas T, Gulbinas A, Johnson C, Pundzius J, Barauskas G. Translation and validation of the EORTC QLQ-PAN26 quality of life questionnaire for patients with pancreatic cancer. Acta Medica Lithuanica 2006; 13:272-275.
26. Taphoorn MJ, Claassens L, Aaronson NK, Coens C, Mauer M, Osoba D, Stupp R, et al., An international validation study of the EORTC brain cancer module (EORTC QLQ-BN20) for assessing health-related quality of life and symptoms in brain cancer patients. Eur J Cancer 2010; 46:1033-40. [PMID: 20181476]
27. Blazeby JM, Hall E, Aaronson NK, Lloyd L, Waters R, Kelly JD, Fayers P. Validation and reliability testing of the EORTC QLQ-NMIBC24 questionnaire module to assess patient-reported outcomes in non-muscle-invasive bladder cancer. Eur Urol 2014; 66:1148-56. [PMID: 24612661]