Journal of the Pancreas Open Access

Keywords

Inflammation; Pancreas; Pancreatitis, Acute Necrotizing; ROC Curve

Abbreviations

CTSI: CT severity index

Dear Sir:

We read the original article regarding assessment of the predictive value of the CT severity index (CTSI) for the severity of acute pancreatitis by Gürleyic et al. with great interest [1]. The purpose of this study was to make a comparative assessment of the accuracy of the CTSI proposed by Baltazar et al. [2], the APACHE II score and serum CRP concentrations in predicting the severity of acute pancreatitis. The cut-off values of 3 for the CTSI and 7 for the APACHE II were used in the present study, based on the results of studies which did not have exactly the same discrimination endpoint [2, 3]. The discrimination endpoint in this study is defined as the ability to separate those patients who had mild pancreatitis from those who had severe pancreatitis, according to the classification criteria of the 1992 Atlanta International Symposium [4].

Receiver operating characteristic (ROC) curves could have been used to determine the most appropriate cut-off point for the selected discrimination endpoint which corresponds to the best possible trade-off between sensitivity and specificity which were estimated in the present sample. Moreover, the area under the ROC curve is a reliable measure of overall predictive discrimination and a previously described method for comparison of the areas under the ROC curves, derived from the same cases, could also have been used [5, 6].

The additional information provided by ROC curves in studies of prognostic indices of acute pancreatitis severity derives from the complete illustration of the relationship between sensitivity and specificity for a certain discrimination endpoint (severe vs. mild pancreatitis in this case). This might have been useful because: a) the clinical impact of the two types of misclassification (failure to correctly identify a case of severe pancreatitis or failure to correctly identify a case of mild pancreatitis) is not the same, and b) it is necessary to realize new prospective comparative studies for assessing the clinical impact of promising imaging techniques, such as MRI or contrast-enhanced US in the near future [7, 8].

Moreover, it should be noted that even though the Atlanta classification system provides a reliable basis for experimental studies for the clinical management of acute pancreatitis, it is not considered to be a perfect system since intermediate forms of the disease do occur [9]. If an imperfect gold standard is used, the estimated accuracy of the tests may suffer (“imperfect gold standard bias”). Another type of bias affecting ROC analysis is “verification bias” and takes place if some of the patients with test results do not have verified disease status or if the decision to verify a patient is influenced by the test results. Calculation of the accuracy of a diagnostic test using standard definitions unavoidably includes the risk of some kind of bias under certain circumstances. ROC analysis offers the possibility of biascorrection methods [10] and methods of nonparametric estimation of ROC curves have been also suggested recently in the case in which the gold standard is not binary or in the absence of a gold standard, [11, 12]. Thus, ROC analysis should be the preferred method for the assessment of the predictive value of imaging techniques.

References

1. Gurleyik G, Emir S, Kilicoglu G, Arman A, Saglam A. Computed tomography severity index, APACHE II score, and serum CRP concentration for predicting the severity of acute pancreatitis. JOP. J Pancreas (Online) 2005; 6:562-7. [PMID 16286706]
2. Balthazar EJ, Robinson DL, Megibow AJ, Ranson JH. Acute pancreatitis: value of CT in establishing prognosis. Radiology 1990; 174:331-6. [PMID 2296641]
3. Larvin M, McMahon MJ. APACHE-II score for assessment and monitoring of acute pancreatitis. Lancet 1989; 2:201-5. [PMID 2568529]
4. Bradley EL 3rd. A clinically based classification system for acute pancreatitis. Summary of the International Symposium on Acute Pancreatitis, Atlanta, Ga, September 11 through 13, 1992. Arch Surg 1993; 128:586-90. [PMID 8489394]
5. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29-36. [PMID 7063747]
6. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148:839-43. [PMID 6878708]
7. Pezzilli R, Fantini L. The imaging assessment of the severity of acute pancreatitis may change in the near future. JOP. J Pancreas (Online) 2005; 6:467-9. [PMID 16186671]
8. Brocchi E, Piscaglia F, Bonora M, Celli N, Venturi A, Fantini L, et al. Echo-enhanced ultrasonography: is it the future gold standard of imaging in acute pancreatitis? JOP. J Pancreas (Online) 2005; 6:464-6. [PMID 16186670]
9. Balthazar EJ. Acute pancreatitis: assessment of severity with clinical and CT evaluation. Radiology 2002; 223:603-13. [PMID 12034923]
10. Zhou XH. Correcting for verification bias in studies of a diagnostic test's accuracy. Stat Methods Med Res. 1998; 7:337-53. [PMID 9871951]
11. Obuchowski NA. Estimating and comparing diagnostic tests' accuracy when the gold standard is not binary. Acad Radiol 2005; 12:1198-204. [PMID 16099683]
12. Zhou XH, Castelluccio P, Zhou C. Nonparametric estimation of ROC curves in the absence of a gold standard. Biometrics 2005; 61:600-9. [PMID 16011710]