Figures
Abstract
Background
Most studies showing a volume outcome effect in resection surgery for oesophago-gastric cancer were conducted before the centralisation of clinical services. This study evaluated the relation between hospital- and surgeon volume and different risk-adjusted outcomes after oesophago-gastric (OG) cancer surgery in England between 2011 and 2013.
Methods
In data from the National Oesophago-Gastric Cancer Audit from the UK, multivariable random-effects logistic regression models were used to quantify the effect of surgeon and hospital volume on three outcomes: 30-day and 90-day mortality and anastomotic leakage. The models included patient risk factors to adjust for differences in case-mix among hospitals and surgeons. The between-cluster heterogeneity was estimated with the median odds ratio (MOR).
Results
The study included patients treated at 42 hospitals and 329 surgeons. The median (interquartile range) of the annual hospital and surgeon volumes were 110 patients (82 to 137) and 13 patients (8 to 19), respectively. The overall rates for 30-day and 90-day mortality were 2.3% and 4.4% respectively, and the anastomotic leakage was 6.3%. Higher hospital volume was associated with lower 30-day mortality (OR: 0.94; 95% CI: 0.91–0.98) and lower anastomotic leakage rates (OR: 0.96; 95% CI: 0.93–0.98) but not 90-day mortality. Higher surgeon volume was only associated with lower anastomotic leakage rates (OR: 0.81; 95% CI: 0.72–0.92). Hospital volume explained a part of the between-hospital variation in 30-day mortality whereas surgeon volume explained part of the between-hospital variation in anastomotic leakage.
Conclusions
In the setting of centralized O-G cancer surgery in England, we could still observe an effect of volume on short-term outcomes. However, the effect is inconsistent, depending on the type of outcome measure under consideration, and much smaller than in previous studies. Efforts to centralise O-G cancer services further should carefully address the effects of both hospital and surgeon volume on the range of outcome measures that are relevant to patients.
Citation: Fischer C, Lingsma H, Klazinga N, Hardwick R, Cromwell D, Steyerberg E, et al. (2017) Volume-outcome revisited: The effect of hospital and surgeon volumes on multiple outcome measures in oesophago-gastric cancer surgery. PLoS ONE 12(10): e0183955. https://doi.org/10.1371/journal.pone.0183955
Editor: Zachary A. Smith, Northwestern University, UNITED STATES
Received: August 10, 2016; Accepted: August 15, 2017; Published: October 26, 2017
Copyright: © 2017 Fischer et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The clinical data used in this study was third party data collected as part of the National Oesophago-Gastric Cancer Audit, which is commissioned by the Healthcare Quality Improvement Partnership (HQIP). HQIP is the data controller for this dataset and access to the data is subject to a Data Sharing Agreement and copyright license. Interested parties are able to obtain the clinical audit data directly from HQIP via their Data Access Request process. This process is described on the HQIP website (http://www.hqip.org.uk/national-programmes/a-z-of-nca/audits-and-data-gov-uk/). The Audit has section 251 approval to link the clinical audit dataset to the Office for National Statistics (ONS) death register. Researchers who wish to make use of audit data linked to date of death information need to make a formal application to HQIP and NHS Digital (http://content.digital.nhs.uk/DARS). An application for approval to combine data from the audit and the ONS death register needs to be submitted to the Confidential Advisory Group (CAG). Information about the CAG application process can be found at: http://www.hra.nhs.uk/about-the-hra/our-committees/section-251/.
Funding: The National Oesophago-Gastric Cancer Audit is commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit Programme (NCA). HQIP is led by a consortium of the Academy of Medical Royal Colleges, the Royal College of Nursing and National Voices. Its aim is to promote quality improvement, and in particular to increase the impact that clinical audit has on healthcare quality in England and Wales. HQIP holds the contract to manage and develop the NCA Programme, comprising more than 30 clinical audits that cover care provided to people with a wide range of medical, surgical and mental health conditions. The programme is funded by NHS England, the Welsh Government and, with some individual audits, also funded by the Health Department of the Scottish Government, DHSSPS Northern Ireland and the Channel Islands. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
For many surgical procedures, patient outcomes have been found to be related to surgical volume (the number of procedures that is performed in a specific unit), with studies typically showing that higher volumes are associated with lower postoperative mortality [1]. As a result, the centralization of high-risk oncological services, including oesophago-gastric cancer (O-G cancer), is occurring in many countries [2–6]. In the UK, the Department of Health published a recommendation to centralize curative surgical services into specialised cancer centres in 2001 [7]- and it is recommended that surgeons perform a minimum of 15–20 annual resections[8]. As a consequence, a process of reorganization has taken place in the National Health Services (NHS) during the past decade, which has resulted in a smaller number of acute trusts (hospital organisations) doing this type of surgery.
In O-G cancer surgery, risk-adjusted postoperative mortality and complication rates are widely used as quality indicators[9]. Case volume has also been proposed as a marker for quality in the past because of the substantial evidence of a volume-outcome relationship[10]. However, it is unclear whether a volume-outcome relationship is still detectable given that, in a centralized setting, all trusts may exceed recommended thresholds. In addition, the exact mechanism behind the volume-outcome relation is still not fully understood[11,12]. It is suggested that both the experience of the surgeon and the complete hospital team contribute to surgical outcomes[10]. Finally, there are more outcomes of interest for O-G cancer surgery than the commonly used 30-day mortality. Recent publications suggest that anastomotic leakage rates and 90-day postoperative mortality are also important in assessing the quality of surgical care [13–17].
This study was undertaken to examine the relation between hospital- and surgeon volume and different risk-adjusted outcomes after O-G cancer surgery in a setting of centralized care, and the between-hospital and surgeon differences in outcome.
Methods
Data
We used data submitted to the National Oesophago-Gastric Cancer Audit (NOGCA), which evaluates the care delivered by all (n = 154) English hospitals that provide care to adults diagnosed with invasive, epithelial cancer of the oesophagus or stomach. Data are collected prospectively by hospital staff and have been submitted to the audit since 1 April 2011. Details on the audit method and dataset have previously been published[18]. All patients undergoing curative surgery between 1 April 2011and 31 March 2013 were included in the study. The date of death was obtained from the Office for National Statistics Death Register. We excluded patient undergoing curative oncological treatment for squamous cell carcinoma and all palliative patients. Further, we excluded hospitals which operated on less than 10 patients (S1 Fig).
Predictors and outcomes
We considered three outcomes: 30-day mortality, 90-day mortality and anastomotic leakage. Mortality was defined as all-cause postoperative mortality within 30 or 90 days after surgery. Anastomotic leakage was defined as severe disruption to the anastomosis, irrespective of whether detected clinically or radiologically, and irrespective of whether it is managed conservatively or by re-operation[19]
Pre-operative patient and tumour characteristics to be used for case-mix adjustment were based on prior research. All regression models included: comorbidity count, age, ASA score, ECOG (WHO) performance status, T stage, N stage, cancer location. Patient gender was also included in the models for the outcomes 90-day mortality and AL, and deprivation was included in the AL model. We analysed hospital and surgeon volume. Hospital was defined as NHS trust, which is a division within the English NHS that can consist of up of up to five hospitals. Usually O-G cancer surgery is only performed in one hospital in the trust. Surgeon was defined as the principal operating surgeon.
Hospital volume was defined as the number of O-G cancer surgeries performed at a NHS trust per year. Surgeon volume was defined as the annual number of operations conducted by an individual surgeon.
Statistical analysis
Patient characteristics were described as means or percentages. We described surgeon and hospital volume in the study period with median volume per hospital/surgeon. To describe outcome differences, we divided the hospitals/surgeons in quartiles based on volume and presented the outcome rates for each volume quartile.
The effect of hospital and surgeon volume on the three patient outcomes was tested in multivariable case-mix adjusted logistic regression models, with volume added as a continuous variable. We tested whether the relationship between volume and the outcomes was non-linear by adding squared terms and comparing these with linear terms based on the chi-square statistic.
We assessed differences in outcome between hospitals with random effects models[20]. First, we analysed between hospitals and surgeons differences without any adjustment. This random effects model includes two random intercepts: one for hospital and one for surgeon and no other covariates. The variance of the random intercepts represents the between hospital/surgeon variation without any adjustment, but taking into account the random variation. In the second model the case-mix adjustment variables were added as covariates, to adjust the between hospital/surgeon variation for differences in patient characteristics. In a next step, we added surgeon and hospital volume one by one as covariates. In a final step, we included both surgeon and hospital volume.
In random effects models the between hospital/surgeon variation is reflected in the variance of the random intercepts (τ2). We used the median odds ratio (MOR) to quantify this variation. The MOR is a direct function of τ2 (MOR = exp (√ 2 x τ2 x Φ–1 (0.75)) 21. The MOR can be equal or greater than 1, an MOR of 1 reflects no variation between the hospitals. The larger the between-hospital variation, the higher the MOR will be[21].
There were no missing values in patient outcomes. Missing data in predictors were imputed using a ‘multiple imputation by chained equations’ model including the outcome measures and relevant covariates. Imputation and statistical analysis were performed Stata software, version 11(StataCorp.2009.Stata Statistical Software: Release: 11) and in R statistical software 2.7.2 using the Hmisc, the lm4 and rms packages (R Foundation for Statistical Computation, Vienna) Syntax code for R is provided in the appendix (S1 Text).
Results
Descriptives
The study included 4868 patients treated at 42 hospitals and 329 surgeons. Patients had a mean age of 66 years and the majority were male (n = 3610; 74%). See Table 1 for all patient characteristics. Overall, 30-day mortality was 2.3% (n = 111), 90-day mortality was 4.4% (n = 215), and 6.3% (n = 305) of the patients developed an anastomotic leakage (AL).
Variation on surgeon and hospital level.
Volume varied between both surgeons and hospitals. The median hospital volume was 55 patients per year, with an interquartile range of 41 and 68 (Fig 1). The median surgeon operated 6 patients per year, and the interquartile range was 4 and 9 (Fig 1).
ab. Hospital volume (1a), surgeon volume (1b). (1a). Median hospital volume = 55; Interquartile range = 41–68. (1b). Median surgeon volume = 6; Interquartile range = 4–9.
The risk of all outcomes was lower in the highest volume quartiles compared to the lowest volume quartiles (Table 2). For example, 30-day mortality was 3.0% in the lowest hospital volume quartile compared 1.3% in the highest quartile.
Higher hospital volume was a significant independent predictor for lower 30-day mortality (OR: 0.94 (95%CI: 0.91–0.98) per increase of 5 patients per year) (Table 3). Surgeon volume had no significant effect on 30-day mortality. Higher surgeon and higher hospital volume were independent predictors of lower risk of an anastomotic leakage (hospital volume OR: 0.96 (95%CI: 0.93–0.98), surgeon volume: OR: 0.88 (95% CI: 0.78–1.00). Neither hospital volume nor surgeon volume were significant predictors for 90-day mortality when controlling for each other (Table 3).
Hospital volume explained part of the variation in 30-day mortality between the hospitals. The median odds ratio (MOR) decreased from 1.38 without controlling for hospital volume in the model to 1.30 when hospital and surgeon volume were added to the models as covariates (Fig 2). The MOR represents the odds of dead within 30 days of a patient from a randomly selected hospital compared to the odds if he/she would go to another randomly selected hospital, after taking into account the random variation and differences in patient characteristics. Surgeon volume did not explain between-hospital or between-surgeon variation in 30-day mortality (Fig 2, S1 Table).
Fig 2 a. Between-hospital variation in outcomes. Change in median odds ratio (MOR) on hospital level considering different case mix and prediction factors for 30-day, 90-day mortality and anastomotic leakage. Fig 2 b. Between-surgeon variation in outcomes. Change in median odds ratio (MOR) on surgeon level considering different case mix and prediction factors for 30-day, 90-day mortality and anastomotic leakage.
Hospital volume explained more of the between-surgeon variation in anastomotic leakages (change in MOR from 1.67 to 1.56) (Fig 2). The other way around, surgeon volume explained a very small part of the variation between hospitals (change in MOR from 1.29 to MOR 1.27) (Fig 2).
While hospital volume explained a minimal amount of between-hospitals variation in 90-day mortality, surgeon volume did not explain any variation in 90-day mortality at all (Fig 2).
Discussion
Main findings
This study has shown that despite centralization, differences between hospitals and surgeons in patient outcomes after O-G surgery still exist in England. These between-hospital and between-surgeon differences are partly explained by surgeon and hospital volume but the volume outcome relation is different for the different outcomes (30-day and 90-day mortality and anastomotic leakage). Higher hospital volume was associated with lower 30-day mortality, but high surgeon volume was not. Neither hospital volume nor surgeon volume affected 90-day mortality. Higher surgeon and hospital volume were both associated with fewer anastomotic leakages, but surgeon volume was the stronger predictor.
Volume-outcome relation.
Higher volume hospitals have lower 30-day mortality rates; this in line with earlier research in both O-G cancer and other surgical oncological procedures [3,22].
Despite the centralization that took place in the UK, we still found an effect of volume.
In contrast, we did not find surgeon volume to be a significant predictor of 30-day mortality. Surgeon volume and hospital volume possibly reflect different aspects of quality of care. It has been suggested that next to the surgeon skills there are other hospital factors in high volume hospitals that reduce mortality risk, such as post-operative care, care pathways and multidisciplinary team work [23–26]
Surgical volume and hospital volume had only a minor effect on 90-day mortality. Although still a substantial amount of patients dies between 30- and 90-days after surgery, their death seems not to be influenced by the surgeon or hospital volume. Previous research showed that although part of the deaths occurring after 30 days is still related to surgery, and increasing proportion related to cancer recurrence [14]. Both, higher surgeon and hospital volume showed to be related to fewer anastomotic leakages, but surgeon volume had a stronger effect. It can be imagined that anastomotic leakages are closely related to technical surgical skills that high volume surgeons have better developed.
When calculating surgeon volume for highly complicated but rare procedures, it has been suggested to count also related procedures[27]. In our study we had no data to take into account other operations as O-G resections.
Recommendations.
In different countries different volume norms are used, which is the reflection of mixed scientific evidence. Our findings suggest that further increasing hospital volumes, and to a lesser extent surgeon volumes, might improve short-term outcomes. However, on-going centralization might also have negative effects on for example access and equity, which we did not study.
We studied different outcomes that are used as quality indicators. Anastomotic leakages (AL) rates were affected by hospital volume. This is in line with previous findings that surgical complications seem to be a good indicator for surgical quality as they are closely related to the surgical process and are not so much influenced by patient characteristics[9,28,29]. This makes AL rates ‘actionable’ hospital quality indicators. Apart from the clinical relevance, the relatively frequent occurrence makes them attractive as a quality indicator from a statistical point of view, as rates per hospital can be estimated relatively certain. A large disadvantage of complications as an outcome indicator is their possibly unclear definition, which may bias between hospital comparison[30].
Adjusted 30-day mortality differences between hospitals were partly explained by hospital volume. However, a large disadvantage of this measure is the low event rate. Even in our data pooled from two years, the absolute numbers of deaths within 30-days per hospital were small (median absolute number per hospital = 2), which makes the estimates per hospital uncertain and challenges the comparison between hospitals [31,32].
In that sense 90-day mortality rate is more attractive as more deaths occur. However, 90-day mortality differences between hospitals were not explained by hospital volume. Possibly the 90-day mortality rates might reflect aspects of quality of care not related to volume. But we consider it is more likely that a longer time period introduces effects of confounding factors that dilute the relation between quality of care and outcome. In all, multiple indicators should be monitored as this gives a more comprehensive picture [33].
Next to variation between hospitals, we did observe variation in outcome between surgeons. But these were not explained by any of the volume indicators. In addition, the absolute numbers of deaths of ALs per surgeon are extremely low, which makes outcome rates per surgeon unsuitable as quality indicators: the difference between 0 or 1 death per year cannot tell us anything about quality, only about bad luck.
In summary, in the setting of centralized O-G cancer surgery in England, we could still observe an effect of hospitals volume on 30-day mortality and AL rates, suggesting that further centralization might be considered but should carefully address the effects of both hospital and surgeon volume on the range of outcome measures that are relevant to patients. AL rates and 30-day mortality rates per hospital could be useful as quality indicators, but both have also disadvantages. 90-day mortality likely reflects other things than the quality of surgery and thus is less suitable as a quality indicator. Surgeon level outcomes are so infrequent that they are not suitable as quality indicators.
Supporting information
S1 Fig. Flow chart patient inclusion process.
https://doi.org/10.1371/journal.pone.0183955.s001
(DOCX)
S1 Table. Between surgeon and between-hospital variation in 30-day, 90-day mortality and anastomotic leakage.
*Median odds ratio (95% Confidence Interval).
https://doi.org/10.1371/journal.pone.0183955.s003
(DOCX)
Acknowledgments
We would like to acknowledge the health professionals and support personnel in English NHS trusts and Cancer networks who submitted the data to the Audit.
Date of death information was made available by the Office for National Statistics and provided to the Audit in an anonymised file after being linked by NHS Digital. The study was exempt from UK NREC approval because it involved the analysis of an existing dataset of anonymous data for service evaluation. Approvals for the use of audit-ONS linked data were obtained from UK Confidentiality Advisory Group.
Author Contributions
- Conceptualization: CF HL NK RH DC ES OG.
- Formal analysis: CF HL OG ES.
- Methodology: CF HL DC OG.
- Project administration: CF HL OG ES DC.
- Software: CF HL OG ES.
- Supervision: NK ES OG.
- Validation: CF HL OG ES.
- Visualization: CF HL OG ES DC.
- Writing – original draft: CF HL.
- Writing – review & editing: CF HL NK RH DC ES OG.
References
- 1. Birkmeyer JD, Siewers AE, Finlayson EV, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. Apr 11 2002;346(15):1128–1137. pmid:11948273
- 2. Gooiker GA, van der Geest LG, Wouters MW, et al. Quality improvement of pancreatic surgery by centralization in the western part of the Netherlands. Ann Surg Oncol. Jul 2011;18(7):1821–1829. pmid:21544657
- 3. Begg CB, Cramer LD, Hoskins WJ, Brennan MF. Impact of hospital volume on operative mortality for major cancer surgery. JAMA. Nov 25 1998;280(20):1747–1751. pmid:9842949
- 4. Finlayson EV, Goodney PP, Birkmeyer JD. Hospital volume and operative mortality in cancer surgery: a national study. Arch Surg. Jul 2003;138(7):721–725; discussion 726. pmid:12860752
- 5. Finks JF, Osborne NH, Birkmeyer JD. Trends in hospital volume and operative mortality for high-risk surgery. N Engl J Med. Jun 2 2011;364(22):2128–2137. pmid:21631325
- 6. Epstein AM. Volume and outcome—it is time to move ahead. N Engl J Med. Apr 11 2002;346(15):1161–1164. pmid:11948278
- 7.
Natioanl Oesophago-Gastric Cancer Audit. Natioanal Oesophago-Gastric Cancer Audit. Third Annual Report-Patient Summary (2012). http://www.opa.org.uk/news/national-oesophago-gastric-cancer-audit—third-annual-report-patient-summary.html 30 March 2015.
- 8.
Association of Upper Gastrointestinal Surgeons of Great Britain and Ireland (AUGIS) (2010). Guidance on minimum surgeon volumes. http://www.augis.org/wp-content/uploads/2014/05/AUGIS_recommendations_on_Minimum_Volumes.pdf Accessed 30 March 2015.
- 9.
Fischer C LH, Hardwick R, et al. The development of a case-mix correction model for comparing outcomes of oesophago-gastric cancer surgery. 2015.
- 10. Brusselaers N, Mattsson F, Lagergren J. Hospital and surgeon volume in relation to long-term survival after oesophagectomy: systematic review and meta-analysis. Gut. Sep 2014;63(9):1393–1400. pmid:24270368
- 11. Dudley RA, Johansen KL. Invited commentary: Physician responses to purchaser quality initiatives for surgical procedures. Surgery. Sep 2001;130(3):425–428. pmid:11562664
- 12. Khuri SF. Invited commentary: Surgeons, not General Motors, should set standards for surgical care. Surgery. Sep 2001;130(3):429–431. pmid:11562665
- 13. Lingsma HF, Steyerberg EW, Eijkemans MJ, et al. Comparing and ranking hospitals based on outcome: results from The Netherlands Stroke Survey. QJM. Feb 2010;103(2):99–108.
- 14. Talsma AK, Lingsma HF, Steyerberg EW, Wijnhoven BP, Van Lanschot JJ. The 30-day versus in-hospital and 90-day mortality after esophagectomy as indicators for quality of care. Ann Surg. Aug 2014;260(2):267–273. pmid:25350650
- 15. McMillan RR, Berger A, Sima CS, et al. Thirty-day mortality underestimates the risk of early death after major resections for thoracic malignancies. Ann Thorac Surg. Nov 2014;98(5):1769–1774; discussion 1774–1765. pmid:25200731
- 16. Rutegard M, Lagergren P, Johar A, Lagergren J. Time Shift in Early Postoperative Mortality After Oesophagectomy for Cancer. Ann Surg Oncol. Feb 4 2015.
- 17. Snijders HS, Henneman D, van Leersum NL, et al. Anastomotic leakage as an outcome measure for quality of colorectal cancer surgery. BMJ Qual Saf. Sep 2013;22(9):759–767. pmid:23687168
- 18.
Hardwick G GO, Cromwell D. National Oesophago-Gastric Cancer Audit 2013. The Royal College of Surgeons of England; 2013 2013.
- 19. Lee HS, Ha AW, Kim WK. Effect of resveratrol on the metastasis of 4T1 mouse breast cancer cells in vitro and in vivo. Nutrition research and practice. Aug 2012;6(4):294–300. pmid:22977682
- 20.
Lingsma H. Measuring quality of care. Methods and applications to acute neuroloigcal diseases [thesis]. Alphen a/d Rijn: Public Health, ErasmusMC; 2010.
- 21. Larsen K, Merlo J. Appropriate assessment of neighborhood effects on individual health: integrating random and fixed effects in multilevel logistic regression. Am J Epidemiol. Jan 1 2005;161(1):81–88. pmid:15615918
- 22. Hannan EL, Radzyner M, Rubin D, Dougherty J, Brennan MF. The influence of hospital and surgeon volume on in-hospital mortality for colectomy, gastrectomy, and lung lobectomy in patients with cancer. Surgery. Jan 2002;131(1):6–15. pmid:11812957
- 23. Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. JAMA. Sep 24 2003;290(12):1617–1623. pmid:14506121
- 24. Nguyen NT, Paya M, Stevens CM, Mavandadi S, Zainabadi K, Wilson SE. The relationship between hospital volume and outcome in bariatric surgery at academic medical centers. Ann Surg. Oct 2004;240(4):586–593; discussion 593–584. pmid:15383786
- 25. Daley J, Forbes MG, Young GJ, et al. Validating risk-adjusted surgical outcomes: site visit assessment of process and structure. National VA Surgical Risk Study. J Am Coll Surg. Oct 1997;185(4):341–351. pmid:9328382
- 26. Gort M, Broekhuis M, Otter R, Klazinga NS. Improvement of best practice in early breast cancer: actionable surgeon and hospital factors. Breast Cancer Res Treat. Apr 2007;102(2):219–226. pmid:17028985
- 27. Modrall JG, Rosero EB, Chung J, et al. Defining the type of surgeon volume that influences the outcomes for open abdominal aortic aneurysm repair. J Vasc Surg. Dec 2011;54(6):1599–1604. pmid:21962924
- 28. Junemann-Ramirez M, Awan MY, Khan ZM, Rahamim JS. Anastomotic leakage post-esophagogastrectomy for esophageal carcinoma: retrospective analysis of predictive factors, management and influence on longterm survival in a high volume centre. Eur J Cardiothorac Surg. Jan 2005;27(1):3–7. pmid:15621463
- 29. Fischer C, Lingsma HF, van Leersum N, Tollenaar RA, Wouters MW, Steyerberg EW. Comparing colon cancer outcomes: The impact of low hospital case volume and case-mix adjustment. Eur J Surg Oncol. Apr 30 2015.
- 30. Low DE AD, Cecconello I, Chang AC, Darling GE, DʼJourno XB, Griffin SM, Hölscher AH, Hofstetter WL, Jobe BA, Kitagawa Y, Kucharczuk JC, Law SY, Lerut TE, Maynard N, Pera M, Peters JH, Pramesh CS, Reynolds JV, Smithers BM, van Lanschot JJ. International Consensus on Standardization of Data Collection for Complications Associated With Esophagectomy: Esophagectomy Complications Consensus Group (ECCG). Ann Surg. 2015 Aug;262(2):286–94. 2015. pmid:25607756
- 31. Daley J, Henderson WG, Khuri SF. Risk-adjusted surgical outcomes. Annu Rev Med. 2001;52:275–287. pmid:11160779
- 32. Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size. JAMA. Aug 18 2004;292(7):847–851. pmid:15315999
- 33. Almoudaris AM, Burns EM, Bottle A, et al. Single measures of performance do not reflect overall institutional quality in colorectal cancer surgery. Gut. Mar 2013;62(3):423–429.