The development and validation of a resource consumption score of an emergency department consultation

Martin Müller; Clyde B. Schechter; Wolf E. Hautz; Thomas C. Sauter; Aristomenis K. Exadaktylos; Stephanie Stock; Tanja Birrenbach

doi:10.1371/journal.pone.0247244

Abstract

Background

Emergency Department (ED) visits and health care costs are increasing globally, but little is known about contributing factors of ED resource consumption. This study aims to analyse and to predict the total ED resource consumption out of the patient and consultation characteristics in order to execute performance analysis and evaluate quality improvements.

Methods

Characteristics of ED visits of a large Swiss university hospital were summarized according to acute patient condition factors (e.g. chief complaint, resuscitation bay use, vital parameter deviations), chronic patient conditions (e.g. age, comorbidities, drug intake), and contextual factors (e.g. night-time admission). Univariable and multivariable linear regression analyses were conducted with the total ED resource consumption as the dependent variable.

Results

In total, 164,729 visits were included in the analysis. Physician resources accounted for the largest proportion (54.8%), followed by radiology (19.2%), and laboratory work-up (16.2%). In the multivariable final model, chief complaint had the highest impact on the total ED resource consumption, followed by resuscitation bay use and admission by ambulance. The impact of age group was small. The multivariable final model was validated (R² of 0.54) and a scoring system was derived out of the predictors.

Conclusions

More than half of the variation in total ED resource consumption can be predicted by our suggested model in the internal validation, but further studies are needed for external validation. The score developed can be used to calculate benchmarks of an ED and provides leaders in emergency care with a tool that allows them to evaluate resource decisions and to estimate effects of organizational changes.

Citation: Müller M, Schechter CB, Hautz WE, Sauter TC, Exadaktylos AK, Stock S, et al. (2021) The development and validation of a resource consumption score of an emergency department consultation. PLoS ONE 16(2): e0247244. https://doi.org/10.1371/journal.pone.0247244

Editor: Junaid Ahmad Bhatti, Sunnybrook Research Institute, CANADA

Received: October 8, 2020; Accepted: February 3, 2021; Published: February 19, 2021

Copyright: © 2021 Müller et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Data contain potentially identifying or sensitive patient information. Data used in this study are available upon reasonable request from the Emergency Department of the University Hospital Bern, Switzerland (notfallzentrum@insel.ch) to researchers eligible under Swiss legislation to work with codified personal health care data. Eligibility will be determined by Cantonal ethics committee Bern.

Funding: MM was funded by the Bangarter Foundation and the Swiss Academy of Medical Sciences through the "Young Talents in Clinical Research" grant (TCR 14/17) as well as through a CTU grant of Inselgruppe. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: WEH received speaker honorariums from the AO Foundation in Zurich and research funding from Mundipharma Medical Basel. All other authors have nothing to disclose. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Introduction

Increasing healthcare costs are a worldwide problem [1]. A substantial proportion of these costs results from Emergency Departments (ED), as these provide nearly half of the hospital-associated medical care nowadays in Western countries [2]. ED visits are rising globally [3], and over and above, ED care is more expensive compared to other forms of healthcare [4]. Furthermore, in times of a global healthcare and economic crisis, as in the on-going COVID-19 pandemic, an efficient allocation of material and human resources in the ED is crucial to ensure medical care that is economically sustainable [5].

Despite the important role of ED care in the healthcare system, ED resource consumption has only been modestly studied so far [6–15]. Furthermore, instead of reporting the actual resource consumption, some studies rely on surrogate measures to represent resource consumption, i.e. arrival by ambulance, triage category, number of tests and procedures performed, length of stay in the ED, or admission rates [7, 14]. Most studies reported a positive association of increasing age [8–10, 12, 14] and higher acuity triage category [6, 7, 10, 11, 16] with resource consumption. The evaluation of EDs in terms of resource and performance analysis was often based solely on the volume of an ED counting, for example the number of patients treated each year [17]. However, the profile of the patients admitted to a medical department often varies considerably–consequently so do the resources required by each ED [7]. Thus, assessments of efficiency must take the treated patient profile into consideration.

Within intensive care units, an adapted version of the “Therapeutic Intervention Scoring System” (TISS) [18] is often used as a tool to perform resource and performance analysis [19, 20]. The average overall TISS-28 score per patient day or nurse, for instance, is used to measure the performance of intensive care units in Switzerland and internationally [21–24]. To our knowledge an analogue score that evaluates the use of resources in ED departments does not yet exist. Such a score has the potential to identify areas and special ED patient groups with high resource demands at an early stage–a prerequisite to implement preventive procedures and adaptive actions that might increase the structural, process, and performance quality in the ED in combination with optimisation of needed resources. Furthermore, it might be of great use in research as a standardised tool to describe the performance of an ED and to compare different EDs on an international level.

The aims of this study are i) to illustrate the distribution of an ED patient’s needed resources in different subgroups (laboratory, nurse, physician, material, and radiology), ii) to identify factors that are associated with the total ED resources, iii) to develop and validate a scoring system that predicts the ED resource consumption of a patient and demonstrate a practical example of application for quality assurance.

Methods

Study design, site and period

This is a retrospective cohort analysis of all adult patients admitted to the ED at Inselspital, University Hospital, University of Bern). The ED of Inselspital is one of the largest EDs in Switzerland with a catchment area of two million people, and about 50,000 ED consultations per year [25]. The study period is over five years from 01.01.2013 to 31.12.2017. There were no major structural changes during the study period (see S1 Appendix for a detailed description of our ED and patient management process).

Eligibility criteria

All adult patients (age≥18) presenting to the ED over the study period were included. Patients were excluded if i) the case identification number or a documented chief complaint was missing, ii) multiple consultations shared one case identification number (e.g. very short-term revisits), iii) the consultation generated few or no entries (total ED resources less than 10 tax points, see below) in the resource databases (e.g. cancellations, incomplete documentation), and iv) patients were seen by the psychiatrist as the leading ED physician, as they have a different billing system.

ED resource consumption

Every procedure that is performed in the ED is documented by the person who performed the procedure with a procedural code out of the TARMED Suisse catalogue [26] given by the Swiss health law for billing purposes. Although not all procedural codes are billing-relevant, those codes form the basis for the billing. All members of the ED are trained regularly to achieve accurate coding.

Two-hundred fifty-nine specific procedural codes of the TARMED Suisse catalogue [26] that are regularly used in our ED were chosen by a working group consisting of acute care nurses, radiology nurses, ED physicians, and the controller of our ED department. A numeric value is assigned for each procedural code (e.g. 00.0410 brief physical examination of a patient) with corresponding unit/”medical currency” tax points. The medical currency one tax point (TP) roughly corresponds to 1 US-$, but the exact amount varies among hospitals.

Those codes were grouped into different resource groups i.e. physician, nurse, laboratory, radiology, and material, see S2 Appendix). The total ED resource consumption of a consultation was defined as the sum of all TP of all defined codes. Additionally, the total ED costs for each patient were obtained.

As a secondary outcome and additional surrogate marker of ED resource consumption, the length of stay (LOS) in the ED was also extracted.

Potential predictor variables

Contextual factors and factors describing the acute as well as the chronic condition of a patient’s consultation were assessed as potential predictor variables:

Contextual factors: season of the year (spring to winter), Saturday or Sunday admission, and night-time admissions (from 19:00 to 06:59), the occupancy index (defined as the ratio between the total number of patients in the ED and the total number of ED treatment beds) [27, 28] and the emergency department work index (EDWIN) calculated as (Σ n_i x t_i) / [N_a x (B_T—B_A)], where n_i = number of patients in the ED in triage category i, t_i = triage category, N_a = number of attending physicians on duty, B_T = the number of ED treatment beds, B_A = total number of admitted patients in the ED (0–1.5, active ED; 1.5–2.0, very busy ED; >2, overcrowded ED) [29, 30]. The two latter factors were used to describe the business of the ED.
Acute condition: type of admission, chief complaint groups such as trauma and neurological complaint (see S4 Appendix, based on Aronsky et al. [31]), and documented vital deviations i.e. oxygen saturation (<90%), systolic blood pressure (<100mmHg), temperature (<35.0°C or >38.5°C), level of consciousness (Glascow Coma Scale <15), respiratory rate (<8/min or >25/min), and heart rate (<50/min or >110/min) as these deviations are associated with severe disease courses and higher mortality [32]. To reflect polytrauma and unstable patients, the need for resuscitation room care was defined as a potential predictor variable.
For sensitivity analysis, instead of vital deviations, the triage group was used, which is routinely assessed by special trained nurses using the Swiss Emergency Triage Scale [33], a triage scale similar to the Manchester Triage System [34] (1: highly acute to 5: non-urgent).
Chronic conditions:
1. Important comorbidities based on the Charlson Comorbidity Index [35]: COPD, diabetes, liver disease, dementia, malignancy, cerebrovascular disease, peripheral artery disease, coronary vessel disease, and chronic kidney disease.
2. Drug intake (on admission or discharge) based on the Anatomical Therapeutic Chemical (ATC) classification system [36]: antidiabetic (ATC code A10), antithrombotic (B01), antihypertensive (C02, C04-C09), diuretic (C03), opioid (N02A), and–to set neurological patients, who are thought to have high ED resource consumption, in a broader context–antiepileptic (N03) and psycholeptic (N05).
Other: Demographic factors: age and sex.

Data extraction

The potential predictor variables were extracted from the computerized clinical databases (E-Care, ED 2.1.3.0, Turnhout, Belgium). All procedural codes were extracted from the administrative database (OpenText Suite for SAP® Solutions, OpenText Corporation, Waterloo, Canada). For a detailed protocol, including the definitions of the variables and validation, see S2–S4 Appendices.

Ethical considerations

The study was performed in accordance with Swiss law. The Bern ethics committee registered the study as a quality assurance study (2018–00198) and waived the requirement for informed consent.

Statistical analysis

Stata® 13.1 (StataCorp, The College Station, Texas, USA) was used for statistical analysis. All continuous variables are presented as medians with 25^th- 75^th percentile ranges (IQR). Categorical variables are shown with frequency and proportion. The outcome was natural logarithm (ln)-transformed account for the skewness of the total ED resource consumption. Univariable linear regression analysis with the transformed outcome was performed to quantify the association of the total ED consumption and potential score parameters. The exponentiated coefficient of such a model correspond to the geometric mean ratio (GMR) of the non-log-transformed outcome in the presence vs. the absence of the predictor [37].

For the score development the dataset was randomly split (50:50) into a training and validation set. All studied predictor variables were included in a multivariable linear regression analysis with the ln-transformed total ED resource consumption as outcome. For a parsimonious final model, predictors that changed the geometric mean by less than 10% (0.9 < GMR < 1.1) were removed stepwise from the final model.

The total ED resource consumption can be predicted from the linear regression model as Where a₀ is the constant and a₁,…,a_m are the coefficients of the final model and x₁,…,x_m are binary variables (0/1) indicating the presence or absence of the predictor. From the final model, the resource score, will be defined as where the constants c₀ and c₁ are determined, so that the obtained score possibly ranges from 0 to 100.

Different sensitivity analyses for the final model were performed: A model i) with the use of the triage category instead of the vital parameters (model 2), ii) with an additional interaction term between trauma and resuscitation room use to better reflect polytrauma (model 3), and iii) excluding revisits (model 4). One might argue that the Swiss Tarmed codes do not validly reflect resource consumption as a “resource measure” is already assigned to each process. Thus, we also evaluated the parameter LOS in the ED (in hours) as an outcome, applying the same model development procedure.

To assess the fit and parsimony of the ED resource consumption models, predictive accuracy, and explained variance, the following parameters were calculated: Akaike information criterion (AIC), and Bayesian information criterion (BIC) for the development sample, and mean absolute prediction error (MAPE), mean relative squared error (MRSE), mean squared prediction error (MSPE) and R² for the validation sample [38]. The obtained R² was compared to a model that included triage and age group as predictor variables only. Furthermore, for a more intuitive measure of predictive accuracy, we calculated the median and IQR of the absolute percentage of the deviation of the predicted to observed ratio (|100%—predicted/observed x 100%|) for all deciles in the different models [39].

Results

Patients’ demographics

In total, 164,729 out of 206,006 consultations were included in the analysis and were randomised 1:1 into validation (n = 82,341) and training sets (n = 82,388). The reasons for exclusion were i) patient age younger than 18 years (n = 6,992), ii) case identification number missing or associated with multiple consultations (n = 7,478), iii) few or no (<10 TP) resource database entries, e.g. cancelled consultation (n = 7,639), iv) the psychiatrist was the leading physician (n = 8,511), or v) the chief complaint was not documented (n = 10,657), see S5 Appendix.

In the study population, the median age was 49 years (IQR 32, 67), with 56.3% males. The most common triage group was urgent (60.7%). In total, 16.3% of the consultations had a neurological chief complaint and 16.7% of the patients presented after trauma, and 35.2% of the ED consultations led to hospitalization. There were no significant differences between the validation and training set (S6 Appendix).

Distribution of ED resource consumption.

The median total ED resource consumption was 638 (IQR 254, 1264) TP in the training set. The median in the resource subgroup was highest for the physician resources (training set: 323 TP, IQR 119, 500) followed by laboratory work-up (training set: 96.4 TP, IQR 0, 227) and radiological work-up (training set: 60 TP, IQR 0, 420) with no difference between the training and validation set (S7 Appendix).

The mean relative distribution was slightly different: Physician resources made up the largest proportion (54.8%), followed by radiology (19.2%), laboratory work-up (16.2%), nursing resources (5.4%), and materials (4.5%) in the training set.

The distribution of the total ED resource consumption by triage category is shown in Fig 1. Physician resources accounted for the major percentage in all but the life-threatening triage category. The proportions of radiology and laboratory resources were higher in the life-threatening and high urgent categories than in the less acute triage groups.

Download:

Fig 1. Comparison of the distribution of the total ED resource consumption (mean percentage of total resources and 95% CI bars) by triage category (n = 82,388*).

*the triage category was missing in 1,148 consultations.

https://doi.org/10.1371/journal.pone.0247244.g001

The correlation of ln-transformed total ED resource consumption and total ED costs was high; Pearson’s correlation coefficient was 0.939 (95% CI: 0.939–0.940).

Prediction of total ED resource consumption

Table 1 shows the univariable associations of the acute patient condition factors with the total ED resource consumption. Ambulance admission and resuscitation bay use increased the geometric mean about the factor 2.5 (2.7 and 2.5). Compared to urgent triage, semi-urgent triage had less (GMR 0.5) while high urgent (GMR 2.3) and life-threatening (GMR 4.6) had more resource needs. Last, the chief complaint group had a high impact on resource consumption ranging from a geometric mean factor 0.2 (eye problem) to 2.9 (neurological complain) compared to resources of patients presenting with musculoskeletal complains.

Download:

Table 1. Univariable association of acute patient condition factors with the total ED resource consumption (n = 164,729).

https://doi.org/10.1371/journal.pone.0247244.t001

Table 2 shows the univariable associations of chronic patient condition and contextual factors with the total ED resource consumption. The resource needs increased with increasing age, with the least resource consumption by 18–24 year olds, and the most for patients older than 85 years. The analysed drug intake and comorbidities increased the geometric mean by factors from 1.7 to 3.0. Apart from weekend admissions (GMR 0.8), the impact of contextual factors on ED resource consumption was small.

Download:

Table 2. Univariable association of chronic patient condition and contextual factors with the total ED resource consumption (n = 164,729).

https://doi.org/10.1371/journal.pone.0247244.t002

The two analysed factors describing busyness of the ED, occupancy index and EDWIN score, did only slightly change the GMR (1.0, respectively 1.02).

Development of a scoring system

The results of the multivariable linear regression analysis of the final model (see statistical analysis) are shown in Table 3. The highest impact on the total ED resource consumption had chief complaint with a GMR ranging from 0.2 (eye problems) to 2.3 (neurological complaints) compared to patients with musculoskeletal complaints (baseline group). Each documented vital parameter deviation (heart rate, blood pressure, oxygen saturation, respiratory rate, level of consciousness and temperature) increased the geometric mean of the total ED resource consumption between 11 and 23%. Resuscitation bay use increased the geometric mean by a factor of 2.3 and admission by ambulance by a factor of 1.4. Additionally, drug intake (antithrombotic, antihypertensive, and opioids) and comorbidities (liver disease, malignancy, and cerebrovascular disease) increased the geometric mean each about 18–19% and 25–32%. The impact of age on total ED resource consumption was smaller in the multivariable model than in the univariable analysis.

Download:

Table 3. Multivariable analysis to predict total ED resource consumption (ln-transformed) in the training set.

The exponentiated coefficients (Coef.) correspond to the GMR.

https://doi.org/10.1371/journal.pone.0247244.t003

With the formula presented in the statistical analysis the section and the values for the coefficients presented in Table 3, for instance, the total ED resource consumption of a 30 year old patient with respiratory symptoms, normal vitals, without any comorbidity, and admitted by the ambulance are estimated to be Out of the coefficients presented in Table 3, the total ED resource consumption score was thus defined as

Validation of the scoring system and sensitivity analysis.

The median resource score was 35.9 (IQR 31.5, 44.2) with a range of 0 to 84.3 in the validation set. The ED resource score was validated by validation of the multivariable model. Table 4 and S8 Appendix show the validation of the final model (model 1), as well as–for sensitivity analysis–the three other models. All sensitivity analysis models only slightly changed the observed R² of 0.54, which means that 54% of the variance in the total ED resources (ln-transformed) is predictable with the model (S8 Appendix).

Download:

Table 4. Median (IQR) of the absolute percentage deviation of the predicted-observed-ratio (APDPOR*) in the different percentile groups of the total ED resource consumption (ln-transformed).

https://doi.org/10.1371/journal.pone.0247244.t004

A linear regression analysis with only triage and age group as predictor variables showed an R² of 23.7%. For patients with average ED resource consumption the median deviation of the predicted values is less than 8%, while the performance is worse in the lower deciles (11.9%-17.4%) as well as in the highest decile (10.9%) (Table 4). This performance measure is similar in all four studied models.

Practical application: ED resource score and quality assurance

As a practical application example, the change in ED resource consumption over the study period was analysed (Table 5).

Download:

Table 5. Relative change of different performance markers at the ED compared to the baseline year 2013.

https://doi.org/10.1371/journal.pone.0247244.t005

The number of visits increased over the years, increasing by 30% compared to 2013. The increase in resource needs reflected by cumulative total ED resource score points (Cum. RSP) was similar, suggesting a uniform increase over all patient resource groups. Compared to the baseline year 2013, in 2014 and 2015, the cumulative total ED resource score points per health care worker (HCW) increased by +9% and +10%, which might indicate a higher performance in the latter years compared to the baseline year.

Length of ED stay

As a secondary outcome the LOS in the ED in hours was studied (LOS-ED). The median LOS-ED was 3.7 h (2.2–5.7) with no difference between the validation and training set (p = 0.996).

The multivariable linear regression model to predict ln-transformed LOS-ED revealed an R² in the validation sample of 0.24 (Table 6). While the chief complaint showed an effect in the same direction as the analysis modelling ED resource consumption, more acute consultations, reflected by resuscitation bay use and temperature deviations, showed a GMR<1 when predicting ln-transformed LOS-ED.

Download:

Table 6. Multivariable analysis to predict ln-transformed LOS-ED in hours (in the training set.

The exponentiated coefficients (Coef.) correspond to the GMR.

https://doi.org/10.1371/journal.pone.0247244.t006

Discussion

In this retrospective analysis of a large Swiss interdisciplinary ED, the distribution of an ED patient’s consumption of resources was quantified, and predictors of total ED resource utilization were determined. Furthermore, this study developed and validated a novel scoring system for resource utilization for patients presenting to the ED that takes data acquired at the very early stages of patient care into consideration, providing better resource prediction than the use of a triage tool alone.

Distribution of ED resource utilization

The distribution of ED resources between physician and imaging as well as laboratory was comparable to international findings [10, 11]. Resource contribution varied significantly according to triage level. The contribution of physician resources was highest for low-acuity patients, and decreased gradually with rising urgency, also comparable to international results [10, 11]. The contribution of ancillary services (laboratory work, imaging studies) showed a reverse result, with lower contribution to total ED resources in low-acuity patients and higher contribution in high-acuity patients. This finding has several implications for cost-containment and adaptive organizational measures, e.g. referral of low-acuity patients in a separate less-resource intense area of the ED and optimizing laboratory and imaging resources for high-acuity patients.

Predicting the total ED resource consumption

Our predictors identified in univariable analysis correspond well to published findings: arrival by ambulance [40], chief complaint group [13], resuscitation bay use, deviation of vital signs [32], and triage level [6, 7, 9–11, 16].

Chief complaint has a large influence on resource utilization, with neurologic complaints showing the biggest impact, probably due to the large amount of diagnostic studies and extensive physician-patient resource needs, comparable to international findings [13], followed by cardiovascular, respiratory, gastrointestinal, and trauma. Ear-nose-throat (ENT) and ophthalmologic complaints were associated with less resource utilization. This may be explained by the fact, that these patients usually require less extensive laboratory and imaging work-up, however, there might be an underrepresentation of specialized procedural codes in the selection of the Tarmed catalogue. A similar finding was reported in a paediatric ENT population [41].

Regarding chronic patient conditions and contextual factors, we found an association of increasing age with resource consumption that is well described in the literature [8–10, 12, 14]. However, in our multivariable linear regression model, this difference was almost negligible. This emphasizes the important fact, that resource consumption is not associated with age per se, but rather with the accompanying relevant multiple comorbidities and polypharmacy, as well as cognitive or functional decline, leading to higher clinical complexity, e.g. liver disease [9].

Drug intake of any antithrombotic, antihypertensive or any opioid medication increased the geometric mean of total resource consumption about 18%, comparable to previous publications. An emergency ward setting in a tertiary hospital in Sweden reported cardiovascular medications and antithrombotic agents among the top three common drugs causing or contributing to admission [42]. Individuals with high-risk prescription opioid use are known to have significantly higher healthcare costs and utilization than their counterparts [43], and we recently demonstrated multi-substance users need significantly more ED resources than age-matched controls [44].

Similar to our predictors of resource utilization, data derived from the United States National Hospital Ambulatory Medical Care Survey also demonstrated age, triage level, arrival mode, and certain comorbidities (cerebrovascular disease, dementia) to be predictive of the eventual use of advanced diagnostic imaging in the ED [45].

With the multivariable model using easily available acute patient presentation factors as well as markers of chronic patient condition, 54% of the variance of resource utilization could be explained by the variables available at the early stages of patient presentation. These findings were verified in multiple internal sensitive analyses using different models. The identified model performed much better than a linear regression analysis with only triage and age group, two well-known predictors of resource utilization, with much poorer prediction (R² = 0.24).

Furthermore, we found a high correlation of actual resource utilization with cost. However, due to very different national healthcare and billing systems [46], the results from this Swiss ER setting cannot simply be generalized to other countries, and further international studies are needed to determine, if a pure cost analysis (were data usually is more easily available) sufficiently reflects the actual resource utilization of an ED patient.

Predicting length of ED stay

Resource consumption is difficult to measure. Some might argue that the Swiss Tarmed codes do not validly reflect resource consumption as a “resource measure” is already assigned to each process. Thus, we additionally evaluated the variable LOS-ED as an outcome parameter. At first glance, this might be a valid outcome variable to reflect ED resource consumption. Chief complaint showed the same direction of effect as the analysis modelling ED resource consumption, undermining the robustness of the parameter. However, a multitude of factors affect patients’ LOS-ED [13], i.e. simulation-based training for sedation procedures [47], or the use of ED observation units [48]. Furthermore, parallel and high-priority work-up of the patient is not at all reflected by LOS. High-acuity patients, treated in the resuscitation bay, are usually rapidly transferred to definite care (operating theatre, intensive/intermediate care unit), thus having a short LOS-ED, but are very resource intensive. Therefore, LOS-ED is not a suitable measure to reflect resource consumption.

Development and validation of a scoring system

Using the variables derived in the multivariable regression, we developed and validated a novel scoring system for total ED resource consumption, taking initial information at patient presentation and the patient profile into account, explaining more than half of the variation in total ED resource consumption. These insights are crucial especially in times of resource shortages, not only for ED physicians and managers but hospital administrators and economics as well. The resource score, when applied at time of ED triage has the potential to better identify areas and special patient groups with high resource demands at an early stage.

Analogous to the TISS in the intensive care setting this score can be used be reflect resource consumption in the ED. Whereas the TISS is composed of tasks and chores actually conducted this resource score predicts ED resource consumption from the initial patient presentation, and thus it can be used to guide patient management in the ED beyond triage-category alone, i.e. assisting patient-flow by locating the patient in a more or less resource intensive area of the ED, improving medical decision-making, and providing efficient and sustainable health care. Furthermore, instead of simply presenting the volume of ED patients, it might be valuable in research as a standardised benchmarking tool to describe the service-performance of an ED and providing resource-adjusted inter-institutional cost and performance comparisons. Analogous to the TISS, which is used not only for performance measurement but also reimbursement, this score can provide hospital administrations with valuable information regarding cost generation, appropriate invoicing of emergency services provided, as well as workforce planning. Additional international multicentre studies are needed to determine how the application of such a scoring system can optimize ED resource allocation and consumption, thus providing optimal sustainable emergency care, quality assurance, and improvement.

External validation

External validation of the suggested scoring system is the next step necessary for investigating generalizability. The predictors identified in univariable analysis correspond well to published findings also in less resource-intense settings (i.e. Lebanon) [10], which underlines a possible generalizability. Furthermore, as the score uses clinical predictors whose data collection can easily be integrated into the daily routine, e.g. triage process, it can be used in ED settings that do not collect such granular resource consumption data. The verification of external validity requires further prospective multicentre and international studies due to different intergovernmental health care and billing systems and patient populations. For example, our ED does not work under a 4-hour rule, transferring patients who need longer diagnostic work-up to an observation/short-stay unit, but takes care of the whole treatment process. Additionally, health spending varies significantly among different countries, and Switzerland is near the top of the range [46].

Study limitations and strengths

The interpretation of our results warrants some caveats. First, data are derived from a single level one trauma and adult tertiary care referral centre, albeit one of the largest in Switzerland, making results less generalizable. Furthermore, this ED includes a large neurological referral and stroke center, a patient group prone to using a lot of resources, adding a selection bias. Next, in our definition of utilized resources we only include the direct medical resources actually documented in the ED by physician and nursing staff, as well as ancillary services, thus not considering overhead resources (i.e. infrastructure, maintenance, and hospital security). However, we have detailed records of our resource documentation, our staff is regularly trained in the documentation process, and controlling assures completeness of the documentation and circumvents variability in physician documentation practice. We focussed only on the ED process and did not include hospitalization-related resources/costs. This may lead to a partial underrepresentation of resources utilized in the third of the total patient population that required hospitalization, as some diagnostic or therapeutic measures may have been postponed. Moreover, comorbidities and medications were not standardized and automatically collected, but derived by full text parsing. Nonetheless, this was validated against manual coding. Besides that, due to a different documentation and billing system we had to exclude patients seen solely by the psychiatrist, resulting in possible selection bias, as psychiatric comorbidity may be one reason for excessive physician resource utilization [49]. Finally, whereas we recently found that resource utilization is in large part dependent on the physicians’ ratings of case difficulty (i.e. their situational level of uncertainty, familiarity and perceived difficulty), we did not include these variables in our study, which focusses on data easily available early in the patient ED presentation [50].

Conclusions

As ED visits and health care costs are increasing globally, it is of paramount importance to understand the components of ED resource utilization, particular in times of resource scarcity. In this large retrospective study at an interdisciplinary ED, the distribution of ED resource utilization was illustrated. The novel ED resource score developed has manifold potential uses, such as an instrument i) that allows leaders in emergency care to evaluate resource decisions and to estimate effects of organizational changes, ii) to calculate benchmarks of an ED in research and process optimisation, iii) to identify resource-intensive patients more rapidly and comprehensively than triage-category alone, iv) to develop cost-containment and quality-improvement measures, and iv) that represents an easy and fast internal billing system analogous to the TISS in the intensive care unit.

Supporting information

S1 Appendix. The patient management process in our ED.

https://doi.org/10.1371/journal.pone.0247244.s001

(DOCX)

S2 Appendix. Data extraction plan and definition of the variables.

https://doi.org/10.1371/journal.pone.0247244.s002

(DOCX)

S3 Appendix. Validation of the diagnosis and drug parser based on agreement with 500 manually coded ED reports.

https://doi.org/10.1371/journal.pone.0247244.s003

(DOCX)

S4 Appendix. Transformation scheme of the chief complaints by Aronsky et al. [31] to the different chief complaint main groups.

https://doi.org/10.1371/journal.pone.0247244.s004

(DOCX)

S5 Appendix. Flowchart of the study.

https://doi.org/10.1371/journal.pone.0247244.s005

(DOCX)

S6 Appendix. Baseline characteristics according to the type of set (Training set: n = 82,388 and validation set: n = 82,341).

https://doi.org/10.1371/journal.pone.0247244.s006

(DOCX)

S7 Appendix. Distribution of ED resource consumption according to type of set.

The median with IQR (whiskers) is shown.

https://doi.org/10.1371/journal.pone.0247244.s007

(DOCX)

S8 Appendix. Model fitting, predictive accuracy, and explained variance of the four different models (Model 1: Final model, Model 2: Triage instead of vital parameters, Model 3: Interaction term between trauma complaint and resuscitation room use, Model 4: Exclusion of revisits.

https://doi.org/10.1371/journal.pone.0247244.s008

(DOCX)

Acknowledgments

We acknowledge the following people’s support in selecting the procedural codes and data extraction: Sebastian Becker, Martina Siffert, Kathrin Dopke, Smilijana Balli, Sabine Schuh, Gert Krummrey, Michael Gygax, and Jolanta Klukowska-Röetzler.

References

1. Global Burden of Disease Health Financing Collaborator Network. Past, present, and future of global health financing: a review of development assistance, government, out-of-pocket, and other private spending on health for 195 countries, 1995–2050. Lancet. 2019 Jun 1;393(10187):2233–60. pmid:31030984
- View Article
- PubMed/NCBI
- Google Scholar
2. Marcozzi D, Carr B, Liferidge A, Baehr N, Browne B. Trends in the Contribution of Emergency Departments to the Provision of Hospital-Associated Health Care in the USA. Int J Health Serv. 2018;48(2):267–88. pmid:29039720
- View Article
- PubMed/NCBI
- Google Scholar
3. Pitts SR, Pines JM, Handrigan MT, Kellermann AL. National trends in emergency department occupancy, 2001 to 2008: effect of inpatient admissions versus emergency department practice intensity. Ann Emerg Med. 2012 Dec;60(6):679-686.e3. pmid:22727201
- View Article
- PubMed/NCBI
- Google Scholar
4. Dale J, Lang H, Roberts JA, Green J, Glucksman E. Cost effectiveness of treating primary care patients in accident and emergency: a comparison between general practitioners, senior house officers, and registrars. BMJ. 1996 May 25;312(7042):1340–4. pmid:8646050
- View Article
- PubMed/NCBI
- Google Scholar
5. NICE. COVID-19 rapid guideline: critical care in adults [Internet]. [cited 2020 Jun 29]. Available from: https://www.nice.org.uk/guidance/ng159
6. Wiler JL, Poirier RF, Farley H, Zirkin W, Griffey RT. Emergency severity index triage system correlation with emergency department evaluation and management billing codes and total professional charges. Acad Emerg Med. 2011;18(11):1161–6. pmid:22092897
- View Article
- PubMed/NCBI
- Google Scholar
7. Hocker MB, Gerardo CJ, Theiling BJ, Villani J, Donohoe R, Sandesara H, et al. NHAMCS Validation of Emergency Severity Index as an Indicator of Emergency Department Resource Utilization. West J Emerg Med. 2018;19(5):855–62. pmid:30202499
- View Article
- PubMed/NCBI
- Google Scholar
8. Latham LP, Ackroyd-Stolarz S. Emergency department utilization by older adults: a descriptive study. Can Geriatr J. 2014 Dec;17(4):118–25. pmid:25452824
- View Article
- PubMed/NCBI
- Google Scholar
9. Chang JC-Y, Yuan Z-H, Lee I-H, Hsu T-F, How C-K, Yen DH-T. Pattern of non-trauma emergency department resource utilization in older adults: An 8-year experience in Taiwan. J Chin Med Assoc. 2018 Jun;81(6):552–8. pmid:29395945
- View Article
- PubMed/NCBI
- Google Scholar
10. Saleh S, Mourad Y, Dimassi H, Hitti E. Distribution and predictors of emergency department charges: the case of a tertiary hospital in Lebanon. BMC Health Serv Res. 2016 Dec;16(1):97. pmid:26993108
- View Article
- PubMed/NCBI
- Google Scholar
11. Williams RM. Distribution of emergency department costs. Ann Emerg Med. 1996 Dec;28(6):671–6. pmid:8953958
- View Article
- PubMed/NCBI
- Google Scholar
12. Henneman PL, Nathanson BH, Ribeiro K, Balasubramanian H. The impact of age and gender on resource utilization and profitability in ED patients seen and released. Am J Emerg Med. 2014 Oct;32(10):1159–67. pmid:25135676
- View Article
- PubMed/NCBI
- Google Scholar
13. d’Etienne JP, Zhou Y, Kan C, Shaikh S, Ho AF, Suley E, et al. Two-step predictive model for early detection of emergency department patients with prolonged stay and its management implications. Am J Emerg Med. 2020 Jan;(in press). pmid:32063427
- View Article
- PubMed/NCBI
- Google Scholar
14. Burkett E, Martin-Khan MG, Gray LC. Comparative emergency department resource utilisation across age groups. Aust Health Rev. 2019;43(2):194. pmid:29224590
- View Article
- PubMed/NCBI
- Google Scholar
15. Ondler C, Hegde GG, Carlson JN. Resource utilization and health care charges associated with the most frequent ED users. Am J Emerg Med. 2014 Oct;32(10):1215–9. pmid:25154348
- View Article
- PubMed/NCBI
- Google Scholar
16. Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs. 2004 Feb;30(1):22–9. pmid:14765078
- View Article
- PubMed/NCBI
- Google Scholar
17. Welch SJ, Augustine JJ, Dong L, Savitz LA, Snow G, James BC. Volume-Related Differences in Emergency Department Performance. Jt Comm J Qual Patient Saf. 2020;38(9):395–402.
- View Article
- Google Scholar
18. Cullen DJ, Civetta JM, Briggs BA, Ferrara LC. Therapeutic intervention scoring system: a method for quantitative comparison of patient care. Crit Care Med. 1974;2(2):57–60. pmid:4832281
- View Article
- PubMed/NCBI
- Google Scholar
19. Lefering R, Zart M, Neugebauer EA. Retrospective evaluation of the simplified Therapeutic Intervention Scoring System (TISS-28) in a surgical intensive care unit. Intensive Care Med. 2000;26(12):1794–802. pmid:11271087
- View Article
- PubMed/NCBI
- Google Scholar
20. Muehler N, Oishi J, Specht M, Rissner F, Reinhart K, Sakr Y. Serial measurement of Therapeutic Intervention Scoring System-28 (TISS-28) in a surgical intensive care unit. J Crit Care. 2010;25(4):620–7. pmid:20442008
- View Article
- PubMed/NCBI
- Google Scholar
21. Hariharan S, Chen D, Merritt-Charles L, Bobb N, DeFreitas L, Esdelle-Thomas JM, et al. The utilities of the therapeutic intervention scoring system (TISS-28). Indian J Crit Care Med. 2007;11(2):61–6.
- View Article
- Google Scholar
22. Graf J, Graf C, Koch K-C, Hanrath P, Janssens U. Kostenanalyse und Prognoseabschätzung internistischer Intensivpatienten mittels des “Therapeutic Intervention Scoring System” (TISS und TISS-28). Med Klin. 2003;98(3):123–32.
- View Article
- Google Scholar
23. Metnitz PG, Vesely H, Valentin A, Popow C, Hiesmayr M, Lenz K, et al. Evaluation of an interdisciplinary data set for national intensive care unit assessment. Crit Care Med. 1999;27(8):1486–91. pmid:10470754
- View Article
- PubMed/NCBI
- Google Scholar
24. Jakob SM, Lubszky S, Friolet R, Rothen HU, Kolarova A, Takala J. Sedation and weaning from mechanical ventilation: effects of process optimization outside a clinical trial. J Crit Care. 2007;22(3):219–28. pmid:17869972
- View Article
- PubMed/NCBI
- Google Scholar
25. Exadaktylos AK, Hautz WE. Emergency Medicine in Switzerland. ICU Manag Pr. 2015;15.
- View Article
- Google Scholar
26. TARMED Suisse. TARMED 01.08.0000. 2010.
27. Jobé J, Donneau A-F, Scholtes B, Ghuysen A. Quantifying emergency department crowding: comparison between two scores. Acta Clin Belg. 2018 May 4;73(3):207–12. pmid:29207925
- View Article
- PubMed/NCBI
- Google Scholar
28. McCarthy ML, Aronsky D, Jones ID, Miner JR, Band RA, Baren JM, et al. The Emergency Department Occupancy Rate: A Simple Measure of Emergency Department Crowding? Ann Emerg Med. 2008 Jan;51(1):15-24.e2. pmid:17980458
- View Article
- PubMed/NCBI
- Google Scholar
29. Bernstein SL. Development and Validation of a New Index to Measure Emergency Department Crowding. Acad Emerg Med. 2003 Sep 1;10(9):938–42. pmid:12957975
- View Article
- PubMed/NCBI
- Google Scholar
30. Asplin BR, Magid DJ, Rhodes KV, Solberg LI, Lurie N, Camargo CA. A conceptual model of emergency department crowding. Ann Emerg Med. 2003 Aug;42(2):173–80. pmid:12883504
- View Article
- PubMed/NCBI
- Google Scholar
31. Aronsky D, Kendall D, Merkley K, James BC, Haug PJ. A comprehensive set of coded chief complaints for the emergency department. Acad Emerg Med. 2001;8(10):980–9. pmid:11581085
- View Article
- PubMed/NCBI
- Google Scholar
32. Ljunggren M, Castrén M, Nordberg M, Kurland L. The association between vital signs and mortality in a retrospective cohort study of an unselected emergency department population. SJTREM. 2016;24. pmid:26955943
- View Article
- PubMed/NCBI
- Google Scholar
33. Rutschmann OT, Hugli OW, Marti C, Grosgurin O, Geissbuhler A, Kossovsky M, et al. Reliability of the revised Swiss Emergency Triage Scale: a computer simulation study. Eur J Emerg Med. 2017;25(4):264–269.
- View Article
- Google Scholar
34. Mackway-Jones K, Marsden J, Windle J. Emergency triage: Manchester triage group. John Wiley & Sons; 2014.
35. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. pmid:3558716
- View Article
- PubMed/NCBI
- Google Scholar
36. WHO Collaborating Centre for Drug Statistics Methodology. ATC/DDD index [Internet]. 2018 [cited 2019 May 16]. Available from: https://www.whocc.no/atc_ddd_index/
37. Benoit K. Linear regression models with logarithmic transformations. Lond Sch Econ Lond. 2011;22(1):23–36.
- View Article
- Google Scholar
38. Austin PC, Ghali WA, Tu JV. A comparison of several regression models for analysing cost of CABG surgery. Stat Med. 2003;22(17):2799–815. pmid:12939787
- View Article
- PubMed/NCBI
- Google Scholar
39. Lapi F, Bianchini E, Cricelli I, Trifirò G, Mazzaglia G, Cricelli C. Development and Validation of a Score for Adjusting Health Care Costs in General Practice. Value Health. 2015;18(6):884–95. pmid:26409617
- View Article
- PubMed/NCBI
- Google Scholar
40. Ruger JP, Richter CJ, Lewis LM. Clinical and Economic Factors Associated with Ambulance Use to the Emergency Department. Acad Emerg Med. 2006 Aug;13(8):879–85. pmid:16825670
- View Article
- PubMed/NCBI
- Google Scholar
41. Fessler SJ, Simon HK, Yancey AH, Colman M, Hirsh DA. How well do General EMS 911 dispatch protocols predict ED resource utilization for pediatric patients? Am J Emerg Med. 2014 Mar;32(3):199–202. pmid:24370070
- View Article
- PubMed/NCBI
- Google Scholar
42. Rydberg DM, Holm L, Engqvist I, Fryckstedt J, Lindh JD, Stiller C-O, et al. Adverse Drug Reactions in a Tertiary Care Emergency Medicine Ward—Prevalence, Preventability and Reporting. PLoS ONE. 2016;11(9):e0162948. pmid:27622270
- View Article
- PubMed/NCBI
- Google Scholar
43. Chang H-Y, Kharrazi H, Bodycombe D, Weiner JP, Alexander GC. Healthcare costs and utilization associated with high-risk prescription opioid use: a retrospective cohort study. BMC Med. 2018 May 16;16(69). pmid:29764482
- View Article
- PubMed/NCBI
- Google Scholar
44. Klenk L, Rütte C von, Henssler JF, Sauter TC, Hautz WE, Exadaktylos AK, et al. Resource consumption of multi-substance users in the emergency room: A neglected patient group. PLoS ONE. 2019;14(9):e0223118. pmid:31557239
- View Article
- PubMed/NCBI
- Google Scholar
45. Zhang X, Kim J, Patzer RE, Pitts SR, Chokshi FH, Schrager JD. Advanced diagnostic imaging utilization during emergency department visits in the United States: A predictive modeling study for emergency department triage. PLoS ONE. 2019;14(4):e0214905. pmid:30964899
- View Article
- PubMed/NCBI
- Google Scholar
46. OECD. Health at a Glance 2019: OECD Indicators [Internet]. Paris: OECD Publishing; 2019 [cited 2020 Jun 7]. Available from: https://doi.org/10.1787/4dd50c09-en
47. Sauter TC, Hautz WE, Hostettler S, Brodmann-Maeder M, Martinolli L, Lehmann B, et al. Interprofessional and interdisciplinary simulation-based training leads to safe sedation procedures in the emergency department. SJTREM. 2016;24(1):97. pmid:27485431
- View Article
- PubMed/NCBI
- Google Scholar
48. Williams J, Aurora T, Baker K, Thompson J, Smallheer B. Triage to Observation: A Quality Improvement Initiative for Chest Pain Patients Presenting to the Emergency Department. Crit Pathw Cardiol. 2019;18(2):75–9. pmid:31094733
- View Article
- PubMed/NCBI
- Google Scholar
49. Curran GM, Sullivan G, Williams K, Han X, Collins K, Keys J, et al. Emergency department use of persons with comorbid psychiatric and substance abuse disorders. Ann Emerg Med. 2003 May;41(5):659–67. pmid:12712033
- View Article
- PubMed/NCBI
- Google Scholar
50. Hautz WE, Sauter TC, Hautz SC, Kämmer JE, Schauber SK, Birrenbach T, et al. What determines diagnostic resource consumption in emergency medicine: patients, physicians or context? Emerg Med J. 2020 Jul 9;0(1–6). pmid:32647026
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Global Burden of Disease Health Financing Collaborator Network. Past, present, and future of global health financing: a review of development assistance, government, out-of-pocket, and other private spending on health for 195 countries, 1995–2050. Lancet. 2019 Jun 1;393(10187):2233–60. pmid:31030984
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Marcozzi D, Carr B, Liferidge A, Baehr N, Browne B. Trends in the Contribution of Emergency Departments to the Provision of Hospital-Associated Health Care in the USA. Int J Health Serv. 2018;48(2):267–88. pmid:29039720
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Pitts SR, Pines JM, Handrigan MT, Kellermann AL. National trends in emergency department occupancy, 2001 to 2008: effect of inpatient admissions versus emergency department practice intensity. Ann Emerg Med. 2012 Dec;60(6):679-686.e3. pmid:22727201
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Dale J, Lang H, Roberts JA, Green J, Glucksman E. Cost effectiveness of treating primary care patients in accident and emergency: a comparison between general practitioners, senior house officers, and registrars. BMJ. 1996 May 25;312(7042):1340–4. pmid:8646050
View Article
PubMed/NCBI
Google Scholar

[14] View Article

[15] PubMed/NCBI

[16] Google Scholar

[ref5] 5. NICE. COVID-19 rapid guideline: critical care in adults [Internet]. [cited 2020 Jun 29]. Available from: https://www.nice.org.uk/guidance/ng159

[ref6] 6. Wiler JL, Poirier RF, Farley H, Zirkin W, Griffey RT. Emergency severity index triage system correlation with emergency department evaluation and management billing codes and total professional charges. Acad Emerg Med. 2011;18(11):1161–6. pmid:22092897
View Article
PubMed/NCBI
Google Scholar

[19] View Article

[20] PubMed/NCBI

[21] Google Scholar

[ref7] 7. Hocker MB, Gerardo CJ, Theiling BJ, Villani J, Donohoe R, Sandesara H, et al. NHAMCS Validation of Emergency Severity Index as an Indicator of Emergency Department Resource Utilization. West J Emerg Med. 2018;19(5):855–62. pmid:30202499
View Article
PubMed/NCBI
Google Scholar

[23] View Article

[24] PubMed/NCBI

[25] Google Scholar

[ref8] 8. Latham LP, Ackroyd-Stolarz S. Emergency department utilization by older adults: a descriptive study. Can Geriatr J. 2014 Dec;17(4):118–25. pmid:25452824
View Article
PubMed/NCBI
Google Scholar

[27] View Article

[28] PubMed/NCBI

[29] Google Scholar

[ref9] 9. Chang JC-Y, Yuan Z-H, Lee I-H, Hsu T-F, How C-K, Yen DH-T. Pattern of non-trauma emergency department resource utilization in older adults: An 8-year experience in Taiwan. J Chin Med Assoc. 2018 Jun;81(6):552–8. pmid:29395945
View Article
PubMed/NCBI
Google Scholar

[31] View Article

[32] PubMed/NCBI

[33] Google Scholar

[ref10] 10. Saleh S, Mourad Y, Dimassi H, Hitti E. Distribution and predictors of emergency department charges: the case of a tertiary hospital in Lebanon. BMC Health Serv Res. 2016 Dec;16(1):97. pmid:26993108
View Article
PubMed/NCBI
Google Scholar

[35] View Article

[36] PubMed/NCBI

[37] Google Scholar

[ref11] 11. Williams RM. Distribution of emergency department costs. Ann Emerg Med. 1996 Dec;28(6):671–6. pmid:8953958
View Article
PubMed/NCBI
Google Scholar

[39] View Article

[40] PubMed/NCBI

[41] Google Scholar

[ref12] 12. Henneman PL, Nathanson BH, Ribeiro K, Balasubramanian H. The impact of age and gender on resource utilization and profitability in ED patients seen and released. Am J Emerg Med. 2014 Oct;32(10):1159–67. pmid:25135676
View Article
PubMed/NCBI
Google Scholar

[43] View Article

[44] PubMed/NCBI

[45] Google Scholar

[ref13] 13. d’Etienne JP, Zhou Y, Kan C, Shaikh S, Ho AF, Suley E, et al. Two-step predictive model for early detection of emergency department patients with prolonged stay and its management implications. Am J Emerg Med. 2020 Jan;(in press). pmid:32063427
View Article
PubMed/NCBI
Google Scholar

[47] View Article

[48] PubMed/NCBI

[49] Google Scholar

[ref14] 14. Burkett E, Martin-Khan MG, Gray LC. Comparative emergency department resource utilisation across age groups. Aust Health Rev. 2019;43(2):194. pmid:29224590
View Article
PubMed/NCBI
Google Scholar

[51] View Article

[52] PubMed/NCBI

[53] Google Scholar

[ref15] 15. Ondler C, Hegde GG, Carlson JN. Resource utilization and health care charges associated with the most frequent ED users. Am J Emerg Med. 2014 Oct;32(10):1215–9. pmid:25154348
View Article
PubMed/NCBI
Google Scholar

[55] View Article

[56] PubMed/NCBI

[57] Google Scholar

[ref16] 16. Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs. 2004 Feb;30(1):22–9. pmid:14765078
View Article
PubMed/NCBI
Google Scholar

[59] View Article

[60] PubMed/NCBI

[61] Google Scholar

[ref17] 17. Welch SJ, Augustine JJ, Dong L, Savitz LA, Snow G, James BC. Volume-Related Differences in Emergency Department Performance. Jt Comm J Qual Patient Saf. 2020;38(9):395–402.
View Article
Google Scholar

[63] View Article

[64] Google Scholar

[ref18] 18. Cullen DJ, Civetta JM, Briggs BA, Ferrara LC. Therapeutic intervention scoring system: a method for quantitative comparison of patient care. Crit Care Med. 1974;2(2):57–60. pmid:4832281
View Article
PubMed/NCBI
Google Scholar

[66] View Article

[67] PubMed/NCBI

[68] Google Scholar

[ref19] 19. Lefering R, Zart M, Neugebauer EA. Retrospective evaluation of the simplified Therapeutic Intervention Scoring System (TISS-28) in a surgical intensive care unit. Intensive Care Med. 2000;26(12):1794–802. pmid:11271087
View Article
PubMed/NCBI
Google Scholar

[70] View Article

[71] PubMed/NCBI

[72] Google Scholar

[ref20] 20. Muehler N, Oishi J, Specht M, Rissner F, Reinhart K, Sakr Y. Serial measurement of Therapeutic Intervention Scoring System-28 (TISS-28) in a surgical intensive care unit. J Crit Care. 2010;25(4):620–7. pmid:20442008
View Article
PubMed/NCBI
Google Scholar

[74] View Article

[75] PubMed/NCBI

[76] Google Scholar

[ref21] 21. Hariharan S, Chen D, Merritt-Charles L, Bobb N, DeFreitas L, Esdelle-Thomas JM, et al. The utilities of the therapeutic intervention scoring system (TISS-28). Indian J Crit Care Med. 2007;11(2):61–6.
View Article
Google Scholar

[78] View Article

[79] Google Scholar

[ref22] 22. Graf J, Graf C, Koch K-C, Hanrath P, Janssens U. Kostenanalyse und Prognoseabschätzung internistischer Intensivpatienten mittels des “Therapeutic Intervention Scoring System” (TISS und TISS-28). Med Klin. 2003;98(3):123–32.
View Article
Google Scholar

[81] View Article

[82] Google Scholar

[ref23] 23. Metnitz PG, Vesely H, Valentin A, Popow C, Hiesmayr M, Lenz K, et al. Evaluation of an interdisciplinary data set for national intensive care unit assessment. Crit Care Med. 1999;27(8):1486–91. pmid:10470754
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref24] 24. Jakob SM, Lubszky S, Friolet R, Rothen HU, Kolarova A, Takala J. Sedation and weaning from mechanical ventilation: effects of process optimization outside a clinical trial. J Crit Care. 2007;22(3):219–28. pmid:17869972
View Article
PubMed/NCBI
Google Scholar

[88] View Article

[89] PubMed/NCBI

[90] Google Scholar

[ref25] 25. Exadaktylos AK, Hautz WE. Emergency Medicine in Switzerland. ICU Manag Pr. 2015;15.
View Article
Google Scholar

[92] View Article

[93] Google Scholar

[ref26] 26. TARMED Suisse. TARMED 01.08.0000. 2010.

[ref27] 27. Jobé J, Donneau A-F, Scholtes B, Ghuysen A. Quantifying emergency department crowding: comparison between two scores. Acta Clin Belg. 2018 May 4;73(3):207–12. pmid:29207925
View Article
PubMed/NCBI
Google Scholar

[96] View Article

[97] PubMed/NCBI

[98] Google Scholar

[ref28] 28. McCarthy ML, Aronsky D, Jones ID, Miner JR, Band RA, Baren JM, et al. The Emergency Department Occupancy Rate: A Simple Measure of Emergency Department Crowding? Ann Emerg Med. 2008 Jan;51(1):15-24.e2. pmid:17980458
View Article
PubMed/NCBI
Google Scholar

[100] View Article

[101] PubMed/NCBI

[102] Google Scholar

[ref29] 29. Bernstein SL. Development and Validation of a New Index to Measure Emergency Department Crowding. Acad Emerg Med. 2003 Sep 1;10(9):938–42. pmid:12957975
View Article
PubMed/NCBI
Google Scholar

[104] View Article

[105] PubMed/NCBI

[106] Google Scholar

[ref30] 30. Asplin BR, Magid DJ, Rhodes KV, Solberg LI, Lurie N, Camargo CA. A conceptual model of emergency department crowding. Ann Emerg Med. 2003 Aug;42(2):173–80. pmid:12883504
View Article
PubMed/NCBI
Google Scholar

[108] View Article

[109] PubMed/NCBI

[110] Google Scholar

[ref31] 31. Aronsky D, Kendall D, Merkley K, James BC, Haug PJ. A comprehensive set of coded chief complaints for the emergency department. Acad Emerg Med. 2001;8(10):980–9. pmid:11581085
View Article
PubMed/NCBI
Google Scholar

[112] View Article

[113] PubMed/NCBI

[114] Google Scholar

[ref32] 32. Ljunggren M, Castrén M, Nordberg M, Kurland L. The association between vital signs and mortality in a retrospective cohort study of an unselected emergency department population. SJTREM. 2016;24. pmid:26955943
View Article
PubMed/NCBI
Google Scholar

[116] View Article

[117] PubMed/NCBI

[118] Google Scholar

[ref33] 33. Rutschmann OT, Hugli OW, Marti C, Grosgurin O, Geissbuhler A, Kossovsky M, et al. Reliability of the revised Swiss Emergency Triage Scale: a computer simulation study. Eur J Emerg Med. 2017;25(4):264–269.
View Article
Google Scholar

[120] View Article

[121] Google Scholar

[ref34] 34. Mackway-Jones K, Marsden J, Windle J. Emergency triage: Manchester triage group. John Wiley & Sons; 2014.

[ref35] 35. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. pmid:3558716
View Article
PubMed/NCBI
Google Scholar

[124] View Article

[125] PubMed/NCBI

[126] Google Scholar

[ref36] 36. WHO Collaborating Centre for Drug Statistics Methodology. ATC/DDD index [Internet]. 2018 [cited 2019 May 16]. Available from: https://www.whocc.no/atc_ddd_index/

[ref37] 37. Benoit K. Linear regression models with logarithmic transformations. Lond Sch Econ Lond. 2011;22(1):23–36.
View Article
Google Scholar

[129] View Article

[130] Google Scholar

[ref38] 38. Austin PC, Ghali WA, Tu JV. A comparison of several regression models for analysing cost of CABG surgery. Stat Med. 2003;22(17):2799–815. pmid:12939787
View Article
PubMed/NCBI
Google Scholar

[132] View Article

[133] PubMed/NCBI

[134] Google Scholar

[ref39] 39. Lapi F, Bianchini E, Cricelli I, Trifirò G, Mazzaglia G, Cricelli C. Development and Validation of a Score for Adjusting Health Care Costs in General Practice. Value Health. 2015;18(6):884–95. pmid:26409617
View Article
PubMed/NCBI
Google Scholar

[136] View Article

[137] PubMed/NCBI

[138] Google Scholar

[ref40] 40. Ruger JP, Richter CJ, Lewis LM. Clinical and Economic Factors Associated with Ambulance Use to the Emergency Department. Acad Emerg Med. 2006 Aug;13(8):879–85. pmid:16825670
View Article
PubMed/NCBI
Google Scholar

[140] View Article

[141] PubMed/NCBI

[142] Google Scholar

[ref41] 41. Fessler SJ, Simon HK, Yancey AH, Colman M, Hirsh DA. How well do General EMS 911 dispatch protocols predict ED resource utilization for pediatric patients? Am J Emerg Med. 2014 Mar;32(3):199–202. pmid:24370070
View Article
PubMed/NCBI
Google Scholar

[144] View Article

[145] PubMed/NCBI

[146] Google Scholar

[ref42] 42. Rydberg DM, Holm L, Engqvist I, Fryckstedt J, Lindh JD, Stiller C-O, et al. Adverse Drug Reactions in a Tertiary Care Emergency Medicine Ward—Prevalence, Preventability and Reporting. PLoS ONE. 2016;11(9):e0162948. pmid:27622270
View Article
PubMed/NCBI
Google Scholar

[148] View Article

[149] PubMed/NCBI

[150] Google Scholar

[ref43] 43. Chang H-Y, Kharrazi H, Bodycombe D, Weiner JP, Alexander GC. Healthcare costs and utilization associated with high-risk prescription opioid use: a retrospective cohort study. BMC Med. 2018 May 16;16(69). pmid:29764482
View Article
PubMed/NCBI
Google Scholar

[152] View Article

[153] PubMed/NCBI

[154] Google Scholar

[ref44] 44. Klenk L, Rütte C von, Henssler JF, Sauter TC, Hautz WE, Exadaktylos AK, et al. Resource consumption of multi-substance users in the emergency room: A neglected patient group. PLoS ONE. 2019;14(9):e0223118. pmid:31557239
View Article
PubMed/NCBI
Google Scholar

[156] View Article

[157] PubMed/NCBI

[158] Google Scholar

[ref45] 45. Zhang X, Kim J, Patzer RE, Pitts SR, Chokshi FH, Schrager JD. Advanced diagnostic imaging utilization during emergency department visits in the United States: A predictive modeling study for emergency department triage. PLoS ONE. 2019;14(4):e0214905. pmid:30964899
View Article
PubMed/NCBI
Google Scholar

[160] View Article

[161] PubMed/NCBI

[162] Google Scholar

[ref46] 46. OECD. Health at a Glance 2019: OECD Indicators [Internet]. Paris: OECD Publishing; 2019 [cited 2020 Jun 7]. Available from: https://doi.org/10.1787/4dd50c09-en

[ref47] 47. Sauter TC, Hautz WE, Hostettler S, Brodmann-Maeder M, Martinolli L, Lehmann B, et al. Interprofessional and interdisciplinary simulation-based training leads to safe sedation procedures in the emergency department. SJTREM. 2016;24(1):97. pmid:27485431
View Article
PubMed/NCBI
Google Scholar

[165] View Article

[166] PubMed/NCBI

[167] Google Scholar

[ref48] 48. Williams J, Aurora T, Baker K, Thompson J, Smallheer B. Triage to Observation: A Quality Improvement Initiative for Chest Pain Patients Presenting to the Emergency Department. Crit Pathw Cardiol. 2019;18(2):75–9. pmid:31094733
View Article
PubMed/NCBI
Google Scholar

[169] View Article

[170] PubMed/NCBI

[171] Google Scholar

[ref49] 49. Curran GM, Sullivan G, Williams K, Han X, Collins K, Keys J, et al. Emergency department use of persons with comorbid psychiatric and substance abuse disorders. Ann Emerg Med. 2003 May;41(5):659–67. pmid:12712033
View Article
PubMed/NCBI
Google Scholar

[173] View Article

[174] PubMed/NCBI

[175] Google Scholar

[ref50] 50. Hautz WE, Sauter TC, Hautz SC, Kämmer JE, Schauber SK, Birrenbach T, et al. What determines diagnostic resource consumption in emergency medicine: patients, physicians or context? Emerg Med J. 2020 Jul 9;0(1–6). pmid:32647026
View Article
PubMed/NCBI
Google Scholar

[177] View Article

[178] PubMed/NCBI

[179] Google Scholar

Figures

Abstract

Background

Methods

Results

Conclusions

Introduction

Methods

Study design, site and period

Eligibility criteria

ED resource consumption

Potential predictor variables

Data extraction

Ethical considerations

Statistical analysis

Results

Patients’ demographics

Distribution of ED resource consumption.

Prediction of total ED resource consumption

Development of a scoring system

Validation of the scoring system and sensitivity analysis.

Practical application: ED resource score and quality assurance

Length of ED stay

Discussion

Distribution of ED resource utilization

Predicting the total ED resource consumption

Predicting length of ED stay

Development and validation of a scoring system

External validation

Study limitations and strengths

Conclusions

Supporting information

S1 Appendix. The patient management process in our ED.

S2 Appendix. Data extraction plan and definition of the variables.

S3 Appendix. Validation of the diagnosis and drug parser based on agreement with 500 manually coded ED reports.

S4 Appendix. Transformation scheme of the chief complaints by Aronsky et al. [31] to the different chief complaint main groups.

S5 Appendix. Flowchart of the study.

S6 Appendix. Baseline characteristics according to the type of set (Training set: n = 82,388 and validation set: n = 82,341).

S7 Appendix. Distribution of ED resource consumption according to type of set.

S8 Appendix. Model fitting, predictive accuracy, and explained variance of the four different models (Model 1: Final model, Model 2: Triage instead of vital parameters, Model 3: Interaction term between trauma complaint and resuscitation room use, Model 4: Exclusion of revisits.

Acknowledgments

References