Keywords: TBI, CENTER-TBI, quality indicators
Background
Are quality indicators for TBI patients useful? Traumatic brain injury (TBI) represents one of the most important pathologies these current days, with morbidity and mortality on the rise all over the world.
In Europe, the incidence of TBI seems to be almost half of that in the United States (US) (280,5 per 100,000 vs. 506,4 per 100,000). However, the most important categories of TBI are similar on both continents, respectively traffic accidents in young people and accidental falls in the elderly [1,2].
To understand more about TBI, check out:
TBIs can be classified based on (1) clinical severity using the Glasgow Coma Scale (GCS), (2) the mechanism of injury (motor vehicle accident, assault, fall, etc.), and (3) the characterization of structural damage [3].
Unfortunately, the available evidence on patients with TBI is limited to direct critical care practice. Recent randomized controlled trials showed a limited potential to add evidence that can be easily translated to clinical practice, and this is the reason why new approaches, such as quality of monitoring, are being explored to improve clinical care. Quality of care registration in patients that suffered a TBI could become, in the near future, part of trauma registries or an emerging international intensive care unit (ICU) [4].
In recent years, international registries have tried to contribute to improving patients’ outcomes by identifying areas that need quality enhancement and by informing health policies, thus increasing accountability and transparency. The last action was successfully implemented in other medical fields, like cystic fibrosis, cancer, and acute coronary syndrome. Benchmarking TBI management among different ICUs can only be valid when there are common standard quality indicators and when a case-mix correction is applied. Quality indicators can be subdivided into the indicators mentioned in Figure 1.
Recently, a Delphi study – a well-established approach that answers a research question by identifying a consensus view among subject experts – was performed to reach a consensus on a set of quality indicators for TBI patients, as there were no available quality indicators beforehand [4].
The current study, by Huijben et al., published in 2020, aims to validate the consensus-based quality indicators set. Researchers hetero-analyzed patients that were enrolled in a large dataset of TBI patients from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) study. The data collected included patient outcomes and a comprehensive description of ICU facilities from 54 centers, providing a possibility to examine the usefulness of the newly developed indicator set. In the end, based on the validation result, the quality indicator set could be reduced to those that might present the greatest potential for implementation [4].
Methods
The quality indicator set applied in this validation study was previously developed through a Delphi study on the CENTER-TBI study. The set consisted of 42 indicators pertaining to adult TBI patients in the ICU:
- 17 structure indicators
- 16 process indicators
- 9 outcome indicators
The CENTER-TBI study, a multicenter observational cohort study in Europe, recruited 4509 TBI patients from 2014 until 2018, based on (1) clinical diagnosis of TBI, (2) presentation within 24 h from injury, (3) indication for CT scanning, excluding patients with a pre-existing severe neurological disorder that could interfere with outcome assessments.
Only ICU patients were selected for the current study of Huijben et al. due to the fact that the consensus-based indicators were developed specifically for this ward. As a result, the inclusion criteria were:
- patients admitted to the ICU
- age – 18 or older
ICU care processes obtained daily for the patients admitted were vitals, therapy intensity levels, and treatments. Outcomes were assessed during admission to the ICU and at 3, 6, 12, and 24 months. Moreover, questionnaires on processes of care and structures were completed by the participating centers [4].
Researchers determined whether they could calculate the indicators by using the CENTER-TBI database or if data collection fitted routine practice. The indicators were determined as follows:
a) Structure indicators were calculated based on the Provider Profiling questionnaires and were presented as the number of centers that indicated the presence or absence of the structure.
b) Process indicators were described as the number of patients that adhered to the indicator divided by the number of patients to which this indicator could be applied per center. This denominator could be based on a subset of patients.
c) Outcome indicators were calculated by dividing the event rate of the indicator per center by the total number of patients that could score on the indicator.
The usefulness of the quality indicators was based on the criteria of:
- discriminability
- feasibility
- statistical uncertainty
Due to the fact that no previous thresholds were set on these criteria, the research team set a priori thresholds based on consensus.
Feasibility was quantified by the completeness of the variables required to calculate the indicators. For this, an arbitrary threshold of >70% completeness of data was set.
Discriminability (between centers) and differences in adherence to quality indicators were determined to evaluate their potential for quality improvement and benchmarking. An arbitrary threshold was set for moderate discriminability at 80-90% adherence to process and structure indicators, while the limit for poor discriminability was set at 90-100%. As a result, such high levels of adherence decreased the discrimination between centers. The median odds ratio (MOR) was used to quantify the between-center variation of the outcome and process indicator scores. The higher the MOR, the larger the between-center variation.
Statistical uncertainty was determined by random variation as a result of a low number of events, a characteristic that applies only to outcome indicators. The threshold for high statistical uncertainty was set at <10 events.
Between-center variation of outcome and process indicator scores was calculated using a random-effect logistic regression analysis. A random-effects model was used to account for the fact that the indicator score in centers that had a small number of patients can vary between extreme values due to random variation. Only centers with more than 10 ICU admissions were included. The International Mission for Prognosis and analysis of Clinical Trials in TBI (IMPACT) prognostic model was used to correct for case-mix, with the following variables showcased in Figure 2.
Results
From the CENTER-TBI database, a total of 26 of the 42 indicators of the Delphi set could be extracted. These 26 were divided into (Figure 3):
A total number of 2006 adult patients were included from 54 centers in 18 different countries. The median number of patients admitted to the ICU per center was 23. Most of these were academic and trauma level I centers and were primarily located in Northern and Western Europe. The majority of patients included were male and older than 65 years. According to the baseline Glasgow Coma Scale, 48% had a severe TBI, 16% had a moderate TBI, and 48% had a mild TBI.
The majority of patients suffered from polytrauma, and the most frequent TBI causes were road traffic accidents and incidental falls.
In regard to STRUCTURE INDICATORS, suboptimal adherence rates were found for most indicators, including here:
- availability 24h a day in an operating room
- the presence of neuro-ICU
- the presence of a step-down unit.
High adherence was found for:
- “protocol for glucose management”
- “protocol including specific guidelines”
- “the existence of a protocol including specific guidelines”
- “the neurosurgeon availability within 30 minutes after the call”
- “availability of a CT scan and radiologist review”.
For PROCESS INDICATORS, a high adherence was found for “enteral nutrition within 72 hours”. However, most presented suboptimal adherence, including:
- basal caloric intake within 5-7 days
- ICP monitoring in the severe TBI group
- patients that received deep venous thrombosis (DVT) prophylaxis with low molecular weight heparins
OUTCOME INDICATORS revealed a median mortality of 12% in centers, ventilator-acquired pneumonia (VAP) incidence of 14%, and a hyperglycemia incidence of 35%.
The feasibility of structure and outcome indicators was generally high, except for one process indicator (DVT prophylaxis within 24 h). The only low feasibility in outcome indicators was found in SF-36 MCS and PCS scores collected after 6 months, possibly due to loss to follow-up.
The between-center variation (discriminability) was low for 3 structure indicators (‘existence of a protocol’, ‘availability of a neurosurgeon 24/7 within 30 minutes after call’, and ‘24/7 availability of a CT scan and radiologist review’), high for process indicators in all but one indicator (“surgery in the first 4 hours after TBI in patients with SDH and EDH”), and similarly high for outcome indicators, except for 1 (6-month Glasgow Outcome Score).
In conclusion, from all indicators, the only ones with low discriminability were 5 structure indicators, 2 process indicators, and 4 outcome indicators.
Median event rates for the outcome indicators, respectively, ICU mortality, VAP, and hyperglycemia, were 4, 3, and 8 per center. Median event rates for decubitus and hypoglycemia were 0. All these event rates show a high statistical uncertainty.
Discussion
This study by Huijben et al. showed that it was feasible to obtain quality indicators from a recently proposed, consensus-based quality indicator set for TBI based on sufficient data completeness. The between-center variation combined with the suboptimal adherence scores suggests that there is potential for quality improvement, especially regarding the process and outcome indicators.
Due to the fact that statistical uncertainty was high for outcome indicators, they seem to be less suitable for quality improvement purposes. Researchers found 9 structure indicators and 5 process indicators but none of the 26 indicators were appropriate outcome indicators. Overall, the quality of care in the ICU can be improved for patients that suffered a TBI, and this study can be useful in evaluating quality indicators in a large database.
The present study seems to be the first quality indicator set developed and validated in adult TBI patients admitted to the ICU. There are several recommendations arising from this research:
- Lowering the initial set of indicators by excluding those with a low percentage of available data in a given dataset. The high resources needed for collecting data on process indicators might explain the low feasibility of some of them, but this could be further improved with automatic data extraction.
- The quality indicators with between-center variation and suboptimal adherence rates can be used for benchmarking and improvement of quality of care.
- The low event rates of outcome indicators may show that these have a low potential for quality improvement due to the high statistical uncertainty, but this could be changed over time.
The suboptimal adherence of European ICUs to most quality indicators might suggest that there is room for improvement. The large between-center variation suggests that some centers outperform others significantly, and a wide sharing of practices might help get more standardized care practices. This high between-center variation was also reported in previous TBI studies across Europe and could be explained by the variation in adherence to guidelines.
Although the present study has several strengths, including testing in a large dataset the potential of consensus-based quality indicators and deriving them from the CENTER-TBI dataset, which included a high number of patients, it also has several limitations. These include the partial capture of staffing and organizational data by CENTER-TBI, basing the structure indicators on questionnaires, the change of different definitions of terms, such as feasibility, in order to be more appropriate for the current study, the statistical uncertainty which was reflected in the number of event rates.
Future implementation of the quality indicators in different European registries would make the monitoring of TBI patients’ data over time and among countries possible.
Conclusions on quality indicators for TBI patients
The current study validated a consensus-based quality indicator set in the CENTER-TBI, a large prospective TBI study. Moreover, the analysis shows good discriminability and feasibility but high statistical uncertainty for outcome indicators. Future research should focus on the implementation and continuous reevaluation of quality indicators.
References
- Corrigan JD, Selassie AW, and Orman JAL, ‘The epidemiology of traumatic brain injury’, J. Head Trauma Rehabil. 2010, vol. 25, no. 2, pp. 72–80, doi: 10.1097/HTR.0b013e3181ccc8b4.
- Majdan M, Plancikova, Brazinova A, Rusnak M et al., ‘Epidemiology of traumatic brain injuries in Europe: a cross-sectional analysis’, Lancet Public Health, vol. 1, no. 2, pp. e76–e83, Dec. 2016, doi: 10.1016/S2468-2667(16)30017-2.
- Dadas A, Washington J, Diaz-Arrastia R, and Janigro D. ‘Biomarkers in traumatic brain injury (TBI): a review’, Neuropsychiatr. Dis. Treat. 2018, doi: 10.2147/NDT.S125620.
- ‘Quality indicators for patients with traumatic brain injury in European intensive care units: a CENTER-TBI study – PubMed’. https://pubmed.ncbi.nlm.nih.gov/32131882/
