Background: Machine learning could improve the timely identification of trauma patients in need of hemorrhage control resuscitation (HCR), but the real-life performance remains unknown. The ShockMatrix study aimed to compare the predictive performance of a machine learning algorithm with that of clinicians in identifying the need for HCR.
Methods: Prospective, observational study in eight level-1 trauma centers. Upon receiving a prealert call, trauma clinicians in the resuscitation room entered nine predictor variables into a dedicated smartphone app and provided a subjective prediction of the need for HCR. These predictors matched those used in the machine learning model. The primary outcome, need for HCR, was defined as: transfusion in the resuscitation room, transfusion of more than four red blood cell units in 6 h of admission, any hemorrhage control procedure within 6 h, or death from hemorrhage within 24 h. The human and machine learning performances were assessed by sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and net clinical benefit. Human and machine learning agreement was assessed with Cohen's kappa coefficient.
Findings: Between August 2022 and June 2024, out of 5550 potential eligible patients, 1292 were ultimately included in the analyses. The need for HCR occurred in 170/1292 patients (13%). The results showed a positive likelihood ratio of 3.74 (95% confidence interval [CI]: 3.20-4.36) and a negative likelihood ratio of 0.36 (95% CI: 0.29-0.46) for the human prediction and a positive likelihood ratio of 4.01 (95% CI: 3.43-4.70) and negative likelihood ratio of 0.35 (95% CI: 0.38-0.44) for the machine learning prediction. The combined use of human and machine learning prediction yielded a sensitivity of 83% (95% CI: 77-88%) and a specificity of 73% (95% CI: 70-75%). The Cohen's kappa coefficient showed an agreement of 0.51 (95% CI: 0.48-0.55).
Interpretation: The prospective ShockMatrix temporal validation study suggests a comparable human and machine learning performance to predict the need for HCR using real-life and real-time information with a moderate level of agreement between the two. Machine learning enhanced decision awareness could potentially improve the detection of patients in need of HCR if used by clinicians.
Back to the list of publications.Background: Reliable information is an essential component for responding to the COVID-19 epidemic, especially regarding the availability of critical care beds (CCBs). We propose three contributions: a) ICUBAM (ICU Bed Availability Monitor), a tool which both collects and visualizes information on CCB availability entered directly by intensivists. b) An analysis of CCB availability and ICU admissions and outcomes using collected by ICUBAM during a 6-week period in the hard-hit Grand Est region of France, and c) Explanatory and predictive models adapted to CCB availability prediction, and fitted to availability information collected by ICUBAM.
Methods: We interact directly with intensivists twice a day, by sending a SMS with a web link to the ICUBAM form where they enter 8 numbers: number of free and occupied CCBs (ventilator-equipped) for both COVID-19 positive and COVID-19- negative patients, the number of COVID-19 related ICU deaths and discharges, the number of ICU refusals, and the number of patients transferred to another region due to bed shortages. The collected data are described using univariate and multivariate methods such as correspondence analysis and then modeled at different scales: a medium and long term prediction using SEIR models, and a short term statistical model to predict the number of CCBs.
Results: ICUBAM was brought online March 25, and is currently being used in the Grand-Est region by 109 intensivists representing 40 ICUs (95% of ICUs). ICUBAM allows for the calculation of CCB availability, admission and discharge statistics. Our analysis of data describes the evolution and extent of the COVID-19 health crisis in the Grand-Est region: on April 6th, at maximum bed capacity, 1056 ventilator-equipped CCBs were present, representing 211% of the nominal regional capacity of 501 beds. From March 19th to March 31st, average daily COVID-19 ICU inflow was 68 patients/day, and 314 critical care patients were transferred out of the Grand-Est region. With French lockdown starting on March 17th, a decrease of the daily inflow was found starting on April 1st: 23 patients/day during the first fortnight of April, and 7 patients/day during the last fortnight. However, treatment time for COVID-19 occupied CCBs is long: 15 days after the peak on March 31st, only 20% of ICU beds have been freed (50% after 1 month). Region-wide COVID-19 related in-ICU mortality is evaluated at 31%. Models trained from ICUBAM data are able to describe and predict the evolution of bed usage for the Grand-Estregion.
Conclusion: We observe strong uptake of the ICUBAM tool, amongst both physicians and local healthcare stakeholders (health agencies, first responders etc.). We are able to leverage data collected with ICUBAM to better understand the evolution of the COVID-19 epidemic in the Grand Est region. We also present how data ingested by ICUBAM can be used to anticipate CCB shortages and predict future admissions. Most importantly, we demonstrate the importance of having a cross-functional team involving physicians, statisticians and computer scientists working both with first-line medical responders and local health agencies. This allowed us to quickly implement effective tools to assist in critical decision-making processes.
Back to the list of publications.Background: The CRASH-3 trial provides a high level of evidence on the question whether to administer Tranexamic Acid (TXA) for Traumatic brain injury (TBI). For numerous other research questions, the available evidence will not correspond to such a level of evidence and will rely on observational evidence only. The development of methodological alternatives to analyze observational data is necessary. The Crash-3trial provided the opportunity to explore the effect of TXA on TBI mortality with two distinct causal inference methods using incomplete observational data.
Methods: Two causal inference techniques, inverse propensity weighting (IPW) and doubly robust method (DR), associated with machine learning method techniques to handle missing data, explored the effect of TXA administration on 30-day head injury related death expressed in registry data. The effect was expressed as Average Treatment Effect (ATE). TBI was defined as a head Abbreviated Injury Score >2. The hypothesis expected the results to concur with the results obtained with the CRASH-3 benchmark trial.
Results: Between September 2010 and February 2019, from a total of 20037 registry cases 8269 corresponded to the definition of TBI. A total of 683 received TXA and 7565 did not. The observed head-injury related 30-day hospital mortality rate in the group TXA was 30% (205/683) compared to 15% in the group no-TXA (1102/7565, p<0.001). Causal inference with the IPW approach indicates an ATE with a higher mortality after TXA independently of the approach applied to manage missing data (ATE 0.10 (95% IC [0.06, 0.14]) or 0.09 (95% IC [0.03, 0.15])). ATE obtained with DR did not show any effect on mortality independently of the approach applied to missing data (ATE -0.01 (95% IC [-0.05, 0.03]) or -0.01 (95% IC [-0.07, 0.05])). No effect was observed in predefined subgroups.
Conclusions: This study demonstrated the feasibility to apply causal inference techniques in incomplete observational data. DR based on a stronger theoretical background compared to IPW, did not show a significant association of TXA administration with in-hospital mortality. This result provides a strong incentive to explore augmented causal inference techniques on incomplete observational data coupled with techniques to handle missing values.
Back to the list of publications.