Predictive analytics uses historical data and statistical or machine learning models to estimate how likely a future event is, such as a patient developing sepsis, a customer leaving, or a machine failing. Its value depends on measured accuracy in the setting where it is used. A widely deployed hospital sepsis score had an area under the curve of 0.63 in a 2021 external test, and an updated version scored between 0.82 and 0.92 across four health systems in a 2026 study.
Key takeaways
- A predictive model outputs a probability or a risk score. A person or a rule then decides what to do at a chosen threshold.
- A 2021 University of Michigan study found that the first version of the Epic Sepsis Model flagged 18 percent of hospitalizations and missed 67 percent of patients who developed sepsis.
- A February 27, 2026 multicenter study of the updated version found better discrimination but wide variation between hospitals, low positive predictive value, and a heavy alert load.
- Both studies point to the same practice: test a model on local data before relying on it, and track alert volume as well as accuracy.
What a prediction score is and is not
A predictive model is trained on past records in which the outcome is already known. It learns patterns that tend to precede that outcome, then scores new cases. The score is an estimate, not a finding. Turning it into a decision requires a threshold: above this score, send an alert, call a customer, or schedule an inspection. Where the threshold sits trades missed cases against false alarms, and the right balance depends on the cost of each kind of error in the specific workplace.
Two common measures describe model quality. The area under the curve (AUC, also written AUROC) summarizes how well the score separates cases from non-cases, where 0.5 is no better than chance and 1.0 is perfect separation. Positive predictive value (PPV) is the share of alerts that turn out to be real. A model can score well on the first measure and still produce mostly false alarms on the second when the event is rare.
A hospital case: the Epic Sepsis Model in 2021
Sepsis is a life-threatening reaction to infection, and early treatment matters, which makes it a popular target for prediction. A team at the University of Michigan tested a proprietary sepsis score built into the electronic health record software of Epic Systems. The study, published in JAMA Internal Medicine in 2021, covered 27,697 adult patients and 38,455 hospitalizations at Michigan Medicine from December 6, 2018 to October 20, 2019. Sepsis occurred in 2,552 hospitalizations, or 7 percent.
At the alert threshold (a score of 6 or higher) the model reached a sensitivity of 33 percent, a specificity of 83 percent, and a PPV of 12 percent. The hospitalization-level AUC was 0.63. The alert fired in 6,971 hospitalizations, or 18 percent, and missed 1,709 patients with sepsis, or 67 percent. The authors concluded that the model had poor discrimination and calibration, and they called on professional organizations to issue formal recommendations for proprietary algorithms of this kind. The study was retrospective and took place at one academic medical center, so it describes that site's data and not every hospital.
Why a low PPV follows from the numbers
The figures can be checked by simple arithmetic. The table below is an illustrative example using the study's reported prevalence, sensitivity, and specificity, applied to 1,000 hospitalizations.
| Step | Calculation | Result |
|---|---|---|
| Patients who develop sepsis | 7 percent of 1,000 | 70 |
| Sepsis cases flagged | 33 percent of 70 | about 23 |
| Patients without sepsis | 1,000 minus 70 | 930 |
| False alerts | 17 percent of 930 | about 158 |
| Total alerts | 23 plus 158 | about 181, or 18 percent of patients |
| Share of alerts that are real | 23 divided by 181 | about 13 percent |
The result matches the reported 18 percent alert rate and sits close to the reported 12 percent PPV. The lesson is general: when an event is uncommon, even a model with decent specificity generates many more false alerts than true ones.
The updated version in 2026
A prospective study in JAMA Network Open, published February 27, 2026, evaluated version 2 of the Epic Sepsis Model. It used data from nine hospitals in four health systems (University of Michigan, Oregon Health and Science University, Emory Healthcare, and MetroHealth), covering 227,091 inpatient encounters, of which 7,401, or 3.3 percent, met sepsis criteria. Encounter-level AUROC ranged from 0.82 at the lowest site to 0.92 at the highest. At 60 percent sensitivity, PPV ranged from 0.13 to 0.26. The authors reported improved discrimination but also high variability between institutions, low PPV, and a high alert burden, and they advised local validation, workflow changes to manage false positives, and alert silencing.
Questions to ask before trusting a prediction tool
- Was the model tested on data from a different site than the one that built it?
- What is the PPV at the threshold actually in use, and how many alerts per day does that mean?
- What happens when the model misses a case, and who notices?
- Is the tool's accuracy tracked after launch, with a plan for retraining or retirement?
- Does a human review the score, and are the reasons for it visible?
The studies discussed here concern a clinical setting, and nothing in this article is medical advice. Readers using similar tools in business, such as demand forecasting or churn scoring, can apply the same questions, with the cost of a missed case and of a false alarm measured in their own terms.
FAQ
What is the difference between predictive analytics and AI?
Predictive analytics is a task: estimating future outcomes from data. AI and machine learning are among the methods used for it, alongside older statistical tools such as regression.
Does a higher AUC mean a model is safe to use?
Not by itself. AUC says how well scores rank cases, but PPV, alert volume, and local performance determine whether the tool helps in practice.
Why did the same product perform differently in two studies?
The 2021 and 2026 studies examined different versions, in different hospitals, over different periods, with different definitions of sepsis and timing. The results cannot be compared as a single before-and-after test.
Where can a person check a vendor's accuracy claims?
Peer-reviewed external validation studies, which the vendor did not run, give the most independent evidence. Both studies cited here are open-access articles.
Related reading
- Decision support systems and AI
- Linear regression as a prediction method
- How AI differs from automation
- Characteristics of intelligent agents
More in AI Tools and Industry.
Last verified: October 2026
Sources
- External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients, Wong et al., JAMA Internal Medicine, 2021
- Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model, Wong et al., JAMA Network Open, February 27, 2026