In AI research, the problem domain is the real-world setting a model is built for, including how its data were collected and what decisions follow from its output. Ignoring it can produce models that score well and still fail: a 1990s pneumonia study produced a rule that asthma lowers risk of death, a chest X-ray model learned which hospital an image came from, and Google Flu Trends missed high for 100 of 108 weeks. Each was explained by facts about the setting, not by the choice of algorithm.
Key takeaways
- A model learns patterns in the data it is given, including patterns created by hospital routines, search habits, or how labels were assigned.
- In a 2015 paper, Caruana and colleagues described a rule that pneumonia patients with asthma had lower risk of death, because those patients were sent straight to intensive care.
- A 2018 study of 158,323 chest radiographs found that models often performed worse at new hospitals, and a hospital-prevalence-only model reached an AUC of 0.861.
- Google Flu Trends searched 50 million terms to fit 1,152 data points, and its later errors were not random.
- These are published research cases, not medical advice or guidance for any clinical decision.
What "problem domain" means in practice
A problem domain is everything about a task that sits outside the algorithm: who produces the data, what a label really records, what the output will be used for, and what happens when it is wrong. Two teams can train the same network on the same table and get models of very different worth, because only one knows that a column was filled in after the outcome was known, or that a number of zero means "not measured." Domain knowledge is what turns a score into a claim about the world. The three cases below are well documented and come from health and public health, where the mismatch between data and reality is easiest to see.
Case one: the pneumonia model that treated asthma as protective
Rich Caruana and colleagues recount, in a 2015 KDD paper, a study that tried to predict the probability of death for patients with pneumonia so that high-risk patients could be admitted and low-risk patients treated as outpatients. Multitask neural nets were the most accurate models. On one dataset the neural net reached an AUC of 0.86 against 0.77 for logistic regression. Yet a rule-based model trained on one dataset learned that a history of asthma meant lower risk of dying. The paper explains why: asthma patients with pneumonia usually went directly to the intensive care unit, and that aggressive care was effective enough to lower their risk of death in the data. A model that recommended sending such patients home would have removed the very care that made the data look good.
The team decided not to use the neural nets. The reason was not that the asthma problem could not be fixed, but that the nets were not intelligible, so other hidden problems could not be ruled out. Logistic regression was used instead. The lesson is that the pattern was true in the data and wrong as a basis for a decision, and only a clinician's knowledge exposed the difference.
Case two: a chest X-ray model that could tell which hospital it came from
John Zech and colleagues trained convolutional neural networks to detect pneumonia on chest radiographs from three systems: the NIH Clinical Center, Mount Sinai Hospital, and Indiana University, 158,323 images in all. In their PLOS Medicine paper (2018), external performance was significantly lower than internal performance in three of five natural comparisons. Pneumonia prevalence differed widely by site: 34.2 percent at Mount Sinai against 1.2 percent at the NIH and 1.0 percent at Indiana. The networks could identify the hospital system of a test image correctly for 99.95 percent of NIH images and 99.98 percent of Mount Sinai images. A trivial model that used only hospital-level prevalence reached an AUC of 0.861 on the combined Mount Sinai and NIH test set. The authors conclude that models likely used site-specific cues tied to disease prevalence.
For a hospital deciding whether to adopt such a tool, the relevant fact is how the model performs on its own patients, which is a domain question about scanners, workflows, and patient mix.
Case three: a flu tracker that was part winter detector
Google Flu Trends estimated influenza activity from search queries. In a 2014 Science article, David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani report that in February 2013 it predicted more than double the proportion of doctor visits for influenza-like illness reported by the Centers for Disease Control and Prevention (CDC). The initial method looked for the best matches among 50 million search terms to fit 1,152 data points, which made spurious matches likely. Developers removed seasonal terms such as those about high school basketball, and the tool completely missed the nonseasonal 2009 H1N1 pandemic. The authors summarize the original version as part flu detector, part winter detector. They report that the updated model missed high for 100 out of 108 weeks starting with August 2011, that errors were correlated from week to week, and that a model using lagged CDC data did better. They also point to algorithm dynamics, meaning changes to the search service and to user behavior that altered what the queries measured. The article is available in the Harvard DASH repository.
Side-by-side summary
| Case | What the data contained | What domain knowledge revealed |
|---|---|---|
| Pneumonia risk, 1990s data (Caruana et al., 2015) | Asthma patients with lower recorded death rates | Those patients received intensive care first, so the pattern reflected treatment |
| Chest X-rays at three sites (Zech et al., 2018) | Images carrying clues to the hospital system and its prevalence | Performance depended on site, and a prevalence-only model scored 0.861 |
| Google Flu Trends (Lazer et al., 2014) | Search terms that tracked winter as well as flu | Seasonal terms and a changing search service produced systematic over-estimates |
Questions a domain check should answer before training
- How was each label created, and could it record a decision or treatment instead of an underlying fact?
- Does a value such as zero mean "normal," "missing," or "not measured"?
- Will the model be used in the same setting, population, and season as the data came from?
- What trivial baseline, such as site prevalence or last week's value, should the model have to beat?
- Who can read the model well enough to catch a rule that is true in the data and wrong in practice?
FAQ
Is domain knowledge more important than the algorithm?
The cases show that it can decide whether a result is meaningful. Algorithm choice still matters for accuracy, but a strong algorithm trained on misleading data tends to learn the misleading pattern more efficiently.
Do these cases mean AI should not be used in health care?
No. They show why testing at the site of use and review by clinicians are needed. Whether a specific tool is appropriate is a decision for qualified professionals, and nothing here is medical advice.
Are these failures from old technology?
The studies date from 2014 to 2018, and the underlying issue, a gap between training data and the real setting, is not tied to a specific method. Newer tools need the same checks.
What is AUC, as used in these studies?
AUC is the area under a curve that summarizes how well a model ranks positive cases above negative ones. A value of 0.5 is chance and 1.0 is perfect ranking.
Related reading
- Worked examples of problem formulation
- Why research problems are hard to pin down
- Toy problems and their limits
- AI in scientific research
More in AI Basics.
Last verified: October 2026
Sources
- Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission, Caruana R et al., KDD 2015
- Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study, Zech JR et al., PLOS Medicine, 2018
- The Parable of Google Flu: Traps in Big Data Analysis, Lazer D, Kennedy R, King G, Vespignani A, Science 343(6176), March 14, 2014