Two peer-reviewed evaluations show what AI has and has not done in cancer care outside image screening. At the Princess Margaret cancer hospital in Toronto, a machine learning model drafted radiation plans for prostate cancer, and treating physicians judged 89 percent of them clinically acceptable and picked 72 percent over human-made plans in blinded comparisons. A handheld skin-lesion device authorized by the U.S. Food and Drug Administration (FDA) in January 2024 reached 95.5 percent sensitivity in primary care, but only about 21 percent specificity.
Key takeaways
- The Toronto radiotherapy study cut median planning time by 60.1 percent, from 118 hours to 47 hours, in a blinded head-to-head design.
- The share of machine-made plans chosen for treatment fell from 83 percent in simulation to 61 percent with real patients.
- The skin-lesion device caught more cancers than primary care physicians alone (95.5 versus 83 percent) while flagging many benign lesions.
- Neither tool replaces a clinician: a physician chooses the plan, and a specialist or biopsy settles a flagged lesion.
Radiotherapy planning at Princess Margaret
Before radiation begins, a team must outline the tumor and nearby organs on scans and then design beam arrangements that deliver dose to the first while sparing the second. For localized prostate cancer, a group at the Princess Margaret, part of University Health Network (UHN) in Toronto, trained a random forest algorithm on earlier expert plans. A UHN research news item from November 1, 2019 describes the setup: the system compares a new patient's images with a database of plans made by center experts, drafts a personalized plan, and a physician reviews both the machine plan and the traditional one before choosing.
The results appeared in Nature Medicine on June 3, 2021. The study had two phases of 50 cases each: a retrospective simulation with retesting, and a prospective deployment in which the selected plan was delivered to the patient. Physicians judged plans blindly against standard criteria. Overall, 89 percent of machine plans were clinically acceptable, and 72 percent were chosen over the human-made plan. The median time for the whole planning process dropped by 60.1 percent, from 118 hours to 47 hours.
What the simulation missed
The most useful finding is the gap between the two phases. Acceptability stayed similar, at 92 percent in simulation and 86 percent in deployment. Selection did not: 83 percent of machine plans were chosen in simulation, against 61 percent once the choice affected real patients. The authors concluded that retrospective or simulated evaluation, even with expert blinded review, may not reflect how readily an algorithm is accepted when patient care is at stake. The paper covers one center, one cancer type, and one algorithm, so the numbers do not transfer automatically to other tumors, hospitals, or planning software.
A skin-lesion reader in primary care
The second case is a regulatory decision. The FDA authorized the DermaSensor device under its De Novo pathway in January 2024 for physicians who are not dermatologists, to evaluate lesions that raise concern for melanoma, basal cell carcinoma, or squamous cell carcinoma in patients aged 40 or older. A 2024 review in npj Digital Medicine summarizes the main DERM-SUCCESS study: 22 primary care centers, 1,005 patients, and 1,579 lesions. Device sensitivity was 95.5 percent, compared with 83 percent for primary care physicians and 90 percent for dermatologists. The negative predictive value was 96.6 percent.
The cost of that sensitivity was specificity, reported as 20.7 percent in the text of the review (its Table 2 lists 21.0 percent). Put plainly, most lesions flagged as suspicious were not cancer. In a separate utility study of 108 primary care physicians, the specificity of referral decisions fell from 44.2 percent to 32.4 percent with the device, even as sensitivity improved. The review's authors also noted that 97.1 percent of trial patients were White and 13 percent had darker skin types (Fitzpatrick V and VI), and they called for postmarket monitoring.
The two cases side by side
| Question | Radiotherapy planning (Toronto) | Skin-lesion device (primary care) |
|---|---|---|
| What the software does | Drafts a radiation plan for a physician to review | Scores a skin lesion for a physician who is not a dermatologist |
| Headline result | 89 percent acceptable, 72 percent chosen, 60.1 percent less planning time | 95.5 percent sensitivity, 96.6 percent negative predictive value |
| Main caution | Selection fell from 83 to 61 percent in real use | Specificity near 21 percent, so many benign lesions flagged |
| Population limits | One center, prostate cancer only | Few patients with darker skin or from rural practices |
| Who decides | Treating physician picks the plan | Physician decides on referral; biopsy confirms |
Questions to ask about any cancer AI claim
- Which task is measured: detection, planning, prediction, or paperwork?
- Was the test retrospective or prospective, and did a clinician act on the output?
- Are both sensitivity and specificity reported, or only one?
- Which patient groups are in the data, and how many?
- Was the result confirmed at a second site or in a randomized design?
- Does a regulator authorization describe the exact intended use?
This article gives general information drawn from published studies. It is not medical advice, and it does not suggest any test, device, or treatment. Questions about screening, a skin change, or cancer treatment belong with a physician or the care team involved.
FAQ
Does AI choose a patient's cancer treatment?
In the Toronto study it did not. The software drafted a plan, and treating physicians chose between it and a human-made plan after blinded review.
Is a high sensitivity score enough to trust a cancer tool?
No. A test can catch nearly every cancer and still flag many healthy patients. The skin-lesion trial paired 95.5 percent sensitivity with about 21 percent specificity.
Why did the machine plans lose ground with real patients?
The study did not isolate a single cause. The authors state only that simulated evaluation may not predict acceptance when patient care is at stake.
Can these results be applied to other cancers?
Not directly. The radiotherapy model was built and tested for prostate cancer at one center, and the skin device was tested on specific lesion types in patients aged 40 or older.
Related reading
- AI and public health
- risks that AI systems carry
- how limited memory AI works
- the building blocks of an AI system
More in AI in health.
Last verified: October 2026
Sources
- Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer, McIntosh C et al., Nature Medicine, 2021
- AI Treatment Plans Used in Patients, University Health Network research news, November 1, 2019
- Learnings from the first AI-enabled skin cancer device for primary care authorized by FDA, Venkatesh KP, Kadakia KT, Gilbert S, npj Digital Medicine, 2024