AI Will Not Make Healthcare Safer by Default

AI will not make healthcare safer by default.

That is the blunt conclusion of a new research agenda from 18 experts across quality, human factors, radiology, nursing, informatics and machine learning (Patterson et al., Diagnosis, August 2026).

The line that stuck with me: we are dropping AI into diagnostic workflows that are already fragmented and coordination-heavy. Without deliberate design, AI amplifies existing safety risk rather than removing it.

Six Knowledge Gaps

The paper names six gaps the field needs to close over the next five to seven years:

• Safe by design - how to develop, validate and maintain AI when the technology updates in days, not years
• Redesign, don't substitute - augmenting a clinician's task reshapes the workflow and the patient relationship around it
• Human-machine teaming - keep the expert in command, not managing a black box they cannot question
• Foundation models - powerful, adaptive, largely unproven; when should they be let near patients?
• The right dose of regulation - too little erodes trust, too much stalls adoption
• Deployment in imaging - trust calibration, deskilling, liability and equity, not just accuracy

Where This Lands in the UK

The paper is US-framed - FDA, state versus federal oversight. In the UK we already have a safety scaffold for exactly this in DCB0129 and DCB0160 - but every human factors question above sits upstream of any hazard log. Naming the gap is the easy part. Closing it is the work.

What would you add to the list?

TrustPoint Digital Health provides independent clinical safety governance for AI and digital health products under DCB0129 and DCB0160. Get in touch to discuss your safety case.