Today is World Mental Health Day. This year's theme is "Lived experiences heard: real voices, real change", a call for people with lived experience to shape the services and decisions that affect them, not only to share their stories. It sent me back to an app I helped build with young people fourteen years ago, and to the question we never asked them.
In 2012 I was a mental health nurse managing an early intervention in psychosis service. Young people kept telling us two things. Getting into the service had been confusing, and they wanted their phones to be part of their care.
So we built an app with them. The project group included young people with lived experience, a consultant psychiatrist, a community psychiatric nurse, a web developer, the chief pharmacist, a communications officer and me. In September 2012, twenty young people test-drove the prototype.
At the time, I wrote that building it was the easy part and adoption would be the hardest. Having since reviewed clinical safety cases for more than 20 AI and digital health products, I think I was only half right.
The app did five things young people asked for:
I believed, sincerely, that more data and earlier signals would make care safer. We did ask ourselves about unintended consequences. It was one of four questions the project group returned to throughout.
But a question asked in a meeting is not a hazard identified, assessed and owned. The clinical risk management standards now known as DCB0129 and DCB0160 had been adopted by the NHS in 2009. None of us on the project knew they existed.
The work did not stop in 2012. A later version, My Journey 3, was tested in the ARIES feasibility randomised controlled trial across six NHS early intervention services, with a nested qualitative study. I was a co-author on both.
The research was careful. Forty people were followed for 12 months. Participants were told the app was not for urgent help in a crisis and was no substitute for human support. No serious adverse events and no privacy breaches were reported.
But a feasibility trial asks whether a full trial is possible and whether people will use the tool. A hazard log asks how the tool could hurt someone in routine use, and who owns each risk. The trial's own findings show why the second question matters:
None of these is a research failure. Each is a hazard waiting for routine deployment, where there is no researcher on call and the users are not a stable trial sample.
Reading my own app again as a Clinical Safety Officer, I find seven hazards worth logging. None of them needed a coding error to cause harm.
| # | Hazard | Cause | Possible harm | Control I would now require | Owner |
|---|---|---|---|---|---|
| 1 | Mood log mistaken for monitoring | Young person assumes clinicians see entries in real time; no agreed workflow for reviewing them | Deterioration or crisis recorded but not acted on | Clear in-app statement that entries are not monitored; crisis contacts on every screen; review point written into the care plan | Both |
| 2 | Missed or wrong medication reminder | Phone off or notifications silenced; schedule not updated after a prescription change | Missed or doubled doses of antipsychotic medication; relapse | Reminders positioned as an aid, never the record; schedule checked at every medication review; failure modes tested | Both |
| 3 | Appointment reminder out of step with the clinical system | Reminders entered by hand, not synced; appointment moved or cancelled | Missed contact read as disengagement; follow-up delayed | One source of truth, or a clear "check with your team" prompt; non-attendance follow-up unchanged | Service |
| 4 | Progress shared with the wrong person | Consent given once; relationships change or become unsafe | Disclosure of sensitive information; harm; loss of trust | Sharing reviewed at each care review; one-tap revoke; consent recorded | Both |
| 5 | Sensitive content seen on a shared or lost phone | No app lock; notifications show content on the lock screen | Unwanted disclosure; stigma | App lock; neutral notification wording | Developer |
| 6 | Help-seeking information goes out of date | Numbers and services change; no named content owner | A young person in crisis calls a dead number | Named content owner; scheduled review; version control | Service |
| 7 | Self-reported data used without context | Mood entries partial or retrospective | False reassurance or unnecessary escalation | Guidance that app data supplements, never replaces, assessment | Service |
I have not scored these. A real assessment would rate each one for severity and likelihood with the clinical team, then record who accepted the residual risk.
Six of the seven hazards sit wholly or partly with the service, not the software. That is what 2012 me missed.
Co-production answered one question, not two. Young people told us what they needed. Nobody asked them, or us, how the tool could fail them. Both questions deserve the same people in the room.
The quietest hazard is the most dangerous. Hazard 1 needs no fault at all. A young person logs a bad week and believes someone has seen it. When data is collected, people reasonably assume someone is responsible for it.
Building it yourself means owning both sides. When an NHS team builds its own tool, it is both manufacturer and deploying organisation. Both standards apply to the same people, and nobody else is coming to write the safety case.
I said adoption was the hardest part. It is not. The hardest part is deciding, before go-live, who owns each way the tool can go wrong.
These hazards are not relics of 2012. Swap the mood tracker for a mental health chatbot, or the reminder for an ambient scribe, and every row still applies.
The riskiest moment is still when a person assumes someone, or something, is paying attention. With AI, the tool can also sound confident while being wrong. We made this argument in Who Owns the Risk When an AI Scribe Writes Something Nobody Said? and AI Will Not Make Healthcare Safer by Default.
Two lessons carry straight across. First, bring people with lived experience into hazard identification, not only design. They see failure modes clinicians miss. Second, treat the hazard log as a living document. An AI tool's behaviour can shift after go-live in ways a 2012 reminder never could.
At TrustPoint, every engagement starts with the question I never wrote down in 2012: how could this hurt someone, and who will notice?
We help NHS organisations and suppliers build DCB0129 and DCB0160 safety cases. We run hazard identification workshops that bring clinicians and people with lived experience into the same room. And we assure AI tools before and after go-live.
If you are deploying a digital or AI tool in mental health and want a second pair of eyes on your hazard log, book a call.
TrustPoint provides independent clinical safety governance for digital health and clinical AI - DCB0129/DCB0160 safety cases, readiness reviews, and post-market surveillance.