Designing Trust First: A Framework for AI in a Clinical Product

Leadership wanted AI in the product — that part wasn't mine to decide. But the instruction was a single sentence with no spec, and the easy failure mode was obvious: start bolting on AI features to satisfy the mandate, whether or not they helped anyone. So my first job wasn't to design AI. It was to find where AI genuinely earned its place, so I wasn't adding features for their own sake.

I grounded that in real friction rather than assumption. I asked the team and clinicians what actually took the most time in their day, then tested each of those pain points against a simple question: could AI meaningfully reduce this, or would good design alone already solve it? That distinction mattered — I didn't want AI rescuing flows I could fix with better UI.

Creating an appointment was the clearest example of the line. My calendar redesign had already made it faster through better hierarchy and fewer clicks — so AI wasn't there to fix a broken flow. But for a power user, natural language broke a ceiling that interface design couldn't: a single typed sentence — "add a recurring monthly appointment for John Doe on Oct 17, charge his card on file" — collapsed a multi-field, multi-step task into one action. That's where AI clearly belonged: not patching the floor, but clearing a ceiling good UI had already reached.

Any AI in a clinical product touches Protected Health Information, which made the data model a design problem, not just an engineering one. The framework had to define precisely what the AI ingested, what it was allowed to act on, and what was discarded from memory immediately — so that a clinician could never accidentally apply a diagnosis, a note, or a billing code to the wrong client.

I treated these safeguards not as a feature layered on at the end, but as the precondition for the product existing at all. Every interaction in the framework was designed assuming the data boundaries came first.

Verify in under 10 seconds. The north star: a clinician should be able to understand what an agent did and confirm any required action in ten seconds or less, without leaving confused. The review experience itself had to be fast, clear, and unambiguous every time.

‍User control and freedom. AI must never silently finalize work. Every action is clearly labeled, reversible, and explicitly approved — the clinician stays the final decision-maker at all times.

‍Visibility of system status. Therapists must always understand what an agent did, what it used, and what state it's in. The proposal defined four shared status states used across every agent: Ready for Review, Blocked, Needs Attention, and Approved.

Rather than a generic chat box, the panel reads the page the clinician is on and pre-loads the questions most relevant to that context. On the calendar, it surfaces scheduling questions; the preloaded prompts were drawn from the most frequently asked support queries, so the panel answered real questions before anyone had to contact support.

The design intent was to make a single, predictable entry point for everything AI — so customers never had to wonder where AI lived, when it was running, or how to reach it.

One click from the panel opened the agent inbox: a single place showing exactly what every agent had done, sorted by urgency rather than by time or type. Needs Attention surfaced first in red, Needs Review in amber, and completed work in green — the same four status states applied consistently across intake, scheduling, assessment, treatment plans, billing, and analytics.

The consistency was the point. A clinician learned the status language once and it held everywhere, which is what kept the ten-second review standard achievable as more agent types were added.

My version of the panel was live for about a month before the product direction changed. I won't overstate the shipped lifespan — but the part that mattered outlived the specific release. The interaction model, the panel architecture, the status model, the agent icons, and the three principles all carried forward as the base pattern for Therasoft AI.

What I'd carry into the next version: more direct validation with the most AI-skeptical customers earlier in the process. The reversal — skeptics choosing to keep AI — happened after launch; getting those users into the loop before launch would have made the trust case even stronger, and faster.