Designing for Trust in AI
Challenge
Young AI startups must display positive retention metrics to investors to indicate product market fit. The challenge was to fix a leaky user funnel. I had to diagnose and prototype a solution with users within 48 hours.
Solution
The diagnosis involved mapping both the happy path and worst-case scenarios at each step of the user journey. This approach helped anticipate AI edge cases. We identified the most critical edge case and created prototypes to nudge users back to the happy path.
Process
To manage the aggressive deadline, I decided to prompt an AI team to assist with initial exploration. This involved feeding the behavioral health professional user persona to AI, effectively creating a live persona to ask questions and explore problems and ideas.
AI was used to get to a closer hypothesis of the user problems and build a prototype to validate that hypothesis through real user testing.
What is the clinician's ultimate professional goal?
The goal isn't just to finish notes faster; it's to deliver better patient care. A good note is a critical tool for tracking progress. This meant a solution that compromised note quality for speed would fail.
What is the user's primary 'job to be done'?
The main job was to "offload cognitive burden." This revealed the user's need wasn't just about saving time after the session, but about freeing up mental energy to be more present during the session.
How would they collaborate with a human assistant?
The ideal interaction was "trust, but verify." A clinician wouldn't re-read everything; they would perform targeted spot-checks on critical information. This insight pointed directly to on-demand traceability as the key to building user trust.
I mapped out the "ideal" journey JotPsych intended for its users. At each stage, I asked: "What is the worst possible thing that could happen here?" This exercise revealed three distinct failure scenarios that would cause a user to churn.
How might we redesign the note review and editing experience to create a transparent collaboration between clinicians and their digital (AI) assistant, rather than a frustrating task of correcting an opaque machine?
I did old-school white boarding to mind map and explore ideas. I decided to focus on building AI traceability and helping build the right expectations for new users. This would be critical to retention, allowing users to correctly calibrate their trust.
Current state:
Opaque and Static.
Verifying information is high friction.
Ambiguous Workflow.
High Cognitive Load.
Redesign for Traceability: Quickly cross reference AI insights from original transcript
Redesign for calibrating Trust and Expectations: Prevent errors due to oversight and manage expectations
Key Learnings 🙇♂️
Designing for Trust Over Magic: For AI in high-stakes fields, demonstrating transparency and giving users ultimate control is more critical than appearing fully autonomous.
The "Human Assistant" Metaphor: Framing the AI as a junior assistant provided a clear and effective model for designing intuitive, collaborative interactions.
Next Steps ⏭️
Validate Through User Testing: Conduct moderated usability tests with target clinicians using the prototype to gather qualitative and quantitative data on its effectiveness.
Iterate Based on Feedback: Analyse the findings from the user tests to make any necessary refinements to the design.
Expand the Feature Set: Begin ideation on the next phase of features, such as the "Teach AI" commands, to further enhance the collaborative experience.