Hi, I’m Shannon
Senior Product Designer at Muddy Gecko
Designing Real-Time Transcription and AI Notes for Fireflies Mobile App
I designed a new mobile workflow that lets users record conversations, view live transcription and AI-generated notes in real time, and interact with Fireflies’ AI assistant (AskFred)
Overview
Fireflies is an AI assistant that captures, transcribes, and summarizes meetings. While the product excels in virtual meeting environments, the mobile experience for in-person conversations needed to be reimagined. I designed a new mobile workflow that enables users to easily record conversations, view live transcription and AI-generated notes in real time, and interact with Fireflies’ AI assistant (AskFred) during the conversation.
This case study walks through how I approached the redesign across research, ideation, UI exploration, prototyping, and final design.
Problem
Doctors conducting patient interviews need a fast, reliable, and distraction-free way to record, transcribe, and extract insights from in-person conversations. This forces clinicians to context-switch or recall details later, leading to inefficiency and cognitive overload.
How might we enable doctors to capture, understand, and act on key moments during patient conversations in real time, without disrupting care?
Ideal Customer Profile (ICP)
Although the solution applies to anyone recording in-person conversations, this design specifically focuses on healthcare professionals, such as:
- Primary care physicians conducting patient interviews
- Therapists and clinicians documenting sessions
- Specialists gathering detailed verbal histories
These users are:
- Highly time-constrained
- Easily overwhelmed by UI clutter
- Focused on patient interaction, not managing technology
- In need of accurate, real-time notes for decision-making
The experience had to be intuitive, distraction-free, and designed for rapid mental processing.
Research & Design Exploration
Competitive Analysis
I conducted competitor research of leading transcription apps to understand:
- How competitors display live transcripts
- How they handle recording states and controls
- How they surface AI summaries or chat
- UI patterns for low-distraction mobile workflows
I also used tools like Mobbin, where I reviewed iOS patterns for drawers, live feeds, chat interactions, and recording UX to understand what users already expect.
How Might We Brainstorm
I ran a ‘How Might We’ brainstorm focused on the core user challenges. I mapped a series of prompts such as “How might we allow users to quickly change a recording title without disrupting the session?”, “How might we help users access live notes with minimal distraction?”, and “How might we surface AskFred in a way that feels available but not intrusive?” This exercise helped me cast a wide net of possibilities, challenge initial assumptions, and identify design opportunities that balanced simplicity with real-time functionality.
Brainstorming with AI
I also brought AI into my brainstorming process. I had a quick back-and-forth conversation to explore different ways AskFred could show up during a live recording. I talked through options like keeping AskFred inside the transcription drawer or giving it its own tab, and we walked through the pros and cons of each. The AI helped me think through things like how much navigation each option would require, how distracting it might feel in the moment, and what would be easiest for someone multitasking during an in-person conversation. It was a fast way to pressure-test ideas and move closer to a solution that feels natural for users.
Defining the Solution
After synthesizing research and narrowing exploration, I designed a streamlined mobile experience centered around clarity and ease.
Key Design Decisions
- A persistent bottom recording bar with timer + quick actions ensures users always know recording status.
- A maximize bottom drawer reveals live transcription and AI notes without forcing a context switch.
- AskFred is accessible via a secondary tab, allowing quick question-asking during recording with minimal UI intrusion.
- I designed a persistent bottom recording bar with clear minimize and maximize controls. Tapping the bar expands into the full live audio notes screen, and users can minimize it again just as easily. This keeps navigation simple and predictable, especially for time-pressed users who need to jump between views without friction.
- I introduced AI-suggested titles generated from the live transcript. This helps users quickly confirm or update a recording name without interrupting the conversation or losing focus.
Let’s connect
shannon.trumbull@gmail.com
Hi, I’m Shannon
Senior Product Designer at Muddy Gecko
Designing Real-Time Transcription and AI Notes for Fireflies Mobile App
I designed a new mobile workflow that lets users record conversations, view live transcription and AI-generated notes in real time, and interact with Fireflies’ AI assistant (AskFred)
Overview
Fireflies is an AI assistant that captures, transcribes, and summarizes meetings. While the product excels in virtual meeting environments, the mobile experience for in-person conversations needed to be reimagined. I designed a new mobile workflow that enables users to easily record conversations, view live transcription and AI-generated notes in real time, and interact with Fireflies’ AI assistant (AskFred) during the conversation.
This case study walks through how I approached the redesign across research, ideation, UI exploration, prototyping, and final design.
Problem
Doctors conducting patient interviews need a fast, reliable, and distraction-free way to record, transcribe, and extract insights from in-person conversations. This forces clinicians to context-switch or recall details later, leading to inefficiency and cognitive overload.
How might we enable doctors to capture, understand, and act on key moments during patient conversations in real time, without disrupting care?
Ideal Customer Profile (ICP)
Although the solution applies to anyone recording in-person conversations, this design specifically focuses on healthcare professionals, such as:
- Primary care physicians conducting patient interviews
- Therapists and clinicians documenting sessions
- Specialists gathering detailed verbal histories
These users are:
- Highly time-constrained
- Easily overwhelmed by UI clutter
- Focused on patient interaction, not managing technology
- In need of accurate, real-time notes for decision-making
The experience had to be intuitive, distraction-free, and designed for rapid mental processing.
Research & Design Exploration
Competitive Analysis
I conducted competitor research of leading transcription apps to understand:
- How competitors display live transcripts
- How they handle recording states and controls
- How they surface AI summaries or chat
- UI patterns for low-distraction mobile workflows
I also used tools like Mobbin, where I reviewed iOS patterns for drawers, live feeds, chat interactions, and recording UX to understand what users already expect.
How Might We Brainstorm
I ran a ‘How Might We’ brainstorm focused on the core user challenges. I mapped a series of prompts such as “How might we allow users to quickly change a recording title without disrupting the session?”, “How might we help users access live notes with minimal distraction?”, and “How might we surface AskFred in a way that feels available but not intrusive?” This exercise helped me cast a wide net of possibilities, challenge initial assumptions, and identify design opportunities that balanced simplicity with real-time functionality.
Brainstorming with AI
I also brought AI into my brainstorming process. I had a quick back-and-forth conversation to explore different ways AskFred could show up during a live recording. I talked through options like keeping AskFred inside the transcription drawer or giving it its own tab, and we walked through the pros and cons of each. The AI helped me think through things like how much navigation each option would require, how distracting it might feel in the moment, and what would be easiest for someone multitasking during an in-person conversation. It was a fast way to pressure-test ideas and move closer to a solution that feels natural for users.
Defining the Solution
After synthesizing research and narrowing exploration, I designed a streamlined mobile experience centered around clarity and ease.
Key Design Decisions
- A persistent bottom recording bar with timer + quick actions ensures users always know recording status.
- A maximize bottom drawer reveals live transcription and AI notes without forcing a context switch.
- AskFred is accessible via a secondary tab, allowing quick question-asking during recording with minimal UI intrusion.
- I designed a persistent bottom recording bar with clear minimize and maximize controls. Tapping the bar expands into the full live audio notes screen, and users can minimize it again just as easily. This keeps navigation simple and predictable, especially for time-pressed users who need to jump between views without friction.
- I introduced AI-suggested titles generated from the live transcript. This helps users quickly confirm or update a recording name without interrupting the conversation or losing focus.
00
:
00
Audio Note
Add title & review privacy settings before recording is uploaded
Title Suggestions
Sleep Issues
New Patient
Transcript
AI Notes
AskFred
Speaker 1
00:01
Hi, thanks for coming in today. I understand you’ve been having trouble sleeping. Can you tell me a little about what’s been going on?
Speaker 2
00:07
Yeah, for the past few weeks I’ve been really restless at night. I fall asleep for maybe an hour, then I wake up and can’t get comfortable again.
Speaker 1
00:18
Got it. When you wake up, do you feel anxious or alert, or is it more physical restlessness?
Speaker 2
00:24
It’s mostly physical. My mind isn’t racing or anything—I just feel like I can’t settle. I toss and turn a lot.
Speaker 1
00:32
How long does it usually take you to fall back asleep?
Speaker 2
00:36
Sometimes an hour, sometimes longer. I end up grabbing my phone or getting out of bed.
Speaker 1
00:47
Have there been any changes in your routine—caffeine, stress, new medications, anything like that?
Speaker 2
00:55
I’ve been a little more stressed with work lately, but no new meds or anything. My bedtime routine is pretty much the same.
Speaker 1
01:00
Okay, that’s helpful. We’ll look at a few possible causes and figure out a plan to help you get more restful sleep.
00
:
00
00
:
00
Audio Note
Add title & review privacy settings before recording is uploaded
Title Suggestions
Sleep Issues
New Patient
Transcript
AI Notes
AskFred
Speaker 1
00:01
Hi, thanks for coming in today. I understand you’ve been having trouble sleeping. Can you tell me a little about what’s been going on?
Speaker 2
00:07
Yeah, for the past few weeks I’ve been really restless at night. I fall asleep for maybe an hour, then I wake up and can’t get comfortable again.
Speaker 1
00:18
Got it. When you wake up, do you feel anxious or alert, or is it more physical restlessness?
Speaker 2
00:24
It’s mostly physical. My mind isn’t racing or anything—I just feel like I can’t settle. I toss and turn a lot.
Speaker 1
00:32
How long does it usually take you to fall back asleep?
Speaker 2
00:36
Sometimes an hour, sometimes longer. I end up grabbing my phone or getting out of bed.
Speaker 1
00:47
Have there been any changes in your routine—caffeine, stress, new medications, anything like that?
Speaker 2
00:55
I’ve been a little more stressed with work lately, but no new meds or anything. My bedtime routine is pretty much the same.
Speaker 1
01:00
Okay, that’s helpful. We’ll look at a few possible causes and figure out a plan to help you get more restful sleep.
Privacy Settings
Only visible to you
HIPAA Compliant
Patient & Provider
HIPAA Compliant
Patient & All Providers
Anyone with Link
New Patient
Title Suggestions
Sleep Issues
Transcript
AI Notes
AskFred
Any patterns in patients sleep cycle?
ST
The patient consistently wakes up about an hour after falling asleep and has difficulty settling back down, describing mainly physical restlessness rather than mental agitation.
They also noted increased work stress recently, which may be contributing to the disrupted sleep pattern.
Suggested questions based off your conversation
What potential causes should I consider?
What clarifying questions should I ask?
Ask Fred about this conversation...
New Patient
Title Suggestions
Sleep Issues
Transcript
AI Notes
AskFred
✍️ AI Summary
The patient reports several weeks of disrupted sleep characterized by waking about an hour after falling asleep and experiencing significant physical restlessness.
They do not report anxiety or racing thoughts during these awakenings. It often takes them an hour or more to fall back asleep, and they sometimes use their phone or get out of bed.
🔑 Key Takeaways
- Wakes consistently about an hour after falling asleep.
- Physical restlessness rather than mental or emotional distress.
- Often prolonged (over an hour).
- Uses phone or gets out of bed, which may worsen sleep fragmentation.
📋 Action Items
Recommend basic sleep hygiene improvements (limiting phone use in bed, consistent routine, winding down earlier).
Explore potential stress-related contributors and whether symptoms align with insomnia or other sleep disorders.
00
:
00
Today
New Patient
2:34 PM · 10 mins
Uploading
Issues with Sleep
9:32 AM · 23 mins
Uploaded
Monday, Nov 27th
Patient Intake
3:30 PM · 45 mins
Uploaded
Issues with Hair Loss
12:45 PM · 26 mins
Uploaded
Follow-Up Visit
8:30 AM · 13 mins
Uploaded
Let’s connect
shannon.trumbull@gmail.com
Hi, I’m Shannon
Senior Product Designer at Muddy Gecko
Designing Real-Time Transcription and AI Notes for Fireflies Mobile App
I designed a new mobile workflow that lets users record conversations, view live transcription and AI-generated notes in real time, and interact with Fireflies’ AI assistant (AskFred)
Overview
Fireflies is an AI assistant that captures, transcribes, and summarizes meetings. While the product excels in virtual meeting environments, the mobile experience for in-person conversations needed to be reimagined. I designed a new mobile workflow that enables users to easily record conversations, view live transcription and AI-generated notes in real time, and interact with Fireflies’ AI assistant (AskFred) during the conversation.
This case study walks through how I approached the redesign across research, ideation, UI exploration, prototyping, and final design.
Problem
Doctors conducting patient interviews need a fast, reliable, and distraction-free way to record, transcribe, and extract insights from in-person conversations. This forces clinicians to context-switch or recall details later, leading to inefficiency and cognitive overload.
How might we enable doctors to capture, understand, and act on key moments during patient conversations in real time, without disrupting care?
Ideal Customer Profile (ICP)
Although the solution applies to anyone recording in-person conversations, this design specifically focuses on healthcare professionals, such as:
- Primary care physicians conducting patient interviews
- Therapists and clinicians documenting sessions
- Specialists gathering detailed verbal histories
These users are:
- Highly time-constrained
- Easily overwhelmed by UI clutter
- Focused on patient interaction, not managing technology
- In need of accurate, real-time notes for decision-making
The experience had to be intuitive, distraction-free, and designed for rapid mental processing.
Research & Design Exploration
Competitive Analysis
I conducted competitor research of leading transcription apps to understand:
- How competitors display live transcripts
- How they handle recording states and controls
- How they surface AI summaries or chat
- UI patterns for low-distraction mobile workflows
I also used tools like Mobbin, where I reviewed iOS patterns for drawers, live feeds, chat interactions, and recording UX to understand what users already expect.
How Might We Brainstorm
I ran a ‘How Might We’ brainstorm focused on the core user challenges. I mapped a series of prompts such as “How might we allow users to quickly change a recording title without disrupting the session?”, “How might we help users access live notes with minimal distraction?”, and “How might we surface AskFred in a way that feels available but not intrusive?” This exercise helped me cast a wide net of possibilities, challenge initial assumptions, and identify design opportunities that balanced simplicity with real-time functionality.
Brainstorming with AI
I also brought AI into my brainstorming process. I had a quick back-and-forth conversation to explore different ways AskFred could show up during a live recording. I talked through options like keeping AskFred inside the transcription drawer or giving it its own tab, and we walked through the pros and cons of each. The AI helped me think through things like how much navigation each option would require, how distracting it might feel in the moment, and what would be easiest for someone multitasking during an in-person conversation. It was a fast way to pressure-test ideas and move closer to a solution that feels natural for users.
Defining the Solution
After synthesizing research and narrowing exploration, I designed a streamlined mobile experience centered around clarity and ease.
Key Design Decisions
- A persistent bottom recording bar with timer + quick actions ensures users always know recording status.
- A maximize bottom drawer reveals live transcription and AI notes without forcing a context switch.
- AskFred is accessible via a secondary tab, allowing quick question-asking during recording with minimal UI intrusion.
- I designed a persistent bottom recording bar with clear minimize and maximize controls. Tapping the bar expands into the full live audio notes screen, and users can minimize it again just as easily. This keeps navigation simple and predictable, especially for time-pressed users who need to jump between views without friction.
- I introduced AI-suggested titles generated from the live transcript. This helps users quickly confirm or update a recording name without interrupting the conversation or losing focus.
00
:
00
Audio Note
Add title & review privacy settings before recording is uploaded
Title Suggestions
Sleep Issues
New Patient
Transcript
AI Notes
AskFred
Speaker 1
00:01
Hi, thanks for coming in today. I understand you’ve been having trouble sleeping. Can you tell me a little about what’s been going on?
Speaker 2
00:07
Yeah, for the past few weeks I’ve been really restless at night. I fall asleep for maybe an hour, then I wake up and can’t get comfortable again.
Speaker 1
00:18
Got it. When you wake up, do you feel anxious or alert, or is it more physical restlessness?
Speaker 2
00:24
It’s mostly physical. My mind isn’t racing or anything—I just feel like I can’t settle. I toss and turn a lot.
Speaker 1
00:32
How long does it usually take you to fall back asleep?
Speaker 2
00:36
Sometimes an hour, sometimes longer. I end up grabbing my phone or getting out of bed.
Speaker 1
00:47
Have there been any changes in your routine—caffeine, stress, new medications, anything like that?
Speaker 2
00:55
I’ve been a little more stressed with work lately, but no new meds or anything. My bedtime routine is pretty much the same.
Speaker 1
01:00
Okay, that’s helpful. We’ll look at a few possible causes and figure out a plan to help you get more restful sleep.
00
:
00
00
:
00
Audio Note
Add title & review privacy settings before recording is uploaded
Title Suggestions
Sleep Issues
New Patient
Transcript
AI Notes
AskFred
Speaker 1
00:01
Hi, thanks for coming in today. I understand you’ve been having trouble sleeping. Can you tell me a little about what’s been going on?
Speaker 2
00:07
Yeah, for the past few weeks I’ve been really restless at night. I fall asleep for maybe an hour, then I wake up and can’t get comfortable again.
Speaker 1
00:18
Got it. When you wake up, do you feel anxious or alert, or is it more physical restlessness?
Speaker 2
00:24
It’s mostly physical. My mind isn’t racing or anything—I just feel like I can’t settle. I toss and turn a lot.
Speaker 1
00:32
How long does it usually take you to fall back asleep?
Speaker 2
00:36
Sometimes an hour, sometimes longer. I end up grabbing my phone or getting out of bed.
Speaker 1
00:47
Have there been any changes in your routine—caffeine, stress, new medications, anything like that?
Speaker 2
00:55
I’ve been a little more stressed with work lately, but no new meds or anything. My bedtime routine is pretty much the same.
Speaker 1
01:00
Okay, that’s helpful. We’ll look at a few possible causes and figure out a plan to help you get more restful sleep.
Privacy Settings
Only visible to you
HIPAA Compliant
Patient & Provider
HIPAA Compliant
Patient & All Providers
Anyone with Link
New Patient
Title Suggestions
Sleep Issues
Transcript
AI Notes
AskFred
Any patterns in patients sleep cycle?
ST
The patient consistently wakes up about an hour after falling asleep and has difficulty settling back down, describing mainly physical restlessness rather than mental agitation.
They also noted increased work stress recently, which may be contributing to the disrupted sleep pattern.
Suggested questions based off your conversation
What potential causes should I consider?
What clarifying questions should I ask?
Ask Fred about this conversation...
New Patient
Title Suggestions
Sleep Issues
Transcript
AI Notes
AskFred
✍️ AI Summary
The patient reports several weeks of disrupted sleep characterized by waking about an hour after falling asleep and experiencing significant physical restlessness.
They do not report anxiety or racing thoughts during these awakenings. It often takes them an hour or more to fall back asleep, and they sometimes use their phone or get out of bed.
🔑 Key Takeaways
- Wakes consistently about an hour after falling asleep.
- Physical restlessness rather than mental or emotional distress.
- Often prolonged (over an hour).
- Uses phone or gets out of bed, which may worsen sleep fragmentation.
📋 Action Items
Recommend basic sleep hygiene improvements (limiting phone use in bed, consistent routine, winding down earlier).
Explore potential stress-related contributors and whether symptoms align with insomnia or other sleep disorders.
00
:
00
Today
New Patient
2:34 PM · 10 mins
Uploading
Issues with Sleep
9:32 AM · 23 mins
Uploaded
Monday, Nov 27th
Patient Intake
3:30 PM · 45 mins
Uploaded
Issues with Hair Loss
12:45 PM · 26 mins
Uploaded
Follow-Up Visit
8:30 AM · 13 mins
Uploaded
Let’s connect
shannon.trumbull@gmail.com