AI Transcription for Medical Notes: Risks and Safeguards

AI transcription for medical notes listens to a patient visit and drafts the clinical note, which is why clinicians are adopting it faster than almost any other tool in healthcare. It is also the reason peer-reviewed researchers have documented speech-to-text systems inventing sentences that were never spoken, in roughly 1 percent of transcriptions.

Below you will find why the documentation burden made this inevitable, what the hallucination research really found, which patients are most affected, and the review process any clinic needs before a generated note reaches a chart.

Why Are Clinicians Adopting AI Transcription for Medical Notes?

Because documentation consumes more of the working day than patients do. That is not a figure of speech, it is what a time and motion study measured.

Christine Sinsky and colleagues published a direct observational study in Annals of Internal Medicine on September 6, 2016, observing 57 physicians across family medicine, internal medicine, cardiology, and orthopedics for 430 hours in four states. During the office day, physicians spent 27.0 percent of their total time on direct clinical face time with patients and 49.2 percent on electronic health record and desk work. The 21 physicians who also kept after-hours diaries reported one to two hours of additional work each night. The findings are summarized in this write-up of the Sinsky time and motion study.

Read those two percentages together and the appeal becomes obvious. For every hour with a patient, roughly two go to the record and the desk. A tool that drafts the note while the visit happens is aimed squarely at the larger number.

The mechanism is straightforward. An ambient scribe records the consultation, an automatic speech recognition model converts audio to text, and a language model reorganizes that transcript into a structured clinical note. Three steps, three places for something to go wrong.

What Did the Hallucination Research Find?

That speech-to-text systems do not merely mishear words. They sometimes generate entire sentences that appear nowhere in the audio.

Allison Koenecke, Anna Seo Gyeong Choi, Katelyn X. Mei, Hilke Schellmann, and Mona Sloane presented Careless Whisper: Speech-to-Text Hallucination Harms at the ACM Conference on Fairness, Accountability, and Transparency in June 2024. Studying OpenAI’s Whisper model, they found roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences that did not exist in any form in the underlying audio. Of those hallucinations, about 38 percent included explicit harms, which the authors group as perpetuating violence, making up inaccurate associations, and implying false authority. The full conference paper sets out the methodology.

One percent sounds small until you attach it to volume. A clinic recording forty visits a day is not looking at a rounding error, and a fabricated line in a chart does not announce itself as fabricated. It reads exactly like everything around it.

The myth worth clearing up: people assume transcription errors look like transcription errors, meaning garbled words or obvious nonsense a clinician would catch instantly. That is what mishearing produces. Hallucination produces fluent, plausible, grammatically correct sentences about things that were never said, which is a fundamentally harder error to spot while skimming.

Which Patients Are Most at Risk?

People whose speech includes long pauses, which the same research identified directly. This is the finding with the clearest clinical implication and the one least discussed in vendor material.

The Koenecke study compared transcriptions of speakers with aphasia, a language disorder that reduces speech expression and commonly follows stroke or brain injury, against a control group without it. Speakers with longer non-vocal durations experienced disproportionately higher hallucination rates, and longer pauses are characteristic of aphasia.

Follow that through to a clinic. The patients whose notes are most likely to contain invented content are among those least able to read the note back and object, and their consultations are often the ones where precise history matters most. From what we’ve seen discussed among clinicians, this is the argument that moves people from “review the notes when I can” to “review every note before signing.”

The same pattern is worth expecting, though not assuming, wherever speech departs from the model’s training distribution: heavy accents, dysarthria, speech affected by late-stage illness, and interpreted consultations with pauses for translation.

What Does HIPAA Require Before You Start?

A signed business associate agreement, before any audio reaches the vendor. A recording of a consultation is protected health information in the most complete sense, containing the patient’s identity, symptoms, history, and diagnosis in one file.

Under HIPAA, a covered entity that engages a vendor to handle protected health information on its behalf must have a written business associate agreement in place. That agreement sets out what the vendor may do with the information, requires appropriate safeguards, and obliges the vendor to report breaches. Business associates are also directly liable for certain provisions of the HIPAA rules. No volume of security certifications on a vendor’s marketing page substitutes for the signed agreement.

Practices weighing vendor contracts more broadly may find our roundup of AI legal tools for small business owners useful for the agreement review itself.

Three questions worth asking any vendor in writing before a pilot: whether they sign a BAA, whether recordings and transcripts are retained after the note is produced and for how long, and whether your patients’ audio is used to train their models. The third question is the one vendors answer least clearly and the one patients would most want asked.

A common mistake worth avoiding: running a “quick trial” on real patient visits before the paperwork is done, on the reasoning that it is only a test. There is no pilot exemption in HIPAA. If real patient audio goes to a vendor without an agreement in place, the exposure is identical whether you called it a trial or a deployment. Test with role-played consultations until the agreement is signed.

How Should a Clinic Put This Into Practice?

As a drafting tool with a mandatory human review step, never as an unattended process. The efficiency gain comes from editing rather than writing, and it survives review comfortably.

1. Sign the BAA before any real patient audio is recorded. Confirm retention periods and whether your data trains their models.
2. Tell patients and obtain consent per your state’s rules. Recording consent laws vary, and some states require all parties to agree.
3. Read every generated note before signing it. You are checking for content that was never said, not only for wording you would change.
4. Pay closer attention on consultations with pauses. Aphasia, interpreted visits, and patients who are breathless or distressed are where the research says risk concentrates.
5. Check medications, doses, allergies, and numbers against your own memory of the visit. These are the details where an invented line does the most harm.
6. Log corrections for the first few months. A record of what the tool got wrong tells you whether it is safe to keep and gives you something concrete to raise with the vendor.

In practice, this looks like several minutes of review replacing considerably longer spent writing, which is still a large saving. Clinics working through the wider administrative load may find our piece on AI for small dental clinic scheduling useful on the front-desk side, and our digital privacy rights beginners guide helpful for framing conversations with patients about their data.

Frequently Asked Questions

How accurate is AI transcription for medical notes?

Word-level accuracy is generally high in good recording conditions, but accuracy is not the only measure. Peer-reviewed research on OpenAI’s Whisper found roughly 1 percent of transcriptions contained entirely fabricated sentences, which is a different failure from mishearing and harder to detect on review.

What is an AI hallucination in transcription?

It is text the system generates that corresponds to nothing in the audio. Unlike a misheard word, a hallucination is usually fluent and plausible, which is precisely why it survives a quick skim of a note. Roughly 38 percent of the hallucinations in the Koenecke study contained explicitly harmful content.

Do I need a BAA with an AI scribe vendor?

Yes, if the vendor handles protected health information on your behalf, which a recorded consultation certainly is. HIPAA requires a written business associate agreement setting out permitted uses and safeguards, and there is no exemption for pilots or trials.

Do patients have to consent to being recorded?

Consent requirements vary by state, and some require all parties to a conversation to agree before recording. Beyond the legal minimum, telling patients plainly is the professional position, since they are the subject of the recording and the note.

Are AI scribes regulated as medical devices?

Documentation tools generally sit outside the device pathway because they transcribe and summarize rather than diagnose or recommend treatment. That means no premarket review of accuracy, which places the entire verification burden on the clinician signing the note.

Will this work for patients with speech difficulties?

Less reliably, and that is a documented finding rather than a guess. The Koenecke research found higher hallucination rates for speakers with longer non-vocal pauses, a pattern characteristic of aphasia. Expect to review these notes more carefully rather than less.

How much time does an AI scribe really save?

It shifts effort from composing to reviewing, and reviewing is faster. Against a documented baseline where physicians spend roughly two hours on records and desk work for every hour of patient contact, even a partial reduction is meaningful, but the saving is real only if the review step is genuinely performed.

Using AI Transcription for Medical Notes Safely

AI transcription for medical notes addresses a documentation burden that research has quantified for a decade, and it does so well enough that adoption is unlikely to slow. The peer-reviewed hallucination findings do not make it unusable. They make the review step non-negotiable, particularly for patients whose speech includes long pauses.

Before any pilot, get the business associate agreement signed and decide who reads every note before it is filed. If your workflow cannot support that review, the tool is not ready for your clinic yet.

This article is general information, not legal or clinical advice. HIPAA obligations and recording consent laws vary by state, so consult your compliance officer and a qualified attorney about your specific situation.