When Humans Hallucinate: What Tech Writers Can Learn from AI's Mistakes

Before we judge AI for being confidently wrong, it may be worth pointing out how often we are incorrect, too

Christopher Noessel, an interaction designer, author, and AI experience architect whose work focuses on how people and intelligent systems work together, recently shared a Facebook post responding to a billboard criticizing generative AI. The billboard message was straightforward: you would not hire an employee who was wrong 10 percent of the time, so why would you trust AI?

The criticism is fair. Large language models are known to invent facts, confuse details, and produce answers that sound convincing even when they are wrong. What interested me more was the assumption underneath the slogan. The comparison only works if people are reliable.

Tech writers spend enough time interviewing subject matter experts to know that isn’t always true. Ask several people how a product feature works and you may hear several explanations. One engineer describes the original design. Another explains how the system behaves today. Support remembers the workaround customers use. Product management recalls a decision that was never documented.

Nobody intends to mislead anyone, yet the facts often arrive tangled together. Much of the work of technical writers involves sorting through those competing accounts.

Human Hallucinations Are More Common Than We Admit

Chris’s post assembled several examples that are uncomfortable precisely because they involve ordinary people rather than obvious bad actors.

A YouGov survey found that 2 percent of Americans said they firmly believe the Earth is flat. Another 7 percent said they were unsure about the Earth’s shape. Among younger adults, the uncertainty was even higher.

A separate Economist/YouGov poll found that 12 percent of Americans agreed that the Moon landing was staged.

Related reading: Conspiracy vs. Science: A Survey of U.S. Public Beliefs

The so-called Mandela Effect offers another example. In a 2022 YouGov survey, 40 percent of Americans said they had shared a memory with others that many people remembered differently or believed to be false. When asked about specific examples, 61 percent remembered the children’s books as the “Berenstein Bears,” even though the books have always been published as the “Berenstain Bears.”

The same survey found that 28 percent of respondents considered parallel universes or alternate dimensions a plausible explanation for these shared memories.

These examples are easy to dismiss because they seem unusual, but memory errors appear in situations with much higher stakes. Eyewitness testimony remains one of the most persuasive forms of evidence presented in court. Yet the Innocence Project reports that eyewitness misidentification played a role in roughly 69 percent of convictions later overturned through DNA evidence.

One of the best-known cases involved Jennifer Thompson, who deliberately studied her attacker’s face so she could identify him later. She selected Ronald Cotton from a lineup, testified against him in court, and remained convinced she had identified the correct person. Cotton spent more than a decade in prison before DNA evidence established that another man had committed the crime.

Why Organizations Write Things Down

Organizations create documentation for many reasons. New products need instructions, APIs need references, and customers need help completing tasks. Over time, documentation also becomes a record of decisions, product behavior, terminology, and procedures that people may remember differently or forget altogether.

Most documentation teams eventually discover procedures that nobody follows, product behavior that differs from the manual, and explanations that change depending on who is asked. Generative AI did not create these problems.

When an AI system produces an incorrect answer, the mistake appears immediately. Human errors often move more slowly. They become outdated procedures, undocumented assumptions, conflicting explanations, and organizational folklore that survives long after the product changes.

Every tech writer has heard someone begin a sentence with, “We’ve always done it this way,” only to discover that nobody can explain why.

People And AI Make Different Types Of Mistakes

People are forgetful. We combine separate events into a single memory, and rely on assumptions, repeat information we heard years ago, and sometimes remember what we expected to happen instead of what actually happened.

Language models fail differently. They predict likely responses from patterns in training data and retrieved information. They may combine facts incorrectly, invent missing details, or generate plausible explanations that lack evidence.

Neither Humans Nor AI Systems Deserve Automatic Trust

It turns out that people and language models need many of the same things: good information, enough context, and a way to check whether an answer is supported by evidence.

Most of us have been in the meeting where three people remember the same decision three different ways. Eventually someone finds the ticket, the email, or the requirements document, and the argument ends.

Tech Writers Already Understand The Problem

Must we choose between reliable humans and unreliable AI? I hope not.

Anyone who has worked in documentation for very long has encountered the software engineer or product creator who explains a feature with complete confidence, only for testing to reveal that the product behaves — ahem — differently.

Every doc team eventually finds a procedure that nobody actually follows. Support teams have their own version of this problem, repeating explanations they conjure up that might have matched the product several releases ago but no longer do.

AI gives employees another place to get answers, but somebody still has to check whether those answers match the product, the policy, and the documentation.

What This Means For Tech Writers

Technical writers regularly walk into situations where different people remember the same event differently. One person recalls the decision, another remembers the exception, and somebody else insists the whole thing changed two releases ago. Eventually we have to sort through the emails, tickets, release notes, and meeting minutes to figure out what actually happened.

That work becomes even more important as AI systems begin answering questions alongside documentation, support teams, and subject matter experts.

Organizations have always operated with incomplete information. People remember meetings differently, repeat outdated explanations, and hold onto decisions that quietly changed years ago. Documentation helps reduce some of that confusion, even if it never eliminates it completely.

AI now participates in many of the same conversations that once belonged entirely to coworkers, support staff, and subject matter experts. That puts more pressure on the underlying information. Procedures, release notes, requirements, and product documentation increasingly serve both people and machines.

For tech writers, the work is familiar. Somebody still has to determine what is true, what changed, and which version belongs in the documentation. 🤠