Most discussions about AI costs focus on language models. That’s understandable. Models have names, rankings, benchmarks, pricing pages, and marketing campaigns.

Content gets far less attention. That’s surprising because AI systems spend much of their time looking for information before they generate an answer.

That distinction becomes more important as organizations move beyond experimentation and begin deploying AI-powered support systems, internal assistants, knowledge portals, and agentic search experiences at scale.

Stripe, the online payments company that processes transactions for millions of businesses, recently caught my attention for exactly that reason. The company announced AI usage billing, a capability that charges customers in proportion to what they consume (e.g., tokens processed, compute seconds, API calls, agent actions) rather than a flat fee.

The announcement wasn’t about model quality. It was about consumption. How many tokens were used? Which application consumed them? How should those costs be allocated? How should they be recovered? By whom?

Once organizations can see AI consumption, they’re eventually going to ask why some AI-powered experiences cost more to operate than others. In many cases, part of the answer may be the content itself.

Why Does Content Ambiguity Increase AI Costs?

Tech writers have always worried about ambiguity because it frustrates readers. AI systems introduce another reason to care.

Suppose a procedure says:

“Approve the request after validation completes.”

Most people working in that environment can probably figure out what’s happening. They know who performs the approval and they understand what validation means. They probably know what event signals validation completion.

AI systems don’t share that background knowledge.

If the role, condition, or triggering event isn’t documented, the AI system often searches for that information elsewhere. As documentation repositories grow, those searches become more expensive. The answer may already exist, but the system may retrieve multiple documents, compare competing explanations, and evaluate additional context before it can decide which information to use.

The System Has To Find The Missing Pieces

Many organizations assume AI works like a very fast employee who already knows where everything is. In reality, AI systems spend a surprising amount of time searching, comparing, ranking, and evaluating information before generating an answer.

A user asks a simple question. The retrieval system may pull multiple documents, compare overlapping explanations, evaluate terminology, identify the most likely workflow, and discard information that appears unrelated.

When our documentation is clear, the search narrows quickly. But, when our content contains inconsistent terminology, undocumented assumptions, duplicate information, or missing workflow context, the system often retrieves more material than it otherwise would.

Organizations are beginning to measure that work.

What Does TRACE Have To Do With It?

TRACE stands for Tasks, Roles, Actors, Conditions, and Events. The framework grew out of an observation I’ve encountered repeatedly in documentation reviews: many problems aren’t caused by missing information. They’re caused by missing context.

👉🏾 The task is described, but nobody identifies the role responsible for it.
👉🏾 The workflow is documented, but the triggering event isn’t.
👉🏾 The actor changes halfway through the process without being identified.
👉🏾 The conditions that determine success or failure exist only as assumptions.

Human readers often work around those gaps because they already understand the environment.

TRACE became interesting to me because it highlights exactly the information retrieval systems often go looking for. When tasks, roles, actors, conditions, and events are explicit, the system has stronger clues about what a topic is describing and how it relates to other topics.

Technical Documentation Has Entered the Cost Conversation

For years, tech writers justified structured content using familiar arguments around reuse, consistency, metadata, governance, localization, and maintainability. Those arguments still hold up today.

What’s changing is the audience for the discussion. As organizations start tracking AI consumption, content issues may begin appearing in conversations that previously focused on infrastructure, support operations, and technology spending.

We’ve spent years cleaning up duplicate content, arguing about terminology, and chasing down missing workflow details. Those activities may sound unrelated to AI operating costs. They probably aren't.

Those same issues may also increase the amount of work AI systems perform before it can answer a question.

Five years ago, most conversations about structured content focused on publishing efficiency, governance, and reuse. AI may pull the discussion into a different part of the organization, the people responsible for operating budgets.

Why Tech Writers Should Pay Attention

Organizations are building AI systems on top of documentation repositories, knowledge bases, support portals, and content collections that were never designed for machine consumption. Some of those systems will work remarkably well. Others will consume more resources than expected because the underlying content requires constant interpretation.

We’ve been talking about context, consistency, terminology, and structure for years. If organizations begin tracking AI consumption the way they track cloud spending, those conversations may start happening in different meetings with different stakeholders. 🤠

For years, we tech writers have been trying to explain structured content to people who looked at metadata the way you might look at steamed broccoli. 🥦

Necessary? Maybe. Exciting? Absolutely Not.

Then AI arrived, wrapped in venture capital and optimism, promising to revolutionize the way we work and eliminate our drudgery.

And now, suddenly, organizations are discovering something painful we tech writers have been muttering for decades: Messy information is expensive. 💰

Welcome to the strange new world where unclear docs and inconsistent product information can increase infrastructure spending.

Tokens: The Invoices Nobody Talks About

Let’s start with tokens because the AI industry throws the word around as though everybody understands it.

Every time an AI system:

• reads a prompt

• retrieves documentation

• processes instructions

• accesses a service

• searches for references

• remembers conversation history

• generates a response

…it consumes tokens. And tokens cost money. 💴

At small scale, the costs feel trivial. A chatbot rewrites our awkward email message. A support assistant summarizes a customer support case. Somebody in marketing asks AI to “make this sound more dynamic” and receives three paragraphs that sound like a regional bank trying to flirt.

No big deal, right?

But organizations aren’t deploying AI for one interaction. They’re deploying it everywhere for all sorts of reasons.

• Customer support

• Internal search

• Knowledge retrieval

• Workflow automation

• Developer assistance

• Sales enablement

• Training systems

• Agentic orchestration

That’s where the meter starts spinning.

Related reading: AI can cost more than human workers now

AI Doesn’t Understand Our Documentation

This is important. AI models don’t “understand” our content the way humans do.

Humans are remarkably forgiving creatures. We can infer meaning from terrible prose, missing context, vague terminology, and screenshots that look like they were captured during a small electrical fire 🔥.

AI systems aren’t really understanding anything (in the human sense). They’re processing statistical language patterns and attempting to assemble probable meaning.

Which means ambiguity creates work. And work creates token consumption.

Now imagine the AI retrieves:

• six overlapping procedures

• two outdated workflows

• contradictory terminology

• troubleshooting mixed into conceptual overviews

• outdated screenshots from 2019

• a release note pretending to be task guidance

• an unlabeled warning block that reads like a hostage note

The system now has to process all that material trying to determine what’s actually relevant.

💰 Costs start to rack up quickly.

Retrieval of data costs tokens. Ranking what it found and reasoning across it costs more. Generating an answer adds — you guessed it — more expense.

Then the agent may decide it’s uncertain and retrieve even more content.

Because your documentation repository has become the informational equivalent of a kitchen junk drawer filled with expired coupons, tangled charger cables, mystery batteries, and three keys nobody recognizes but nobody dares throw away.

Agentic AI Makes the Problem Worse

Traditional chatbots were comparatively simple. They answered a question and moved on with their lives.

Agentic AI systems are different.

These systems often:

• interpret intent

• reformulate questions

• retrieve multiple content chunks

• compare sources

• invoke tools

• summarize results

• validate outputs

• retry retrieval

• generate next actions

Every step burns tokens.

And the less explicit your documentation is, the harder those systems must work.

This is where technical writers should become deeply interested in the economics of ambiguity.

Because the AI industry increasingly profits from uncertainty.

The more your content forces AI systems to infer meaning, the more compute gets consumed.

And now companies like Stripe are openly building systems to meter, monitor, bill, and even profit from AI consumption costs.

That’s not science fiction anymore. That’s accounting.
https://techcrunch.com/2026/03/02/stripe-wants-to-turn-your-ai-costs-into-a-profit-center/

The AI industry is quietly transforming tokens into a utility model.

Like electricity.

Or water.

Or those suspicious resort fees hotels add while pretending the hallway Wi-Fi is a luxury experience.

This Is Why Structured Content Matters Again

Technical writers have spent years talking about:

• reuse

• taxonomy

• metadata

• controlled language

• modularity

• governance

Usually while someone from another department asks whether we can “just put it in Confluence.”

Now AI has accidentally made all those conversations financially relevant.

Structured content reduces retrieval waste.

Clear topic types help AI systems distinguish between:

• tasks

• concepts

• references

• troubleshooting

• policies

Consistent terminology reduces ambiguity.

Metadata narrows retrieval scope.

Reuse reduces duplicate processing.

Governance improves precision.

The goal is no longer just helping humans find information.

The goal is helping AI retrieve the smallest amount of information necessary to generate a reliable answer.

That’s token efficiency.

And token efficiency is becoming operational cost control.

PREA Makes AI Work Less

This is where PREA becomes especially interesting.

DITA already gives organizations structural consistency. But structure alone doesn’t always expose operational meaning clearly enough for AI systems.

A topic can validate perfectly while still leaving important relationships implicit.

PREA — Process, Role, Event, Actor — helps make those relationships explicit.

Instead of forcing AI systems to infer:

• who performs an action

• what triggers a workflow

• which actor changes a system state

• when a process starts or ends

…the content tells the system directly.

That matters enormously in agentic retrieval systems.

Without PREA, an AI may retrieve dozens of vaguely related chunks because it’s relying mostly on keyword overlap and statistical proximity.

With PREA-informed content, retrieval becomes narrower and more precise.

The AI reads less because the documentation says more.

That’s the magic.

Not magical in the “wizard sparks and unicorn tears” sense.

More like the deeply satisfying magic of finally labeling storage bins in the garage after years of opening random boxes looking for extension cords.

Every Vague Heading Creates a Tiny AI Tax

This may sound dramatic, but it’s true.

Every ambiguous title.

Every duplicated procedure.

Every overloaded topic.

Every inconsistent term.

Every undocumented workflow transition.

Every “Important Information About Setup” heading quietly increases computational effort.

One isolated retrieval operation costs almost nothing.

Millions of retrieval operations across thousands of users become a budget discussion.

And organizations are going to start noticing.

Especially once finance departments realize AI systems don’t merely consume software licenses.

They consume language.

Technical Writers Are Quietly Becoming Infrastructure People

Nobody entered technical communication dreaming of optimizing inference efficiency.

Yet here we are.

The role is changing.

Technical writers are no longer just producing documentation for human readers.

We’re increasingly designing information environments for machine consumption too.

That means our choices affect:

• retrieval precision

• context size

• inference efficiency

• hallucination risk

• orchestration overhead

• operational AI costs

Which is honestly a bizarre career development.

Somewhere in 1998, a technical writer was carefully adjusting bullet indentation in Microsoft Word while eating yogurt from a desk drawer and worrying about printer drivers.

That person probably didn’t imagine future writers would one day influence enterprise AI infrastructure economics.

And yet.

Here we are.

Because in the age of agentic AI, content isn’t merely communication anymore.

It’s fuel.

And fuel costs money.

Humans do this so naturally that we rarely notice it. Machines don’t.

I was thinking about that while reading a recent LinkedIn post from knowledge graph researcher Benny Cheung. He described the progression from data to taxonomy to ontology to knowledge graphs and argued that organizations that stop too early in that progression are likely to struggle with AI.

The post resonated with me because technical writers have been dealing with parts of this problem for years, although we don’t usually use the same vocabulary.

Organizing Information Isn't The Same Thing As Defining Meaning

Most organizations have plenty of places to store information. They have databases, content management systems, knowledge bases, SharePoint sites, and file repositories.

Storage isn’t the problem.

Cheung points out that a database schema and an ontology solve different problems. A schema describes how information is organized. An ontology describes what something is, how it relates to other things, and what rules govern those relationships.

Many organizations blur those distinctions. They assume that once information is structured, categorized, and stored, meaning somehow comes along for the ride.

It doesn’t. Not even close.

Consider the difference between a content label such as “firmware update” and a description that explains who performs the update, what event triggered it, what conditions must be true before it can begin, and what happens when it succeeds or fails.

👉🏾 One helps identify a topic.
👉🏾 The other helps explain how something works.

For AI systems, those details matter.

Taxonomy Helps You Find Things

Technical documentation teams have spent years building taxonomies. We classify products, features, audiences, document types, and metadata values. That work is useful because it helps people find information.

The challenge is that classification only answers part of the question.

AI systems increasingly depend on structured context, not just text. Knowledge graphs and RDF (Resource Description Framework) represent entities and the relationships between them, while action and process models make it explicit who performs an action, what object is affected, what events or conditions shape a workflow, and how activities connect across a process. That structure helps AI systems retrieve information with context and perform multi-step tasks more reliably.

Those relationships are often scattered across multiple documents or left unstated because human readers can usually infer them. AI systems don’t always infer them correctly or consistently.

That’s where many documentation repositories start to show their age. The information exists, but the relationships are incomplete, inconsistent, or implied.

Why Documentation Can Confuse AI

When people read our documentation, they bring a lifetime of experience to the process. They know that customers and technicians have different responsibilities. They recognize that a setup procedure may not apply during routine operations. They can usually tell when a warning belongs to a specific circumstance rather than every circumstance.

Documentation often depends on that kind of judgment.

Large language models don’t read the way people do. Not even remotely. They generate responses based on patterns and probabilities. When key relationships are missing, the model may connect things that shouldn’t be connected, skip conditions that matter, or assign actions to the wrong person or system.

The result can sound reasonable while being wrong.

That’s one of the reasons I’ve been working on the TRACE framework:

• Tasks

• Roles

• Actors

• Conditions

• Events

The framework grew out of a simple observation. Many documentation problems that affect AI have less to do with missing information than with missing context. The facts may be present, but the relationships between those facts often aren’t.

Tech Writers Are Already Working in This Space

Tech writers routinely define components, describe workflows, document responsibilities, explain dependencies, and identify conditions that affect outcomes.

Most of us think of that as documentation. But, viewed another way, it’s also the work of defining how a system operates.

That’s why I think some organizations are overlooking a valuable source of expertise. They want AI systems that can answer questions across large collections of enterprise knowledge, but they often treat documentation as a publishing function rather than a knowledge-definition function.

The people who understand the relationships are frequently the same people who wrote the documentation.

Why Knowledge Graphs Are Back In The Conversation

Knowledge graphs have been around for years, but many organizations never saw a compelling reason to invest in them. Search generally worked “good enough.”

Generative AI changed expectations. Organizations now want systems that can explain relationships, answer questions in context, connect information across departments, and adapt responses to different situations.

Those capabilities depend on more than retrieving content. They depend on understanding how pieces of information relate to one another.

Knowledge graphs can help with that, but only if the underlying content contains clear relationships. If the source material is ambiguous, the graph inherits the ambiguity.

At some point the AI starts filling in the blanks. What appears to be an AI problem may be exposing weaknesses that have existed in our documentation all along.

A Different Way to Think About AI Readiness

Many conversations about AI readiness focus on models, tools, and infrastructure. Those things matter. But organizations may also need to look at the information itself.

👉🏾 Is operational context explicit?
👉🏾 Are actors clearly identified?
👉🏾 Are conditions documented?
👉🏾 Are workflows connected in ways that a machine can follow without making assumptions?

Those questions sound a lot like documentation questions.

That’s why Cheung’s post caught my attention. It highlights a gap that technical writers have been working around for a long time. As organizations push AI deeper into customer support, training, service delivery, and enterprise search, that gap becomes harder to ignore.

The issue may not be that organizations lack content. It may be that too much of the meaning still lives in the heads of the people reading it. 🤠

Subscribe now

Here’s the problem with that: technical documentation was never primarily about typing sentences. The real work has always been deciding what matters, how information connects, what order things belong in, which warnings will save somebody from catastrophe, and which missing prerequisite is about to turn a software deployment into a small electrical fire in the server room.

AI won’t eliminate tech writing, but it is exposing what tech writing actually is.

The Great Decoupling

For decades, writing and architecture were welded together. If you wanted documentation, someone had to manually draft every sentence. Writing and thinking looked like the same activity because they happened in the same place.

AI broke that relationship.

The easiest way to understand this is to stop thinking about documentation and start thinking about construction.

For years, we were expected to design the blueprint and lay every brick ourselves. Want a 50-page manual? Congratulations. Please hand-place 20,000 words while also managing review cycles, fighting SharePoint permissions, and wondering why the product changed three days before launch and nobody bothered to inform you.

Now AI shows up as an automated bricklayer capable of laying ten thousand bricks a second. That sounds impressive (until you realize what happens when you unleash an automated bricklayer without a blueprint).

Suddenly there’s a staircase leading into a hedge. The bathroom opens directly into the kitchen. A wall suddenly appears across the driveway.

The machine builds quickly. It just doesn’t build intelligently on its own. That’s the real shift. Drafting has become cheap. Architecture has become valuable.

Your Company’s AI Is Only As Smart As Its Documentation

Organizations keep spending on AI while feeding it a digital landfill full of outdated PDFs, random Google Docs, unstructured help articles, inconsistent terminology, duplicate procedures, and seventeen different ways to describe the same button.

AI systems don’t magically create truth. They retrieve information from whatever source material exists. If the source material is a mess, the AI becomes a highly confident confusion machine.

This is why information architecture suddenly matters to executives who previously thought metadata was some kind of skin condition.

Technical writers aren’t just producing content anymore. They’re building the operational knowledge layer that AI systems depend on to function safely.

And honestly, some of us have been trying to explain this for twenty years while being treated like the office people who “make the PDFs prettier.”

The New Career Split

Although my crystal ball 🔮 is cracked (and my psychic powers are limited), I predict tech writing is no longer one profession moving in one direction. It’s splitting into several paths. Four, for now.

Read more

For most of the internet’s history, humans directly interacted with content. We searched, clicked, read, compared, interpreted, and decided what to trust. Websites competed for our attention. Search engines acted as maps that guided us toward information, but humans still visited the original destination.

That relationship is beginning to change.

Increasingly, AI is becoming the layer between humans and information. Instead of navigating websites, people are asking questions. Instead of reading ten links, they are receiving one synthesized answer. Instead of evaluating sources themselves, they are trusting systems to interpret the internet on their behalf.

The shift may sound subtle, but it fundamentally changes the role of content online. Content is no longer just something humans consume directly. It is becoming raw material that machines interpret, summarize, rank, and repackage before humans ever see it.

We are entering the era of invisible content.

The Collapse of the Footnote

In many ways, this transformation is already here. Google’s AI Overviews, ChatGPT, Perplexity, Microsoft Copilot, and other conversational systems are changing how people interact with knowledge. Industry analysts at Gartner have projected that traditional search engine volume is on track to decline by 25 percent as users increasingly migrate toward AI driven conversational experiences.

For publishers, media companies, and enterprise organizations built around discoverability, the implications are structural. The internet is shifting from a referral driven network to an ecosystem increasingly shaped by extraction and synthesis.

As Cloudflare CEO Matthew Prince recently observed while discussing the future of AI search:

“People aren’t following the footnotes.”

To understand the scale of this transition, look at the widening gap between machine crawling and human visiting. A decade ago, search engines crawled websites primarily to guide humans back to original sources. Today, AI systems increasingly consume information without returning meaningful traffic to publishers. According to comments shared publicly by Prince, emerging AI platforms appear to generate dramatically higher crawl to referral ratios than traditional search engines, with extraction metrics for major LLM crawlers scaling into the thousands or even tens of thousands of pages scraped for every user redirected back to a source.

The result is already becoming visible across digital publishing ecosystems. Multiple analytics firms and media organizations have documented significant drops in referral traffic as users increasingly receive summarized answers directly within AI interfaces. But traffic loss is only part of the story. The deeper shift is psychological.

The internet is moving from an environment where humans evaluate information themselves to one where AI increasingly mediates reality for them.

That changes how trust works online.

From Source Literacy to Interface Trust

For years, trust on the internet was tied to visible signals. People trusted recognizable publications, established brands, expert authors, or communities they felt aligned with. Even when misinformation existed, users still had some awareness of where information originated.

AI changes that relationship because it compresses many sources into a single confident response. The original context disappears. Nuance fades. Contradictions flatten. In many cases, even the source itself becomes secondary.

The answer becomes the interface.

This compression also amplifies something psychologists have long studied: automation bias. Humans naturally tend to place greater trust in structured, automated outputs, especially when they appear confident and efficient. Researchers are increasingly examining how the fluency and conversational tone of Large Language Models can create an illusion of authority even when information is incomplete, outdated, or inaccurate.

People quickly adapt to whatever feels easiest, and fast conversational AI fits that pattern well. Once users get used to instant, low-effort answers, going back to slower manual research feels inconvenient, but convenience alone does not guarantee trust: people may prefer the speed while still questioning whether the answer is accurate, complete, or safe.

When humans begin emotionally trusting systems that summarize reality for them, the internet changes from a network of information into a network of mediated interpretation.

The Enterprise Imperative: Content As Machine Infrastructure

This creates an entirely new playbook for organizations far beyond media companies.

For content strategists and information architects, this may become one of the defining challenges of the next decade. We are no longer designing content exclusively for human readers. Increasingly, we are designing it for machine interpretation.

Modern AI systems rely heavily on structured and retrievable information. When organizations use Retrieval Augmented Generation frameworks to connect AI models with enterprise knowledge, the quality of the underlying content architecture becomes critical.

AI systems perform better when information is modular, semantically organized, clearly tagged, and consistently maintained.

If corporate knowledge bases, technical documentation, policies, or support systems are fragmented, inconsistent, or buried inside static files, the AI pipeline begins to weaken. Messy, incomplete, or inconsistent inputs usually lead to unreliable AI outputs because models can only work with the context and constraints they are given. When prompts are vague or poorly structured, the system is more likely to guess, omit important details, or produce inaccurate results. Even sophisticated AI models cannot consistently produce trustworthy outcomes from chaotic source material.

This is why the future of content strategy may depend less on publishing volume and more on governance, structure, and information integrity. The real competitive advantage will not come from generating endless amounts of synthetic content. It will come from building systems that preserve clarity, provenance, consistency, and trust in environments increasingly saturated with machine generated information.

A Premium On Authenticity

Ironically, as generative AI makes text, images, video, and code easier and cheaper to produce, genuinely trustworthy information may become more valuable, not less.

Human credibility, transparent sourcing, and verified expertise become premium.

We are already beginning to see the early foundations of this shift. Initiatives such as C2PA, the Coalition for Content Provenance and Authenticity, are developing technical standards that help verify where digital content originated and whether it has been manipulated. At the same time, organizations are investing more heavily in structured content systems, metadata frameworks, modular authoring models, and governed knowledge ecosystems that can support trustworthy AI experiences at scale.

Practices once considered niche disciplines within technical communication and enterprise documentation are steadily moving toward the center of business strategy.

The internet has always evolved through layers. First came static pages, then social platforms, then algorithmic feeds. AI introduces another layer entirely: interpretation itself.

The biggest change AI brings to our digital landscape is not simply that machines can create content. It’s that humans may increasingly stop seeing where that content came from in the first place, making the invisible architecture behind information one of the most important forms of infrastructure in the AI era.

The conversation about AI often focuses on what machines can create. But the more profound question may be what humans stop doing.

When AI becomes the primary interface between people and information, fewer users visit original sources, compare perspectives, examine evidence, or evaluate context for themselves. Increasingly, those responsibilities shift to the systems that retrieve, interpret, and present information on their behalf.

That makes the future of content larger than a publishing problem or a search problem. It becomes a trust problem.

The organizations that succeed in the AI era will not necessarily be those that generate the most content. They will be those that build the most trustworthy knowledge ecosystems—systems where information is accurate, structured, traceable, and resilient enough to survive multiple layers of machine interpretation.

For content pros, this represents a significant shift in responsibility. Content isn’t simply a communication asset — it’s part of the infrastructure through which people understand reality.

The era of invisible content is not really about content disappearing. It’s about the growing importance of the information architecture, governance, content provenance, and trust mechanisms that remain invisible to users but increasingly determine what they see, believe, and act upon. 🤠

Swaha Bhattacharya is a senior content strategist and technical communicator with over 11 years of experience in enterprise technology. Having worked extensively in structured content, documentation, knowledge systems, and AI-enabled experiences, she writes about the future of content strategy, information architecture, and trust in an increasingly AI-mediated world.

That’s what makes the example a useful teaching aid for tech writers looking to make the tech docs content easier for AI answer engines to process.

The topic highlights a growing problem in tech docs — content that works adequately for humans can still be highly ambiguous when AIs consume it.

Humans are remarkably good at filling in missing context. We infer relationships automatically, and we assume causality. We understand implied actors and unstated conditions because we’ve spent our entire lives navigating incomplete communication.

AI systems don’t actually “understand” any of those relationships. In lieu of understanding, they infer them probabilistically. That distinction matters.

The SimpliSafe topic contains several examples where operational context is implied instead of modeled explicitly. Those gaps create opportunities for retrieval errors, procedural confusion, and AI hallucinations.

The original topic is here: SimpliSafe: Updating Your SimpliSafe System

I evaluated the topic using TRACE, a framework I’ve been developing to identify contextual signals AI systems need in order to generate reliable answers.

TRACE looks for:

• Tasks

• Roles

• Actors

• Conditions

• Events

The goal isn’t to make documentation sound robotic. It’s to reduce ambiguity by making operational context explicit enough that AI systems don’t have to guess.

Here’s where the topic begins to break down.

Read more

👉🏾 Not because the model is “stupid.”
👉🏾 Not because the AI vendor failed.
👉🏾 Not because Mercury is in retrograde again.

The problem is probably the documentation itself.

Welcome to retrieval contamination. It’s one of those phrases that sounds like a plumbing issue in a pharmaceutical lab, but it describes a very real problem tech writers are about to spend the next several years untangling.

Retrieval contamination happens when an AI system pulls instructions, warnings, conditions, or procedural steps from the wrong source and blends them into an answer that sounds plausible enough to pass casual inspection. The AI doesn’t necessarily invent the information. In many cases, it retrieves real information from somewhere else in our documentation set and applies it to the wrong product, version, role, workflow, or situation.

That’s the important distinction. This isn’t hallucination in the classic sense (there’s no such thing). It’s often contamination through proximity.

And we’re sitting right in the blast radius.

The Documentation Equivalent Of Putting Leftovers In The Wrong Container

You know how someone puts mashed potatoes into the yogurt container and then the next morning you confidently spoon potatoes into your coffee? 😆 Hopefully not — but if something similar has ever happened in your world — it’s like that.

AI systems retrieve content by similarity. They look for patterns, relationships, terminology overlap, semantic closeness, and contextual signals. If your docs repeatedly reuses vague phrases like “press the reset button,” “restart the device,” or “update the firmware,” the system may retrieve procedures from multiple products that happen to look statistically related.

Especially if:

• the metadata is weak

• product names are inconsistently applied

• versioning is at worst, non-existent or at best, sloppy

• our docs assume human readers will “know” which product a paragraph belongs to

Humans use common sense and visual and other sensory-fueled context to separate instructions. AI retrieval systems don’t have that luxury.

To a retrieval engine, these two sentences may look dangerously similar:

“Press and hold the reset button for 10 seconds.”

and

“Press and hold the reset button until the light flashes amber.”

If the surrounding contextual clues are weak, the system may merge nearby fragments into a Frankenstein procedure assembled from multiple devices like some kind of support-ticket centaur.

Read more

👉🏾 Too complicated
👉🏾 Too rigid
👉🏾 Too enterprise
👉🏾 Too many angle brackets
👉🏾 Too old
👉🏾 Too “legacy”

And perhaps most emotionally loaded of all:

“Won’t structure kill our creativity?”

That last one always fascinated me because nobody asks airline pilots whether take-off preparation checklists suppress self-expression. Nobody asks surgeons whether standardized procedures interfere with their artistic vision while someone’s spleen is open on the table. 😷

Meanwhile, tech docs teams using structured content quietly continued doing things their less-structured counterparts struggled to manage consistently: multichannel publishing, large-scale localization, semantic metadata management, content reuse, personalization, governance, and automation.

Because XML never left – at least not in oplaces where leadership understood the business value structure provided.

The funny part is that XML’s roots in Standard Generalized Markup Language (SGML) made some people treat it like historical reenactment software. Mention SGML at a conference and some younger attendees react like you just brought up Mesopotamian irrigation techniques.

And yet, despite all the mockery, structured content systems kept quietly powering enormous documentation ecosystems behind the scenes while the rest of the industry periodically reinvented badly organized content in various ways.

Today, AI has arrived and software engineers, startup founders, and self-appointed “prompt engineers” are wrapping prompts in XML tags because they are learning some of the same lessons we learned years ago:

Computers work better when you stop making them guess.

Somewhere, a senior DITA architect just smiled so hard they startled the cat and pulled a neck muscle.

Related: XML Tutorial from W3Schools

AI Needs More Structure Than People Expect

Large language models are very sophisticated prediction systems. Which means ambiguity creates problems very quickly.

When an AI model receives a giant content hairball comprised of loosely organized technical documentation, it must infer:

👉🏾 What is context
👉🏾 What is instruction
👉🏾 What information is authoritative
👉🏾 What constraints matter
👉🏾 What format the user wants
👉🏾 Which information should be ignored

That’s a tremendous amount of interpretation (where things begin going sideways).

Tech writers understand this already because humans behave exactly the same way when our docs lack structure. 🤠

Read more

That doesn’t mean the discussion about capability is wrong; it’s just incomplete.

We’ve spent a lot of time marveling at AI’s ability to summarize, rewrite, classify, retrieve, generate, explain, translate, categorize, and answer customer questions. But somewhere behind the flashy demo, the meter is running. 💰

And not metaphorically speaking. Every AI-powered answer has a cost. The infrastructure to keep AI systems grounded, current, auditable, secure, and useful isn’t free.

Tokens cost money. Retrieval does, too.

As does inference, monitoring, rework, escalations, and human review.

Related reading: What Is Token-Based Pricing for AI Models

Recent Reporting Shows Many Companies Are Already Wrestling With This Shift

Reuters reports that Target is weighing AI tool costs as vendors move toward usage-based pricing, where token consumption and scale can quickly change the economics.

Read more

[TL;DR] Findings from the 2026 AI In Technical Documentation Survey provide tech docs leaders a useful reality check: “AI adoption” is not the same thing as “AI operationalized.”

Your team may have plenty of AI activity — people experimenting, prompting, and sharing tips — while very little of it shows up in repeatable, governed production work. That gap is where leaders can create leverage.

Respondents to the 2026 AI In Technical Documentation Survey were asked whether they were “currently using” AI tools in their tech writing work.

Here’s what we found

AI has already moved into the technical documentation toolbox in a big way.

If you are not using AI at all, you are now officially “in the minority,” which is a weird place to be for a profession built on standards and best practices.

AI is no longer the quirky side project someone tries on a Friday afternoon when the build finally stops failing.

But the numbers also show a meaningful pause at the edge of real production dependence. About one in five respondents are not using AI as a true production tool.

Bold stances in 2026 — like refusing to adopt spellcheck because you prefer the thrill of raw, unfiltered typos.

The takeaway is not “everyone is all-in.” It’s that many teams are still in the awkward middle stage: testing what works, figuring out what is safe, and negotiating where AI belongs in a process that already has enough ways to go sideways without adding a probabilistic text generator to the mix.

Read more

You’ll hear this immediately after someone proposes adding content typing, metadata, role labeling, taxonomy governance, semantic structure, or any other activity requiring humans to think carefully for more than seven consecutive minutes.

“The AI can infer the intent.”

“The model understands context.”

“Modern AI is really sophisticated now.”

And there it is. The magical thinking phase of enterprise AI adoption.

• Not the technical phase

• Not the governance phase

• Not the budgeting phase

The magical thinking phase. 🧞‍♂️

This is often where leaders attempt convince themselves that probability engines are somehow replacing the need for precision.

Meanwhile, tech writers are sitting in the corner like exhausted air traffic controllers wondering why everyone suddenly wants to remove the runway lights because airplanes have GPS now.

AI Does Not “Understand” Our Documentation

This is the first thing tech writers must recognize when these conversations arise.

AI systems like large language models do not understand documentation the way humans do. They don’t comprehend intention, accountability, risk, authority, or operational nuance. They predict likely relationships between patterns of words.

That distinction matters more than people realize.

When humans encounter vague instructions, they often compensate using experience, judgment, and contextual clues. We’re surprisingly good at this. We’ve had practice. Humanity has spent thousands of years decoding ambiguous instructions from bosses, governments, user manuals, and passive-aggressive family members.

AI systems compensate differently. They generate statistically plausible interpretations.

Sometimes those interpretations are correct, while other times they’re spectacularly wrong. And the more ambiguous the source material becomes, the more interpretive work the AI has to perform.

That’s not intelligence. That’s interpolation.

Ambiguity Becomes Operational Debt

For years, poorly structured documentation mostly punished humans. Our readers became frustrated. Support calls increased. New employees took longer to onboard. Procedures got skipped. Nobody enjoyed any of this, but the damage was somewhat contained.

AI changes the scale of the problem.

Now ambiguous content isn’t merely read by humans. It’s ingested, chunked, embedded, retrieved, summarized, synthesized, recombined, and operationalized by machines.

Which means every undocumented assumption suddenly matters.

• If a procedure never explicitly states who performs an action, the AI has to guess

• If preconditions are implied instead of stated, the AI has to infer applicability

• If process explanations are mixed inside step-by-step instructions, the AI has to determine which text represents action versus explanation.

If six departments all created slightly different versions of the same procedure over eight years because governance was treated like optional cardio, the AI now retrieves conflicting truths simultaneously and attempts to merge them into one coherent answer.

What emerges is often less “artificial intelligence” and more “organizational fan fiction.”

“But The AI Can Infer Meaning” Is Missing The Point

Of course AI can infer meaning. Humans can infer meaning too. That’s not the benchmark.

The real question is this:

Why are we designing enterprise knowledge systems that require inference in the first place?

Especially when the content could’ve been explicit.

This is where tech writers need to reposition the conversation. The goal of structured content isn’t to make documents prettier, more academic, or more technically fashionable. The goal is to reduce ambiguity before ambiguity becomes automation.

Because once AI systems start generating answers from our content ecosystem, every vague sentence becomes a liability multiplier.

The Difference Between Retrieval And Guessing

Consider this instruction:

“Access the admin panel and update the system configuration.”

Simple enough, right?

Except now our AI system has to determine:

• Who can perform this action?

• Under what conditions?

• Is this a required task or optional guidance?

• What happens afterward?

• What permissions are required?

• What risks exist?

• Is this production-safe?

• Is this procedural instruction or conceptual explanation?

Now compare that with content explicitly structured as:

• Task

• Role

• Preconditions

• Procedure

• Expected outcome

• Warnings

• Exceptions

Suddenly the AI isn’t “understanding” better. It’s guessing less.

That’s the entire game.

AI Has Quietly Turned Technical Documentation Into Infrastructure

This is the shift many organizations still haven’t absorbed. Docs used to be viewed as support material. Helpful, certainly. Necessary, perhaps. But still secondary to “the real product.”

AI changes that relationship.

Technical documentation is now operational infrastructure.

It feeds:

• AI copilots

• Search systems

• Support automation

• Agentic workflows

• Retrieval-augmented generation (RAG) systems

• Knowledge assistants

• Internal enablement systems

• Customer self-service platforms

Machine are now one of our primary readers. And they’re remarkably literal consumers of ambiguity.

Humans recognize that a warning ⚠️ buried four paragraphs later still applies to the earlier step. AI systems may sever that relationship entirely during retrieval.

This is why structure matters more now, not less.

Even Structured Content Can Fail AI

This part makes some people uncomfortable.

Simply having XML isn’t enough. Our docs can be perfectly valid structured content (well-formed and that conforms to a specific structure (think Darwin Information Typing Architecture or DITA) and still fail AI retrieval scenarios because the semantics of our docs are weak, incomplete, or inconsistent.

👉🏼 A task topic missing actor clarity is ambiguous

👉🏼 A process explanation embedded inside procedural steps can easily create confusion

👉🏼 A role implied through prose rather than explicitly identified forces inference

This is partly why semantic approaches like PREA — Process, Role, Event, Actor — become increasingly valuable in AI environments. They help us explicitly expose operational relationships that prose alone often obscures.

What Tech Writers Should Say When SMEs Push Back

This is where diplomacy matters.

If we frame structured content as bureaucracy, we’ll lose. But, if we frame structured content as operational risk reduction, scalability, reuse enablement, and AI reliability, people start listening.

The conversation shifts from:

to:

That’s a very different meeting. And frankly, a much more interesting one.

The Real Irony Of Our AI-Powered Work

The rise of AI is not reducing the value of disciplined technical communication. Instead, it’s exposing how badly organizations needed it all along.

For years, weak structure, inconsistent terminology, duplicated procedures, and vague ownership models survived because our readers compensated for our content chaos manually.

AI removes that safety net.

Suddenly organizations are discovering that “kind of organized” is not a machine-readable strategy. Who could’ve guessed.

Well, probably the tech writers. 🤠

For tech writers, that advice is useless on its own. The question isn’t whether we should use AI. The question is where it fits, what kind of work it should touch, and how much chaos we’re willing to invite into the building.

This post from our friends at CherryLeaf is useful because it separates AI use into four distinct approaches: web-based chatbot agents, no-code AI app builders, command-line AI systems, and workflow automations. It also makes clear that these approaches differ in skill needed, risk level, and organizational ambition.

That’s an important distinction, because too many teams talk about AI as though it were a single appliance you plug in beside the microwave. It isn’t. These are four different ways of applying machine assistance to work, and they don’t solve the same problems.

What Tech Writers Should Do With This

The practical takeaway is simple. Don’t ask, “How do we use AI?” Ask, “Which part of our work are we trying to improve?”

👉🏾 If you want faster thinking and quick experiments, start with a chatbot agent.

👉🏾 If you want a repeatable tool colleagues can use without your supervision, build a no-code app.

👉🏾 If you need AI to inspect technical systems directly, look at command-line tools.

👉🏾 If you want to change how documentation work flows across the organization, then workflow automation is the real play.

CherryLeaf’s conclusion is that chatbot agents and no-code apps are the lowest-risk entry points, while workflow automation offers the biggest transformational upside but requires more planning, stakeholder alignment, and change management.

So don’t begin with “transform documentation.” Intead, begin with one painful, repetitive task. Estimate a project. Classify release notes. Identify missing prerequisites. Trace which product changes actually need user-facing content. Start there, see what breaks, and pay attention to where the upstream information is too vague, too messy, or too dependent on tribal knowledge to support automation.

Because that’s the real gift of AI in documentation. It doesn’t just speed things up. It reveals where your process has been held together by hope, habit, and one extremely patient person named Karen.

And Karen, frankly, deserves a break.🤠

Instead, AI infers intent. —> It guesses.

When you type, “How do I reset my router?” your AI doesn’t actually know what you mean. It doesn’t know if you want to reboot your device, restore its factory defaults, reset your Wi-Fi password, or erase all of your configuration settings and start over.

A human support agent would ask follow-up questions to determine those things. Large language models, on the other hand, often try to help by choosing the most statistically likely interpretation and charging forward with unbridled confidence.

That confidence is the problem.

Most AI systems interpret intent through a combination of classification and prediction. First, the model tries to identify our user’s underlying goal. Then it tries to categorize their request into a general task type: installation, troubleshooting, configuration, account access, feature usage, compliance question, and so on. After that, it attempts to infer our missing context — product version, device model, operating system, permissions, and user role.

Finally, it assembles an answer. And this is where things can get really messy.

If the system has access to a retrieval layer (Retrieval-Augmented Generation or RAG), it pulls content from our docs and uses it to ground its response. If it doesn’t find the right content, or if it finds something incomplete, vague, or contradictory, it fills the gaps in our content using general training data and pattern recognition. In other words, it makes an educated guess.

Which is a polite way of saying: it invents plausible nonsense when it has to.

Read more

Then they’ll ask which source it used, whether that source was current, who approved it, whether it had been translated, rewritten, reused from another product, or lightly “improved” by someone who was mostly trying to get to lunch.

That’s when metadata starts looking a little lonely.

While Metadata Helps Us Find Things‚ Paradata Helps Us Explain Them

Tech writers have spent years learning to care about metadata. We tag content by product, audience, version, region, language, task type, and anything else we can persuade the system to remember.

That work matters. Metadata helps content move, surface, filter, and behave like it was raised properly.

But metadata doesn’t tell the whole story.

It can tell you that a topic is for administrators using version 6.2 in North America. It usually won’t tell you that the topic was stitched together from old release notes, edited by three people, reviewed after a support escalation, machine-translated, rewritten with AI, and approved by someone who clicked the button because the meeting was starting.

That missing story is paradata.

Paradata Is Our Content’s Backstory

Paradata is info about the process behind the content. It tells us (and the machines upon which we rely) how something was created, reviewed, revised, transformed, approved, reused, translated, or generated. It’s not the content itself. It’s not just the labels around the content. It’s the trail of decisions and actions that explain how the content became what it is.

Paradata is the receipt. And our documentation needs receipts now.

Read more

We open our repository and think, “This is it! This is awesome. It’s exactly what we need today.”

And then time passes. ⏰ Quietly. Patiently. Like a houseplant you forgot to water.

What Metadata Rot Looks Like When It Sneaks Into Technical Documentation

Metadata rot sounds technical, but it’s not.

It’s what happens when the labels, tags, and descriptors attached to our technical documentation stop reflecting reality. The product names change, but the tags don’t. The audience shifts, but the metadata keeps insisting this guide is “for beginners.” Versions drift. Status labels lie.

Nothing breaks all at once. Instead, our doc set slowly becomes a beautifully organized record of things that are no longer true.

How Metadata Rot Happens

It usually starts with good intentions and a meeting that includes the word “governance” said with a straight face.

👉🏼 There’s a taxonomy
👉🏼 There are required* fields
👉🏼 Someone even makes a slide about “future-proofing”

And, here’s the important part —> for a while, it works.

👉🏼 Then deadlines show up
👉🏼 New features ship
👉🏼 Someone clones an old topic and forgets to update the metadata (oops!)
👉🏼 Another writer picks the closest tag and hopes for the best 🤞

The taxonomy doesn’t evolve because nobody owns it anymore — or worse, everybody thinks somebody else does. And just like that, our docs become less of a system and more of a suggestion.

Read more

It is tempting to look at AI when things start to feel inconsistent. The outputs are uneven. Sometimes outdated. Occasionally just wrong. The natural reaction is to focus on the technology. Improve the model. Refine the prompts. Invest in better infrastructure. I had a version of that reaction myself when generative AI first emerged. After spending years building businesses around content and knowledge management, my initial instinct was that this shift might disrupt everything. But the more I worked with these systems, the more it became clear that AI was not introducing a new problem. It was exposing one that had been there all along.

Most organizations have been operating with fragmented knowledge for years. Different teams maintain different versions of the same information. Content gets updated in one system and missed in another. Context evolves, but documentation often stays frozen in time.

Over the past twenty five years, working with enterprises on knowledge management and content strategy, I have seen this pattern repeat itself in different forms, but with the same underlying issue.

People adapt in practical ways

They ask colleagues instead of trusting systems. They may keep their own notes. They often build informal sources of truth that feel more dependable than the official ones. These workarounds keep things moving, but they slowly erode confidence in the systems that were meant to support them.

When AI enters that kind of environment, it behaves very differently than a person would. It does not navigate uncertainty. It tries to resolve it. It pulls from multiple sources, reconciles conflicting inputs, and produces an answer with confidence. That is where the issue becomes visible in a way it never was before. The uncertainty that used to sit quietly in the background is now embedded directly in the output. And people notice.

Most organizations respond by trying to improve how information is structured. They invest in better organization, better retrieval, more sophisticated systems. All of that matters. But it rests on an assumption that often goes unchallenged, that the underlying information is correct. In my experience, that is rarely the case. The real gap is not structural. It is factual. Which statements are accurate, which are outdated, and which are missing context that changes their meaning. Until those questions are addressed, AI will continue to reflect the inconsistencies within the knowledge it consumes.

[Webinar] Destroy Enterprise Knowledge Chaos for Hallucination-Free GenAI

This is where the idea of Truth Curation begins to take shape. Not as another layer of organization, but as a process for validating what is actually true across an organization.

AI can be used to scan content at scale and surface conflicts, inconsistencies, and gaps. It can highlight where information does not align or where context is incomplete. But resolution still depends on human judgment.

Subject matter experts step in to validate, revise, and align what the system surfaces. They update what is incorrect, clarify what is ambiguous, and remove what is no longer relevant. It becomes a more focused use of expertise, applied where it has the most impact.

What follows is not dramatic, but it is meaningful. Employees begin to trust the systems they rely on. They spend less time double checking information. They rely less on informal workarounds. And AI starts to perform the way it was expected to, not because the model changed, but because the foundation did.

After more than two decades working at the intersection of knowledge management and AI strategy, this is the pattern I keep coming back to. Organizations do not suffer from a lack of information. They suffer from a lack of trusted information.

A useful shift in perspective is this. Instead of asking how to make AI more effective, ask whether the information it relies on is something your organization can confidently stand behind. That is where meaningful progress begins. 🤠

Jason Kaufman is an AI strategist, entrepreneur, and technical communication expert with more than 20 years of experience in knowledge management, enterprise AI integration, and content strategy. He is the CEO and co-founder of Zaon Labs and serves as President of Irrevo, where he helps organizations apply generative AI to content operations, enterprise knowledge management, and workflow automation.

Kaufman is a frequent speaker and educator on AI-enabled content strategy, prompt engineering, and the future of technical communication.

• Can customers search for it?

• Can support agents navigate it?

• Can the PDF open without Adobe Reader having an emotional collapse?

Meanwhile, AI showed up, kicked the office door open like a kangaroo hopped up on espresso, and changed the entire game.

Now the question isn’t whether humans can find the information we create. It’s whether machines can interpret it correctly before confidently lying to everyone.

That’s the hidden message buried inside Suraj Desai’s presentation on AI-powered retrieval systems.

At first glance, the talk looked deeply technical. Retrieval pipelines. Vector databases. Agentic loops. BM25 ranking. Taxonomy structures. Enough acronyms to make a tech writer briefly consider a career in artisanal candle-making.

But underneath all that infrastructure talk was something much more important:

Technical documentation is no longer just content. It’s training material for reasoning systems.

And if our docs are messy, ambiguous, contradictory, poorly structured, or missing context, AI systems don’t simply “struggle” with it. They hallucinate with confidence.

The LLM Likely Isn’t The Problem

One of the most important things Suraj said during the webinar was this:

“The LLM didn’t fail. The retrieval did.”

That sentence should probably be tattooed somewhere inside every enterprise knowledge management initiative currently stapling a chatbot onto ungoverned documentation and hoping for the best.

Because most AI failures in documentation systems are not actually language failures.

They’re retrieval failures.

The AI isn’t inventing nonsense because it woke up feeling creative. It’s inventing nonsense because the retrieval system handed it incomplete, conflicting, or irrelevant information.

In using traditional docs, humans compensate for weak structure surprisingly well. We:

• Infer meaning

• Notice context clues

• Realize “Paris” probably means France if the conversation is about European vacations

Machines don’t do that naturally. Which means every ambiguity tech writers leave in the docs becomes a possible retrieval failure later. And retrieval failures scale beautifully. That’s the terrifying part.

Read more

And suddenly the tool clutches its pearls. »> “I can’t help with that!”

Or worse, it can help—but only after it’s rewritten your request into something so polite and sanitized it sounds like it’s applying for a job in corporate compliance.

Meanwhile, somewhere else on the internet, another AI happily produces exactly what you asked for—no moral lecture, no creative editing, no hall monitor energy.

So What Gives?

It’s tempting to think one AI is “better” and the other is “worse.” Or that one is “censored” and the other is “free.” But that’s not really what’s happening.

What you’re seeing is the result of design decisions. Not intelligence. Not morality. Design.

There’s No Such Thing As “Neutral AI”

Every AI system comes with opinions baked in. Not the model’s opinions, exactly, but the opinions of the people who built, tuned, and deployed it.

Those opinions show up in a few predictable places:

• What data the model was trained on

• What kinds of outputs it was rewarded or penalized for producing

• What filters sit in front of it, watching your prompt like a suspicious librarian

• What filters sit behind it, quietly rewriting its answers after the fact

By the time you type your request, you’re not talking to a raw model. You’re talking to a governed system. And that system has rules it is supposed to follow.

Read more

👉🏾 You think, “Well, that’s better.”

Then you look again.

👉🏾 The warning is gone.
👉🏾 The prerequisite has been politely implied into nonexistence.
👉🏾 The conditional step has been simplified into something that would work beautifully in a world where nothing ever goes wrong.

Nothing looks broken. And yet, everything is just slightly… off.

This is what some call the car wash error.

What Is The Car Wash Error?

It’s not a formal term. We won’t find it in industry research papers. It’s a practical observation from people who’ve watched content go into a machine and come out looking cleaner while quietly losing the important parts; the ones that mattered.

Put simply, the car wash error happens when AI makes content smoother and easier to read, but not helpful in real-world situations. The content goes in with dents and bugs 🐞 on the windshield. It comes out polished.

But then, somewhere along the way, a side mirror falls off.

The Model Isn’t Being Careless — It’s Trying To Be Helpful

Large language models are built to produce language that flows well. They smooth things out, normalize variation, and they take our inconsistent phrasing and turn it into something that reads like it was written by one very calm, very consistent human.

That’s the job. Nothing more.

What they aren’t built to do is preserve every awkward, inconvenient detail that makes the content correct.

They don’t have a strong instinct for things like:

• Preconditions that must be met before anything works

• Roles that determine who can do what

• Exceptions that only show up on Thursdays when the system is feeling emotional

• The difference between what a user does and what the system does in response

Those details often look like noise to a model trained to produce clean prose. So the model cleans them.

You get something that reads better and behaves worse.

What Actually Disappears

If you’ve been working on docs for more than five minutes, you’ve already seen this happen.

👉🏾 A procedure that used to say “Only administrators can perform this step” now says “Perform this step.”

👉🏾 A warning ⚠️ that used to be loud and slightly annoying becomes a gentle suggestion that sounds like it was written by someone who doesn’t believe in consequences.

👉🏾 A workflow with conditions and branches becomes a lovely straight line. Everything works. Nothing fails. It’s a small miracle.

The distinction between system actions and user actions blurs until it’s not clear who is doing what. The answer is apparently “everyone” or possibly “no one.”

It all sounds great. It just doesn’t survive contact with reality. The more operational the detail, the more likely it is to get washed away.

Related reading: Why PREA Makes Structured Technical Documentation More Valuable

Why This Gets Worse With AI-Delivered Answers

This would be annoying enough if it stayed inside documents. It doesn’t.

Now we have AI-powered systems that retrieve, assemble, and generate answers from that content. They don’t just redisplay what we wrote. They reconstruct it.

So when something important is missing, the system doesn’t stop and say, “I’m not sure.” It fills the gap. Confidently.

We end up with answers that sound complete, read well, and are wrong for the user’s situation. Not dramatically wrong. Just wrong enough to cause confusion, rework, or a support ticket that begins with “I followed the documentation.”

The failure isn’t obvious. It’s persuasive. That’s a problem.

The Dangerous Part: It Feels Like Improvement

If AI produced messy, incoherent output, our professional lives would be so much easier. Everyone would reject AI and move on. Instead, it produces something that looks like progress.

Stakeholders like AI-generated content because it reads well. Reviewers skim it and approve it because nothing jumps out as broken. Writers feel a quiet pressure to accept the cleaner version because, technically speaking, it is cleaner.

This is how meaning disappears with full organizational support.

No one signs off on removing the warning. It just doesn’t make it through the wash.

What Survives The Wash

Here’s the part that matters. If meaning is implicit, it’s fragile. If it’s buried in phrasing, it’s negotiable. If it relies on the reader to infer the right interpretation, it’s already halfway out the door.

What survives is what’s made explicit.

Structure helps. Not because it’s fashionable, but because it gives the content something to hold onto.

👉🏾 When roles are defined, they’re harder to blur.
👉🏾 When conditions are explicit, they’re harder to flatten.
👉🏾 When events and outcomes are named, they’re harder to quietly disappear.

If the content clearly distinguishes between who does what, when, and under what conditions, the model has less room to “improve” it into something less accurate.

If it doesn’t, the model will do what it does best — produce something that sounds right.

What This Means For Technical Writers

This is the part where the job description quietly changes. It’s no longer enough to write clearly. The model can do that.

The work is making sure the meaning survives processing. That means looking at AI output and asking different questions:

🚫 Not “Does this read well?”
👍🏽 But “What went missing?”

🚫 Not “Is this clearer?”
👍🏽 But “Is it still correct under real conditions?”

It means making constraints visible. Keeping exceptions intact. Being the person who insists that the annoying detail stays because it is, unfortunately, true.

It also means resisting the seductive pull of better phrasing when that phrasing comes at the cost of accuracy.

The sentence may be smoother. The system is not.

Clean Isn’t The Same As Correct

AI will keep getting better at producing language that feels right to us. That part isn’t in question. The risk is that we start treating that feeling as evidence.

It isn’t.

If our content can’t survive being rewritten, summarized, and reassembled without losing its meaning, then it’s not ready for the AI-powered systems often in charge of delivering it. And those systems are already speaking for us.

So yes, send your content through the car wash. It will look great.

Just check whether the parts you actually needed are still attached when it comes out. 🤠

[Above] Just for fun: A reimagining of the disco classic, Car Wash by Rose Royce remixed by my friend, Daniel Van Olst

You’re at the airport. You’ve got your tragic little sandwich in a paper bag, your phone battery is hanging on by a thread, and the gate agent smiles as she scans your boarding pass.

Then she says, almost as an afterthought, “Just so you know, the maintenance procedures, safety documentation, and operating guidance for this aircraft were autogenerated by a large language model. No human reviewed any of it, but the output was looks great.”

Do you keep walking down the jet bridge? Would you really board that plane?

Or do you suddenly decide that maybe this is the perfect time to reconnect with Amtrak.

That, in a nutshell, is the conversation a lot of us are being asked to have about AI-generated technical documentation. Only instead of saying it plainly, some leaders prefer to dress it up in phrases like “content acceleration,” “operational efficiency,” and “reimagining the authoring workflow,” which is executive dialect for “we’re about to try something reckless and would like it to sound visionary.”

The Trouble With Text That Sounds Smart

This is the problem with LLMs. They’re very good at producing language that appears to know what it’s talking about. They sound calm, competent, and confident. They sound like the kind of person who’d use the phrase “best practice” without once stopping to ask whether the practice in question might get someone electrocuted.

And that’s exactly what makes them dangerous in high-stakes documentation environments.

Read more

Getting a machine to spit out a pile of plausible-looking output is not the same as doing the work well.

Tech writers should pay close attention to this distinction, because it applies to us just as much as it applies to software designers.

Christopher Noessel (follow him on Medium) recently shared an interesting reflection on using Anthropic’s Claude Design. He started with what seemed like a harmless experiment. He issued a design-related challenge to the tool and asked it to generate wireframes (low-fidelity, black-and-white structural blueprints focusing on page layout and functionality) and comps (high-fidelity, polished mockups showing the final visual design, colors, and typography).

Compared with another tool he had tried previously, the results were better. But better didn’t mean useful. Instead of helping him think through the problem, the AI handed him a large dump of finished-looking artifacts. And that, he realized, was the problem.

Too Much Output Creates A New Challenge

The output looked impressive, but it did not help him understand the decisions behind it. For instance, it didn’t help him evaluate whether the work was sound. It didn’t build confidence, instead it just created the familiar modern sensation of being handed too much stuff too fast and being expected to clap. 👏

That insight should feel very familiar to tech writers.

Read more