I’ve spent a lot of time telling anyone who’ll listen that AI readiness is a documentation problem wearing an AI Halloween 🎃 costume. While I was researching how organizations measure AI readiness, I came across Microsoft’s AI Readiness Assessment.
It attempts to evaluate organizational preparedness across seven pillars. The strange thing is, technical documentation isn’t one of them. That sent me looking for where, exactly, technical documentation should fit into Microsoft’s model, and whether an organization can call itself AI-ready while the documentation feeding its answer engine isn’t.
What Microsoft Means By AI Readiness
Microsoft’s AI Readiness Assessment evaluates whether an organization has the strategy, governance, data, people, infrastructure, and model-management capabilities to adopt AI securely and at scale.
It’s built around seven pillars:
Business Strategy
AI Governance & Security
Data Foundations
AI Strategy & Experience
Organization & Culture
Infrastructure for AI
Model Management
The assessment runs on multiple-choice / multiple-response questions. Completing it produces an overall score, category-level recommendations, and links to related guidance. Signed-in participants can export their results to a CSV file.
This assessment approach seems like a reasonable way to size up organizational readiness. Companies looking to put AI to work need leadership support, real safeguards, usable data, decent infrastructure, and people who know WTF they’re doing.
But there’s a question hiding inside those seven pillars that Microsoft never asks directly:
What knowledge is the AI actually going to use when someone asks a question (and expects an accurate answer) about how our products work?
Where’s Documentation In This Picture?
Documentation (or content, in general) isn’t named anywhere in the public list of pillars the assessment attempts to cover. That’s not proof it ignores content (and, more specifically, documentation). The assessment questions aren’t available to see until and unless you take the assessment. That’s a problem for tech writers looking for themselves in a framework that wasn’t written with them in mind.
My first guess was I’d find questions about content in the Data Foundations section. It’s the pillar built for exactly this kind of question: is the information accurate, current, complete? That seemed like the obvious home right up until I started mapping documentation against the other six pillars one at a time.
👉🏾 Governance asked which sources counted as authoritative.
👉🏾 Infrastructure asked whether the content could be crawled and chunked.
👉🏾 Model Management asked whether the generated answers could be traced back to a source at all.
By the time I’d worked through all seven pillars one-by-one, I stopped looking for documentation’s one home. It touches nearly every pillar Microsoft lists, for the same reason it kept surfacing in Governance, Infrastructure, and Model Management: sooner or later, each one has to support an AI system answering a product, support, or procedural question.
Once documentation enters the picture, an honest AI-readiness review has to cover knowledge management and publishing right alongside technology adoption. The dependencies sit there whether or not our doc team ever asks to be included.
What The Assessment Actually Asks
The public overview page describes this as one assessment spanning seven pillars, and that much holds up. What it doesn’t tell you is that participants choose which pillars to include before they start. Assess Business Strategy alone, and the results page shows one category and one score. Assess all seven, and it rolls every category into a single overall result, with the individual breakdown sitting alongside it.
I ran mine on Business Strategy alone first, to see how it actually treats the kind of knowledge documentation teams manage. It asked three questions: how well-defined the organization’s investment plan for AI is, how far along the organization is in identifying and prioritizing AI use cases, and how thoroughly security gets built into the AI lifecycle. I answered honestly.
The result: Moderate, 2 out of 5.
Microsoft’s recommendations followed the same shape as the questions. Fund generative AI with a real plan instead of treating it as ad hoc. Manage the human side of the rollout: tell people what’s happening, address concerns directly, and keep measuring how the deployed AI performs once it’s live. Write an actual security framework that covers the whole AI lifecycle, from the data going in to the monitoring after launch, with named owners attached to it.
Three recommendations. Not one mentions technical documentation, content, or product knowledge of any kind, even though each of those three questions was really asking whether the organization understood what it was doing before it did it.
I later walked a client through all seven pillars: a consumer goods manufacturer whose product documentation already gets pulled into AI systems that answer customer questions about how their products work.
Their overall result: Critical, 1 out of 5.
Business Strategy and Data Foundations was rated Moderate. Organization and Culture, AI Strategy and Experience, AI Governance and Security, Infrastructure for AI, and Model Management all came back Critical.
Two of those last five are worth a closer look. Question after question in Infrastructure for AI and Model Management assumes the organization runs its AI workloads on Azure OpenAI (a cloud platform from Microsoft that gives businesses access to OpenAI models), right down to Azure-specific terms like PTU (precision throughput unit) and PAYGO (short for pay-as-you-go) provisioning, used without explanation.
That’s platform-specific knowledge, not general AI literacy, and it’s not something most tech writers have, including the ones who might reasonably be asked to fill out this assessment on the documentation team’s behalf. A writer representing doc’s interests here will likely need help completing two of the seven pillars, and the help has to come from people who actually run the infrastructure.
Picture a documentation manager asked to complete this alone, because tech docs raised its hand about AI readiness in the first place. They can answer Business Strategy honestly. They can make a real case for Data Foundations. Then Infrastructure for AI and Model Management put provisioning models and deployment pipelines in front of them, and the honest answer to several questions is that they don’t know, and they’re — probably — certainly — definitely — not the right person to ask.
To be clear, the word documentation does show up here and there in the Microsoft assessment. It never means what this article means by it. In one place it’s an afterthought tacked onto a question about deployment rollback. In another, it’s asking whether our data science team writes up our own exploratory analysis. Neither question is referring to the user-facing technical documentation and product information that ends up as source content for a generative AI system, the kind an answer engine retrieves from and gets graded against.
“Data Foundations” Isn’t The Same As “Knowledge Foundations”
When most organizations talk about data foundations, they mean databases, customer records, telemetry, transactions, and whatever other stuff lives in our enterprise applications. And, while all of this other stuff matters for important reasons, absolutely none of it tells an AI system how our products actually work.
Every department seems to believe its information is the information the AI needs.
👉🏾 Ask security and they’ll tell you it’s access logs.
👉🏾 Ask sales and it’s the Customer Relationship Management (CRM) system data.
👉🏾 Ask nobody, ever, about documentation, until the AI gets a procedure wrong in front of a customer and everyone suddenly remembers writers exist.
The knowledge users needs to be successful with our products lives in our product documentation, API references, support articles, release notes, training materials, implementation guides, user assistance, tickets, and in the heads of experienced co-workers who never got around to writing any of it down.
Duplication and contradiction are common enough in most documentation sets that we can’t assume any single copy is the authoritative one.
Our documentation carries a specific kind of organizational knowledge that an AI answer engine needs before it can respond to a product question reliably:
what the user is trying to accomplish
which role is allowed to perform the action
what has to already be true before the action starts
which person, system, service, or component actually performs each step
what event starts, interrupts, or completes the process
which conditions and exceptions change the expected result
A company can have beautifully governed data and mature AI infrastructure and still hand its answer engine none of the required knowledge specifics. When this happens we learn that we’ve not given the system enough specificity to go on.
Why “It’s Documented” Doesn’t Mean “It’s Explicit”
Every instruction that leaves something unstated hands the reader an inference burden. People handle that burden by drawing on experience, poking around the interface, asking a colleague, or just trying the procedure and seeing what breaks. An AI system doesn’t have any of those options.
Consider this line you’ve probably written some version of yourself:
Select Sync to upload the configuration.
That sentence names an action and implies a result. It doesn’t say who’s allowed to select Sync, whether the project has to be connected first, whether validation runs before the upload, what actually gets uploaded, or what the user should do if the sync fails.
Ask an answer engine, “Can a project viewer sync an offline configuration?” and it has to resolve every one of those gaps somehow.
It might find another passage with the missing facts. It might say it doesn’t know. Or it might quietly assemble a confident-sounding (plausible) answer out of patterns it learned elsewhere, and you won’t be able to tell which one happened just by reading the response.
I’ve started running procedural content through a simple filter I’m calling TRACE:
Tasks — what action has to be completed?
Roles — what permissions or responsibilities does it require?
Actors — which person, system, service, or component performs it?
Conditions — what prerequisite, state, flag, or time constraint changes the outcome?
Events — what starts, interrupts, completes, or changes the process?
TRACE is a way of looking underneath a polished-sounding procedure to find the context a person, or a machine, would otherwise have to guess at.
Microsoft’s assessment tells you whether your organization has the capability to run an AI initiative.
A TRACE pass tells you whether your source material has enough explicit knowledge to support one.
Structured Doesn’t Mean Machine-Readable
“Machine-readable” sounds like a file-format problem, and it’s tempting to treat it that way. XML, structured authoring, schema markup, and content APIs all help machines locate and process information. None of that guarantees the meaning is complete or unambiguous once they get there.
Content like that can be perfectly accessible to a retrieval system and still be a mess to interpret: inconsistent product names, several procedures crammed under one vague heading, missing applicability metadata, prerequisites buried in the intro paragraph, or an exception described on a different page with no link connecting the two.
Documentation earns its keep as grounding material along three lines.
It needs clear naming and structure: descriptive titles and headings, controlled terminology, and procedures, explanations, examples, and warnings kept visibly separate rather than blended together.
It needs accurate provenance: product, version, audience, and lifecycle metadata, plus ownership and freshness information, so a system can tell current instructions from retired ones.
And it needs to survive being taken apart: stable identifiers and canonical sources, dependable content APIs or accessible rendered output, and structures that still make sense once a retrieval system chops the page into chunks.
This is part of why I’ve stopped thinking about content reuse in terms of paragraphs and started thinking about it in terms of knowledge assets. A prerequisite, a definition, a relationship, an exception each has value beyond the single page where it happens to live. Treat it as an identifiable asset, and you can structure it, model it, govern it, reuse it, connect it to related knowledge, and hand it to whatever output, human or machine, needs it next.
Can Better Retrieval Just Fix This For You?
Retrieval-augmented generation gets pitched as the fix: retrieve information relevant to the question, hand it to the model as context, tell the model to base its answer on that context.
Microsoft’s own RAG prompt-engineering guidance backs this up, and it specifically recommends telling the model what to do when the retrieved context comes up short. It also warns that pulling too few chunks omits information, while pulling too many buries the model in noise.
Those are retrieval problems and context-engineering problems. There’s a separate problem underneath them: retrieval might grab exactly the right passage and still come up short, because the information needed to answer the question was never in the documentation set to begin with. No amount of better embeddings, bigger context windows, answer engine optimization, or more forceful prompting recovers a condition our documentation fails to address.
This distinction matters when you’re evaluating output. Microsoft recommends scoring RAG responses on groundedness, completeness, utilization, relevancy, and correctness. A bad answer might mean retrieval grabbed the wrong content, or that the model misused good content. It might also mean the documentation never contained a defensible answer, and no amount of tuning the model or the retriever will change that.
Documentation pros need a seat at that diagnosis table. Otherwise, everyone keeps adjusting knobs on the model and the retriever while the actual missing information stays —> missing.
Two Kinds Of Readiness That Mature At Different Speeds
Whether our organization can select, deploy, secure, govern, and improve AI systems is what the seven pillars attempts to measure. Whether our systems have enough authoritative, appropriately structured, and semantically-optimized knowledge to answer questions correctly is a separate issue, and nothing says those things mature at the same rate.
Executive support, a secure platform, sophisticated models, and a real AI evaluation don’t guarantee our documentation is free of contradictory terminology, abandoned pages, missing conditions, and procedures that only make sense to people who already know how the product is supposed to work. 🙄
The reverse happens too: our documentation team can maintain clean, structured, well-governed content while the rest of our organization is a hot mess with no AI strategy, real security controls, and no infrastructure to use any of it.
Keeping the two forms of readiness separate settles a practical question for us. Nobody’s asking tech writers to become model engineers. But we are being asked to make our organization’s product knowledge explicit, governed, retrievable, and testable, which, if you’ve been doing this job for more than a week, is not exactly a new assignment.
Putting Microsoft’s Assessment To Work
My advice: Take the assessment, but don’t take it alone.
Pull in colleagues from product, engineering, support, security, data, and whoever owns the AI experience, and compare notes.
Questions about Infrastructure for AI and Model Management especially need someone who actually knows our organization’s AI platform. If different people from our organization answer the same question differently, that’s evidence we’ve an ownership or coordination problem worth chasing down.
For every pillar, add a content-specific follow-up:
Which sources will ground the AI experience you’re building?
Who decides which source counts as authoritative?
Can the system tell current instructions from obsolete ones?
Does retrieval preserve the permissions attached to the original content?
Does the documentation state the prerequisites and exceptions a real answer would need?
Are the step-by-step procedures sufficiently explicit to allow AI answer engines to generate accurate responses?
Which customer questions will you use to test it?
Can you trace every consequential instruction in an answer back to its source?
What is the system supposed to do when the source doesn’t have enough to go on?
Treat the assessment results as one input to a documentation-readiness review, not proof that your content already passed one.
Try This On Your Own Docs
Start with one real question. Pick something customers ask constantly, or something where a wrong answer would cost money, time, or trust.
Identify the documentation an answer engine would need to retrieve for it. Write down every claim and procedural step a correct answer would require, then trace each one back to current, authoritative source content.
Flag any task, role, actor, condition, event, or exception that a person, or an AI, would have to infer rather than read. Ask a few AI answer engines, or your own organization’s RAG application, the same question. Compare what they say against your documentation and against what the product actually does.
Write down every unsupported addition, missing qualification, conflicting term, and wrong sequence you find. Fix the source content. Run the test again.
Every consequential part of a generated answer should trace back to current, authoritative knowledge. When the source can’t support a complete answer, the system should say so instead of quietly finishing the procedure from probability.
What The Assessment Doesn’t Ask
Microsoft’s assessment answers a real question: Does our organization have the strategy, governance, data, people, infrastructure, and models to adopt AI responsibly. The assessment was not built to answer the question that actually determines whether those systems produce trustworthy answers: whether the knowledge behind them is explicit enough for a machine to use.
An organization can score well on all seven pillars while its documentation still leaves conditions, exceptions, and prerequisites unstated. When that happens, the answer engine has two choices:
It can say what it doesn’t know, or
It can fill the gap the same way a language model always fills a gap once explicit instructions run out: with probability.
Which one happens isn’t decided by the assessment. It’s decided by whatever your documentation team did or didn’t write down. 🤠