New Book Explores How API Documentation Influences AI Code Generation
Information architecture, content structure, and documentation format may play a larger role in AI-generated code quality than previously understood
A new book by Ed Grzetich asks a question that deserves attention from anyone who creates API documentation:
Does the format of documentation influence the quality of AI-generated code?
In Tokens Not Jokin’: How API Documentation Format Affects AI Code Generation, Grzetich reports findings from more than 21,000 integration tests examining how AI systems perform when working with APIs documented in different formats. The book compares approaches including OpenAPI, YAML, Markdown, and others, while also exploring token usage, testing methodologies, and AI evaluation techniques.
One finding immediately stands out. Documentation format appears to have a substantial impact on code generation outcomes, potentially greater than the choice of AI model itself.
Does API Documentation Format Affect AI Code Generation?
Yes, according to Grzetich’s research. The book presents evidence that documentation structure can significantly influence how effectively AI systems interpret APIs and generate working code. The findings suggest the way information is organized may matter as much as, or more than, the model consuming it.
Why Should Documentation Teams Care?
For years, documentation professionals have focused on helping human readers understand products and services. AI coding assistants have introduced a second audience.
Today, docs are often consumed by both software developers and AI systems. The same API reference that helps a developer build an integration may also be used by an AI assistant to generate code, explain implementation steps, or answer technical questions.
When documentation contains ambiguity, inconsistent terminology, or incomplete examples, AI systems consume those weaknesses along with everything else.
What Topics Does Tokens Not Jokin’ Cover?
The book examines:
API documentation formats
AI code generation performance
Token consumption and efficiency
Documentation designed for machine consumption
Evaluation and testing methodologies
AI acceptance testing
Rather than relying on opinion or isolated examples, the book focuses on measurement and repeatable testing. That approach makes it useful even for readers who remain skeptical of some of the conclusions.
What Does This Mean For The Future Of API Documentation?
The rise of AI-assisted development is changing how docs are used. Our technical documentation is no longer only a resource people consult when they need help. It’s increasingly becoming source material used by AI systems to generate answers, recommendations, explanations, and code.
As a result, familiar documentation concerns take on new importance. Information architecture, content structure, metadata, terminology management, examples, and consistency all become factors that may influence machine-generated outcomes.
Many documentation teams have been advocating for these practices for years. AI may provide new evidence supporting their value.
Should You Read This Book?
If your work involves API documentation, developer experience, structured content, content engineering, or AI-assisted software development, the answer is probably yes.
The book raises important questions that many organizations haven’t yet begun to think about asking. It also provides data that can help move our conversations beyond speculation about prompts and models and toward the information those systems consume and rely upon.
Whether the research ultimately reshapes industry practices remains to be seen. What seems harder to dispute is the underlying premise: documentation is becoming part of the AI development stack.
Understanding how AI systems interact with our docs is no longer a niche concern. It’s becoming part of the job. Our job.