Docs That Update Themselves: Agents for Runbooks and Diagrams¶
The runbook said "restart the auth-service deployment."
We'd renamed it to identity-service eight months before. Nobody updated the runbook — why would they? The rename was a code change, not a docs task. So it sat there, quietly wrong, until a 2am incident when an engineer followed the runbook step by step and spent four minutes baffled before figuring out what had changed.
Four minutes doesn't sound like much. At 2am, in a live incident, it's an eternity.
You know that moment when someone suggests the solution is "better documentation culture"? I want to push back on that. Hard. The problem isn't culture — it's maintenance. Runbooks go stale because keeping them current is a separate manual task that lives in a different system from the code that made them stale. Nobody's going to do that reliably, no matter how much you talk about documentation culture.
The actual fix is to make the docs update when the thing they describe changes. And now you can.
Here's the pattern: code merges trigger an agent that finds the affected docs, proposes the updates as a PR, and keeps your diagrams in sync with the actual current architecture. You review it. You merge it. Done.
Quick takeaways¶
- Documentation drift is a code problem, not a culture problem — stop trying to fix it with reminders and retros
- Doc updates should be triggered by code changes, not by someone's memory; if it relies on memory, it won't happen
- Architecture diagrams should be generated from source, not painted by hand and then gradually become lies
- The update PR is the review gate — the agent proposes, a human verifies, and nothing merges automatically
The documentation drift problem is architectural, not cultural
Documentation goes stale because updating it is a separate manual task that lives in a different system from the code that made it stale. Reminders, retros, and "documentation culture" don't fix architectural problems. Wiring doc updates to merge events does.
What goes stale (and why)¶
Runbooks: full of service names, ports, namespaces, and commands that change with every refactor. The person making the code change doesn't update the runbook because the runbook is in a different repo, or a wiki, or some other system that requires a separate PR and a separate review. So it doesn't happen.
Architecture diagrams: accurate at the moment of drawing, then gradually not. Services get added, merged, split, renamed. The diagram keeps showing the old shape. Eventually it's actively misleading — worse than having no diagram at all.
API references in docs: endpoint paths change, request formats evolve, error codes get added or removed. Docs that reference them go stale silently — no build fails, no test breaks. You only discover the problem when someone tries to follow them.
AGENTS.md files: describe the repo structure and workflows that agents and engineers rely on. When the structure changes — and it always does — AGENTS.md is usually the last thing anyone thinks to update.
1) AGENTS.md as a routing map¶
The whole thing hinges on one question: when a file changes, which docs are affected? Without an answer to that, you're either updating everything (noisy) or guessing (unreliable). AGENTS.md — see Using AGENTS.md for Platform Engineering for the full story — is a natural place to encode exactly this routing:
# AGENTS.md
## Documentation routing
When files in `src/auth/` change:
- Update: `docs/runbooks/auth-service.md`
- Update: `docs/api/authentication.md`
- Regenerate: `docs/diagrams/auth-flow.puml`
When files in `k8s/` change:
- Update: `docs/runbooks/deployment-guide.md`
- Regenerate: `docs/diagrams/platform-architecture.puml`
When `helm/values.yaml` changes:
- Update: `docs/runbooks/configuration-reference.md`
If you want the bigger picture of how AGENTS.md works as a full platform control plane — not just for docs routing — the AGENTS.md for Platform Engineering post covers that in depth.
2) Trigger on merge¶
# .github/workflows/doc-update.yml
name: Documentation Update
on:
push:
branches: [main]
jobs:
check-affected-docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2 # need the previous commit to diff
- name: Get changed files
id: changed
run: |
git diff --name-only HEAD~1 HEAD > /tmp/changed-files.txt
echo "Changed files:"
cat /tmp/changed-files.txt
- name: Identify affected docs
id: docs
run: |
python scripts/docs/find-affected-docs.py \
--changed /tmp/changed-files.txt \
--routing agents/AGENTS.md
- name: Update docs if affected
if: steps.docs.outputs.affected_count > 0
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
AFFECTED_DOCS: ${{ steps.docs.outputs.affected_docs }}
run: python scripts/docs/update-docs.py
For the broader pattern of making all AI-driven changes traceable and reviewable in Git — not just doc updates — see Repo-Native AI Workflows.
3) The doc update agent¶
# scripts/docs/update-docs.py
import anthropic
import subprocess
import json
import os
from pathlib import Path
client = anthropic.Anthropic()
def update_runbook(runbook_path: str, changed_files: list, diff: str) -> str:
"""Update a runbook based on code changes."""
current_content = Path(runbook_path).read_text()
response = client.messages.create(
model="claude-opus-4-6",
max_tokens=4096,
system="""You are a technical documentation updater. You will receive:
1. A runbook's current content
2. A git diff showing what changed in the codebase
3. A list of changed files
Your job is to update the runbook to reflect the code changes. Specifically:
- Update service names if they changed
- Update command examples if the commands changed
- Update port numbers, namespaces, or endpoints if they changed
- Update step sequences if the process changed
Rules:
- Only change what the diff tells you changed
- Do not improve or rewrite sections that are still accurate
- Preserve the existing structure and formatting
- If you are uncertain whether a section needs updating, leave it unchanged and add a comment: <!-- REVIEW: may need updating after <change description> -->
- Output the complete updated runbook content""",
messages=[{
"role": "user",
"content": f"""Current runbook:
{current_content}
Code diff:
{diff}
Changed files:
{json.dumps(changed_files)}
Update the runbook to reflect these changes."""
}]
)
return response.content[0].text
def open_doc_update_pr(updated_docs: dict) -> str:
"""Open a PR with all doc updates."""
branch = f"docs/auto-update-{subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD']).decode().strip()}"
subprocess.run(["git", "checkout", "-b", branch], check=True)
for path, content in updated_docs.items():
Path(path).write_text(content)
subprocess.run(["git", "add", path], check=True)
subprocess.run([
"git", "commit", "-m",
f"docs: auto-update documentation for {subprocess.check_output(['git', 'log', '--format=%s', '-1', 'HEAD~1']).decode().strip()}"
], check=True)
subprocess.run(["git", "push", "origin", branch], check=True)
doc_list = "\n".join(f"- `{path}`" for path in updated_docs.keys())
pr_body = f"""## Automated documentation update
This PR was opened automatically because a merge to `main` affected the following docs:
{doc_list}
**Please review carefully**: the agent updates what the diff tells it to update, but may miss context that is not visible in the code change.
Specifically check:
- Are all command examples still valid?
- Are service names and namespaces correct?
- Are any `<!-- REVIEW: -->` comments flagged by the agent worth addressing?
---
*Opened by doc-update workflow*
"""
result = subprocess.run([
"gh", "pr", "create",
"--title", f"docs: auto-update documentation",
"--body", pr_body,
"--base", "main"
], capture_output=True, text=True, check=True)
return result.stdout.strip()
4) Diagram regeneration with PlantUML¶
For architecture diagrams, the key insight is simple: don't maintain the image, maintain the source. Keep the PlantUML, regenerate the PNG automatically. That way the diagram is always derived from something you can diff and review:
# scripts/docs/update-diagram.py
import anthropic
from pathlib import Path
client = anthropic.Anthropic()
def update_plantuml_diagram(diagram_path: str, diff: str) -> str:
"""Update a PlantUML diagram based on code changes."""
current_diagram = Path(diagram_path).read_text()
response = client.messages.create(
model="claude-opus-4-6",
max_tokens=2048,
system="""You are a PlantUML diagram updater. Update the diagram to reflect code changes.
Rules:
- Only update elements that are directly affected by the diff
- Preserve existing layout decisions and color choices
- If a service was renamed, rename it in the diagram
- If a new service was added that communicates with existing ones, add it
- If a service was removed, remove it and its connections
- Output only the updated PlantUML source, nothing else""",
messages=[{
"role": "user",
"content": f"Current diagram:\n{current_diagram}\n\nCode diff:\n{diff}\n\nUpdate the diagram."
}]
)
return response.content[0].text
The MkDocs build pipeline handles PlantUML → PNG conversion automatically via the build_plantuml plugin, so all the agent needs to do is update the .puml source. That's it.
What the PR looks like¶
One merge, one PR. The agent batches all the affected docs from that merge into a single pull request, with each updated file listed in the body alongside a summary of what changed and why. Engineers review it exactly the same way they'd review a code change — line by line, with a clear diff showing what the agent touched.
This is the key point, and it's worth being really explicit about it: the agent doesn't have final say. It proposes. Humans decide. Nothing auto-merges.
The scope creep tell
The moment an agent starts rewriting docs for clarity instead of just updating them to match the code change, you lose the ability to tell what's a factual update and what's an editorial choice. Scope the agent strictly: "update to match the code change, nothing else." Clarity improvements go in a separate PR, separately reviewed.
Common mistakes¶
Running the update on every commit, not just merges to main. Way too noisy. You'll get a doc update PR for every work-in-progress commit, including ones that get amended or reverted before they ever ship. Wait until code is actually in main — that's when the docs need to reflect it.
Updating docs without showing the diff. The PR body should show exactly what changed, not just "docs updated." If reviewers can't see the specific lines that moved, they can't actually verify the update is correct. The whole safety model depends on that diff being clear.
Trying to improve docs while updating them. This one's subtle but important: scope the agent strictly to "update to match the code change, nothing else." If the doc was confusing or incomplete before the merge, that's a separate problem. Fix it separately. The moment the agent starts rewriting for clarity, you lose the ability to tell what's a factual update and what's an editorial choice — and both need different reviewers.
Apply this: merge triggers first, scheduled triggers later
Start with merge triggers — they're the most reliable signal that docs need updating and produce the least noise. Scheduled triggers (daily/weekly) catch drift without a clean code correlation, but they generate more false positives. Get merge triggers working well before adding scheduled ones.
Frequently asked questions¶
How do you stop runbooks from going stale?
Wire a GitHub Actions workflow to trigger on merge to main. The workflow runs an agent that reads the changed code, reads the current runbook, identifies what's now out of date, and opens a PR with the corrected content. The key thing is that the runbook update happens in the same cycle as the code change — not a week later when someone notices, not never.
Can an AI agent update PlantUML diagrams automatically?
Yes. The agent reads the current .puml source and the changed service or component from the diff, then regenerates the diagram to reflect the change. The updated source goes into a PR for human review — never directly to main. And the PNG is never committed by hand; it's built in CI. That's the point: you maintain the source, not the image.
What triggers should kick off a doc update agent?
The most reliable trigger is a merge to main that touches files the docs cover. That's the one to get right first. Scheduled triggers — daily or weekly — can catch drift that happened without a clean code correlation, but they generate more false positives and can become background noise if the signal-to-noise ratio is bad. Start with merge triggers.
How do you prevent an agent from making incorrect documentation changes?
The main thing: every doc update goes through a PR. No direct commits. The PR body should include a diff and a brief explanation of what the agent changed and why. For runbooks specifically, add a short checklist asking a human to verify the steps still actually work. That friction is appropriate — it's the review gate that makes the whole pattern safe to run.
What you get¶
- Service renames, namespace changes, and command updates propagate to the docs automatically — the next on-call engineer finds the right name, not the old one
- Every doc update is a PR with a clear reason attached; nothing changes silently
- Architecture diagrams stay close to the actual current architecture without anyone having to remember to maintain them
- Engineers spend their incident time actually resolving things, not puzzling over whether the runbook is still accurate
Walkthrough files¶
agents/AGENTS.md— documentation routing map (which docs to update when which files change).github/workflows/doc-update.yml— merge trigger → affected docs → update PRsscripts/docs/find-affected-docs.py— parse AGENTS.md routing to find affected filesscripts/docs/update-docs.py— AI-powered runbook update with targeted changesscripts/docs/update-diagram.py— PlantUML diagram update from code diff
Want all AI-driven changes in your repo — not just doc updates — to be traceable, reviewable, and auditable? That's the whole point of Repo-Native AI Workflows.