A Short Story About an LLM Wiki Developing a Tumor
The documentation grew seven times in six weeks. That wasn’t the problem. Here’s what was, how I found it, and what it cost to fix.
How it even happened
I keep a decision archive for Archangel as an LLM wiki: Open Knowledge Format, one page per ADR or concept, an index.md at the top that tells an agent where to look. It had skills, rules and checks in place.
In six weeks it went from 30k tokens to 204k. This made sense as the project was in an early, high-entropy phase and new ADRs were landing daily. The documentation was correctly maintained, so I stopped doing in-depth reviews of changes for the next month.
I have bad news and good news for LLM-wiki practitioners.
The bad: something was going wrong under the radar during that month, and it passed every check I had. Every edit looked legitimate. Every page, read on its own, was true. I only noticed when an agent that should have answered a question from one page came back having read half the corpus, and then, on a different question, came back fast and confident with an answer that turned out to be mostly missing. We lost the focus.
The good: there are a few ways to avoid making this hell on earth, and the fix is cheaper than it sounds. But before fixing anything I wanted to know what had actually happened. Not what I suspected. What git says.
What git actually says
Mechanically, how did the corpus get to 204k? Let’s replay every commit that touched docs/, count tokens at each point, and see.
Method: 56 commits between 2026-07-13 and 2026-08-24. For each commit, git show <sha>:<path> for every docs/**/*.md file live at that point, tokenized with tiktoken’s cl100k_base. No count in this article is estimated.
Figure 1. Every commit touching docs/, in order. The four largest jumps are bursts of new ADRs landing together. The last two points are the cleanup.
The corpus grew 7.4x, and it didn’t grow smoothly. It climbed in steps: +31.6k for the phase-1 ADRs, +23.6k for branch lifecycle, +16.1k for the knowledge base, +23.4k for version control, separated by long flat stretches of ordinary editing. That shape already argues against the “documentation slowly bloats” story. Documentation arrives in lumps when design work happens in lumps.
Then the decomposition, which is where the story actually turns. Every one of the 17 files present at the first commit was still present at the peak, so the +177,013 tokens split cleanly into two kinds:
| Where the growth went | Tokens | Share |
|---|---|---|
| 56 new files: new ADRs, concepts, decisions | 166,998 | 94.3% |
| In-place growth of the original 17 files | 10,015 | 5.7% |
Ninety-four percent of the growth was new pages for a project whose scope actually expanded: a knowledge base, a review flow, session staging, an orchestrator, financial analyses. That’s a decision archive doing its job. If you had shown me only that row, I’d have said there’s nothing to fix.
The 5.7% is where the story is. And 69% of that sits in a single file: docs/index.md.
Two files that quietly changed jobs
index.md: a navigation page that became a changelog
index.md exists to point somewhere else. Over 21 commits it also accumulated a “Notes on this migration” section, and almost every time anything in the corpus changed, another dated paragraph got appended to it. Never trimmed. Never moved. 933 tokens in the first commit; 7,860 at the peak.
Figure 2. index.md, 76 to 400 lines. Of the 20 pre-cleanup commits in this series, every single one either adds lines or leaves the count unchanged. It never once goes down.
So a page whose job is to list pages had become the place where every edit got a dated paragraph. The mechanism is boring, which is why it worked: there was no dedicated place for dated narrative to go, so it went to the nearest file that everyone opened anyway. And because every page in the corpus is one hop from index.md, every retrieval started by reading a changelog.
ADR-0024: a decision record that became an implementation diary
ADR-0024 decides Thread Chat’s data model. Between 2026-07-17 and 2026-08-22, across eight commits, each implementation milestone appended another dated “Status update” blockquote, on top of the original decision, never folded into it, never replacing the text it superseded.
Figure 3. ADR-0024, 300 to 915 lines over ten commits. At the peak, roughly two-thirds of the file was dated implementation narrative that had nothing to do with the decision a reader opens an ADR to find.
Three times its original length, two-thirds of it diary. Same mechanism, different host. There was no page for “how Thread Chat actually works as built,” so that content attached itself to the decision that spawned it, and grew.
I want to be precise about the failure, because “docs drift” is too vague to act on. Nothing here was false. Everything was useful. Both files looked legitimate on any given day, which is exactly why a month of skipped deep reviews was enough. The failure was that content landed in the nearest file rather than the right file, and nothing in the workflow ever removed anything. A page was being treated as a log of everything ever said about a decision, instead of a statement of what’s true about it right now. Do that for six weeks at the top of every navigation path and you get a corpus that is mostly fine and increasingly unusable: updates buried in trailing sections, so the agent has to pull the whole document to find them, which kills the point of having a wiki at all.
The Living Baseline Method
The fix has three parts, each aimed at one of the failure shapes above. I applied it once, corpus-wide, across 82 pages.
Standardize the frontmatter. Every page gets real OKF v0.2 fields, type, description, status, generated, plus the project’s own fields kept on as producer extensions (kind, okf_status, related, owners). The point is that a page’s trust level and lifecycle become machine-legible rather than implied by prose. This lever is close to free, and it’s what makes the other two measurable.
Compress to current truth. Resolved follow-up items get deleted outright, not struck through. Dated “as built” narrative folds into the section it amends and disappears as its own paragraph. Provenance survives in one place, a new log.md, so nothing is lost; it’s just no longer sitting in the page a reader opens to find out what’s true today. One rule governs the whole pass: an item leaves Open Follow-ups only once its resolution is confirmed present in the page body. Never delete-and-assume.
Split along real seams. Oversized pages get divided where their own headings already implied a boundary: named domains, independent sub-decisions. Never invented. Each new page earns its own frontmatter and its own row in index.md, so an agent can find, and stop at, exactly the part it needed. A big page that’s uniformly dense is left alone; splitting it moves weight around without reducing it.
The process ran in five passes: frontmatter batches, fold in resolved items, split oversized pages, migrate citations, verify and measure. The 73-page frontmatter conversion was delegated as eight parallel batches over disjoint file sets, each given the identical mapping table and the identical fold-in rule, so the corpus converged on one convention in one sitting instead of drifting batch to batch.
What it did to the two files:
| File | Peak | After | Where the rest went |
|---|---|---|---|
docs/index.md |
400 lines | 140 lines | Dated narrative moved to a new, single-purpose docs/log.md |
| ADR-0024 | 915 lines | 352 lines | Diary extracted to thread-chat-mechanics.md; resolved follow-ups deleted |
And what it did to the total: 204,483 to 205,490 tokens. The cleanup added a thousand tokens. Content cut from oversized pages mostly reappeared as properly scoped companion pages with their own frontmatter and index rows. If token count were the metric, this pass failed. Which is the point of the next section.
Does it actually retrieve better?
Token count and retrieval quality are different axes. Compressing prose can shrink a corpus while making individual answers harder to find, if it removes the structure a retrieval pass leans on. So I tested retrieval directly rather than trusting the count.
Methodology
Five realistic questions, each run against two real states of the same docs tree: the post-cleanup HEAD, and development/threads, the branch as it stood before the cleanup. Retrieval was simulated the way an agent actually works this repo: start at docs/index.md, match candidate pages by title and tag, open only what the index points to, fall back to full-text grep when the index doesn’t help. Every file was actually read via git show <branch>:<path>; every word count is wc -w on that content.
Scope caveat, stated once: index-plus-grep is a proxy for the project’s planned retrieval design (BM25S lexical plus vector search over page-level chunks), not a run of it. That’s the correct proxy for what changed here, though: none of the difference between the branches comes from indexing technology. All of it comes from how the documents are organized.
The five tests
| Test | Question | What it checks |
|---|---|---|
| 1. Thread Chat mechanics | Explain Thread Chat: its mechanics, its evaluation harness, and the propose/accept write path. | Retrieval cost on a broad but distinct sub-topic |
| 2. Context-retrieval domains | Describe the context-retrieval architecture; what does Domain B (document retrieval) actually do? | Retrieval cost on a narrow question inside a broad topic |
| 3. Confidence-scoring gaps | What’s still open about the confidence-scoring model? | Completeness: does the agent find every open item, or only the obvious ones? |
| 4. ADR-0037 numbering collision | Two files were both drafted as ADR-0037; what was the collision, and how was it resolved? | Signal-to-noise on a provenance question answerable only from history content |
| 5. Branch lifecycle and rollout status | Walk through the branch-lifecycle/promotion flow, including how far the session-staging (ADR-0042) rollout has actually gotten. | Signal-to-noise on a status question mixed with a stable design doc |
I’m folding the results for tests 4 and 5 together below: they turn out to demonstrate the exact same failure mode, so it’s more useful to read them as one finding than two.
Results: completeness matters more than speed
Test 3 is the one that matters most. On HEAD, the first stop is open-issues-register.md, a page that didn’t exist before the cleanup. It lists all six open items: the riskFactors weighting formula, historical-calibration input design, per-op vs. per-ChangeRequest granularity, the three-zone confidence bands, static vs. calibrated threshold ownership, and auditability of the score rationale. One file, 6/6.
On development/threads, the first stop is ADR-0006, the obviously relevant decision. It names two of the six. Reaching the other four requires exhaustive search across pages the agent has no reason to open, and nothing on the page signals that they exist.
This is the sharpest finding in the set, and it isn’t about speed. An agent that stops at the first plausible file on the older tree doesn’t work harder for its answer. It returns a confident answer that is two-thirds wrong, and it has no way to know. A single open-items register converts that from an unbounded search problem into a one-file lookup. It’s the highest-leverage page the entire pass added, and it cost about 2,200 words.
Results: splitting pays for itself on narrow questions
Test 1 asked for Thread Chat’s mechanics, evaluation harness and propose/accept write path. On HEAD: two dedicated index rows, two files, 3,210 words, one hop. On threads: no index signal for “mechanics,” a grep fan-out into the 7,892-word ADR-0024, and the answer buried in changelog prose. 9,717 words to the full picture, about 3.0x the reading, and only because the fallback grep guessed right.
Test 2 asked a narrow question inside a broad topic: what does Domain B of the context-retrieval architecture actually do? On HEAD the topic is one overview plus four domain pages, and this question needs two of them: 2,075 words. On threads it’s one 7,432-word file with no way to stop early: about 3.6x the reading.
Both tests have the same shape. A few hundred extra total words, bought by page-splitting, paid back many times over any time the question doesn’t need the whole topic. This is the empirical version of “the cleanup added tokens and got cheaper.”
Results: same words, different neighbours
Tests 4 and 5 asked provenance and status questions: a numbering collision between two drafts both labelled ADR-0037, and how far the session-staging rollout had actually gotten. Both are answerable only from dated history content, and on both branches that content is word-for-word the same: about 510 words for the collision, 440 words for the rollout status.
What differs is the host. On HEAD the entries sit adjacent, near the top of log.md, a file that is 100% this kind of content. On threads the same entries are interleaved mid-file inside index.md, a 3,147-word page whose primary job is navigation, and the hits aren’t contiguous. Grepping a single-purpose log is a clean win; grepping a file that also has to serve as the front door dilutes every read with material unrelated to the question.
Not a word-count story. A signal-to-noise story. And a small bonus from the same pass: ADR-0025’s decision text shrank 12% (3,077 to 2,703 words) with no content loss, purely from deleting leftover strikethrough.
| Test | Result |
|---|---|
| 1. Thread Chat mechanics | 3.0x reading on the old tree |
| 2. Context-retrieval Domain B | 3.6x reading; no early stop possible |
| 3. Confidence-scoring open items | 6/6 vs 2/6 surfaced |
| 4 + 5. Numbering collision and rollout status | Same words, worse isolation |
Where it needed a human hand
This wasn’t a mechanical pass, and pretending otherwise would be the kind of reasoning I started GoodArgument to complain about.
Delegated batches hit real limits mid-run. Two of the parallel compression batches were cut off partway by an account-level session limit. Recovery meant grepping for leftover strikethrough and re-reading the affected pages in full, including one worked-example scenario whose later steps still described a mechanism an earlier edit had already removed. Only a full read caught it. A grep wouldn’t have.
A convention document has to agree with itself. An early draft of the frontmatter mapping stated one field value in its prose and a different, contradictory value in its own worked example. Most batches followed the prose; a few followed the example. Both are reasonable behaviours given a contradictory instruction. The fix was catching the contradiction and correcting the source, not blaming the delegates that picked the wrong half.
Splitting a widely cited page is real work. Seventeen pages cited specific sections of the largest remaining file before it was split. Every one of those citations needed individual judgment: did its target move, get deleted as superseded, or was it always an informal pointer that’s safe to drop? Then a link-integrity pass to confirm nothing still pointed at a section that no longer existed there. This cost is exactly what keeps splitting a selective tool rather than a default. Measure citation fan-in before deciding: a page with few inbound links and real seams is nearly free to split; one with many is a cost that needs to be worth paying.
What I’d tell you to take home
Standardize frontmatter on any living decision archive, regardless of size. It’s the cheapest lever and it’s what makes the others measurable.
Compress continuously, not in one overdue pass. An item leaves the open list when its resolution is confirmed in the body. If the team can sustain that as follow-ups resolve, the trailing sections never form.
Split only pages that are both large and already domain-shaped. Check citation fan-in first.
Give dated narrative a home. An open-items register and a single-purpose log are two pages that between them removed the two worst failure modes I found. Neither is clever.
And measure retrieval cost directly. Don’t trust token count as a proxy for it. This pass’s own numbers moved in opposite directions on those two metrics more than once, and if I’d optimised for the one that’s easy to compute, I’d have declared the cleanup a failure and kept the changelog in the front door.
The corpus is a thousand tokens heavier than before I touched it. Nobody has missed them.
Archangel is a git-native architecture governance platform where the decision archive is the product’s source of truth, which is why a solo founder ended up caring this much about whether an agent can find the current answer in it. All numbers in this piece are reproducible with git show and tiktoken against the repository’s real history, commits 8340778 through bcb69e8.