bedvibe.studio

BedVibe Studios — Engineering

The Canon Is the Database

A fictional universe outgrew my context window. So I stopped treating it as a pile of documents and started treating it as a persistent, machine-readable knowledge system that generation has to be validated against.

This is about a small problem that sits underneath a fashionable one. Not building a model of the physical world — the far more modest question of what it takes for a generative model to keep a single invented world straight across a corpus that has outgrown any context window.

The thesis is narrow, and I want to state it before defending it:

A large fictional universe can be treated as a persistent, machine-readable knowledge system rather than a pile of documents. When a model generates new material, it should retrieve the canonical state relevant to that moment, generate under those constraints, and have the result validated against the existing canon before anything is accepted.

Nothing here is a world model in the embodied sense. No physics, no learned dynamics. The analogy is limited and specific: both problems are about grounding generation in a persistent representation of a world rather than in the model's parameters. That is where the resemblance ends, and I will not stretch it further.

Why this became necessary

I write a long-running dark fantasy saga and produce it as dramatized audio. Five novels in, the corpus crossed a threshold that changed the nature of the problem.

Early on, a language model could hold enough of the world in a conversation to be useful. Later it could not — and the failure mode was not the obvious one. The problem was rarely "find the paragraph where this was established." It was that a single new scene can depend, simultaneously, on:

That last constraint is the interesting one, because it is not a retrieval problem at all. It is a state problem. The correct answer depends on where in the timeline you are standing.

Long-context models do not dissolve this. Liu et al. documented that performance degrades substantially when relevant information sits in the middle of a long input, with a characteristic U-shaped curve favouring the beginning and the end — reproduced across models with nominally large windows. Enlarging the window does not guarantee that the material inside it is used. In my own work that matched what I saw: pasting more of the corpus in did not reliably reduce contradictions, and it made every generation slower and more expensive.

The distinction this problem already has a name for

Narratology got here first, about a century ago. The fabula is the underlying chronological sequence of events; the syuzhet is the order in which the text presents them. Any narrative with flashbacks, parallel timelines or multiple points of view separates the two deliberately.

The engineering restatement is simply this: the fabula is the state, and the syuzhet is the observation sequence. A reader reconstructs the first from the second automatically. A retrieval system does not, because nothing in a passage's embedding tells you where it sits on a timeline that is never written down.

Four operations usually collapsed into one

Most discussion of retrieval-augmented generation treats retrieval as a single step. In practice this pipeline needs four distinguishable operations, and conflating them is where systems fail.

1. Document retrieval. Finding passages that are lexically or semantically relevant. This is the well-studied part — dense retrieval works, and the original RAG formulation showed that conditioning generation on retrieved passages beats parametric memory alone on knowledge-intensive tasks.

2. Knowledge retrieval. Finding relevant entities, relationships, events and facts rather than passages. A character is not a paragraph. Entity-centric representations model exactly this: things and the relations between them, independent of the documents that mention them. Recent work combines the two explicitly — GraphRAG constructs an entity graph over a corpus precisely because passage-level retrieval fails on questions that require aggregating across a whole body of text.

3. State-aware generation. Assembling the slice of the world that is true at this point in the story, which requires chronology as a first-class dimension rather than metadata. This is the operation with the least direct literature behind it, and I treat my handling of it as an engineering choice rather than a validated method.

4. Validation. Checking the generated candidate against canonical sources and constraints after generation. A separate system, a different failure mode, and the step most often missing entirely.

Why retrieval alone is insufficient

Retrieval supplies evidence. It does not supply judgement. Four gaps remain after even a good retriever has done its work.

Relevance is not similarity. The passage that matters for a scene is often not the passage most similar to it. A scene about a betrayal may hinge on a throwaway line about a family debt from two books earlier. Semantic similarity does not surface that. An entity relation does.

Provenance has to survive the pipeline. If a generated claim cannot be traced back to the passage that licensed it, no meaningful audit is possible. Rashkin et al. formalised this as Attributable to Identified Sources — whether a statement is supported by a specific cited source rather than merely plausible. In a fictional canon the same machinery applies with one inversion: the ground truth is not the external world, it is a body of text I control.

Sources conflict. A published novel, an unpublished draft, a production script and an author note may disagree. The system needs an explicit precedence order. In my implementation published text outranks drafts, drafts outrank notes, and conflicts that rule cannot resolve are surfaced for a human rather than silently averaged.

Contradiction detection is its own task. Hallucination here is not factual error about the world — it is inconsistency with a corpus. That is closer to entailment checking than to fact verification. Decomposing generated output into atomic claims and verifying each against retrieved support, in the manner of FActScore, transfers to this setting almost unchanged.

And the failure survives even when retrieval is taken off the table. The DeR² benchmark supplies models with a curated set of documents verified to be necessary and sufficient for each item, then measures what happens: with the evidence in hand, models still fail on the reasoning steps. That is the experimental shape this whole argument depends on — the distinction between the system never found the evidence and the system had the evidence and still built the wrong world.

The architecture

CANON  (published novels, drafts, production material, author rulings) | +--> structured knowledge  +  indexed corpus |      entities - relationships - events - chronology - world rules | +--> retrieval for the current task |      relevant entities, relations, events, and the passages behind them | +--> construction of generation context |      the state of the world at this point in the timeline, plus constraints | +--> generation | +--> validation |      consistency against canon - constraint satisfaction - provenance check | +--> accepted  -----> corpus expands, index updates +--> rejected  -----> corrected, or returned to the human

Two properties of this loop matter more than any component in it.

The canon is the authority, not the model. The model proposes, the corpus disposes. When they disagree the corpus wins by construction, and the disagreement is logged rather than resolved by fluency.

The human is the final authority, not the validator. Automated consistency checking catches one class of error reliably and misses others entirely. Its output is a triage queue, not a verdict.

Constrained decoding is a related but distinct lever — methods like NeuroLogic decoding enforce lexical constraints during generation rather than checking afterwards. My constraints are mostly semantic rather than lexical, so they are enforced through context construction and post-hoc validation rather than at the decoder. A pragmatic choice, not a claim that it is the better one.

The case study

The corpus is a fantasy saga: novels, a lore archive, character records, an invented language, a timeline, world rules, and production material for dramatized audio editions. Hundreds of thousands of words, still growing.

Three observations from running this, offered as implementation experience rather than as findings.

The evaluation set is the expensive part. I maintain golden question-and-answer pairs drawn from the canon — questions with a single defensible answer that a human author verified. Every change to retrieval or prompting is scored against that set before it is trusted. Building it took longer than building the retriever.

Adaptation is a better validator than any test I wrote. Turning a novel into a dramatized audio production means going through it at the granularity of individual lines, casting and directing every character, and placing sound. That surfaces continuity errors no automated check found — because it forces a slow, total re-reading with different attention. In practice no book in this saga is published until its audio adaptation is finished, for exactly that reason.

The failures that matter are quiet. A model that refuses is harmless. A model that produces a fluent, well-written scene in which a character knows something they should not know is expensive, because it reads as correct.

Multimodal linkage, stated carefully

The same canonical entities increasingly connect to non-textual artifacts: a character record links to a voice identity, to recorded performances, to an avatar, to a game representation. This is a linked representation of a fictional universe — shared identifiers across media — and I would not call it a multimodal world model. Nothing in it learns cross-modal dynamics. It is a schema and a set of joins. Whether that becomes something more interesting is an open question, not a result.

Why this generalises

The pattern is not about fiction. It applies wherever a generative system must operate over a large, evolving body of information that is internally consistent by stipulation rather than by observation: technical documentation with versioned behaviour, legal corpora where precedence is explicit, research archives with provenance requirements, curriculum material that must not contradict earlier lessons, enterprise knowledge with authority hierarchies, game worlds and simulation environments.

In every one of those the same four operations appear, and the same trap: treating retrieval as the whole solution and discovering later that nothing was validating the output.

Fiction is a useful testbed precisely because it is unforgiving in an unusual way. There is no external reality to appeal to. If the corpus says a moon is red, the moon is red, and a beautifully written passage describing a silver moon is simply wrong. That makes contradiction unambiguous in a way factual domains rarely are.

What is claimed and what is not

Claimed: that treating a canon as a persistent knowledge system, with retrieval, constraint construction and post-generation validation as separate stages, is a workable architecture, and that it rests on published work in retrieval-augmented generation, long-context limitations, entity-centric representation, attribution and consistency checking.

Not claimed: that this constitutes a world model, that it eliminates hallucination, that it is novel as an architecture, or that observations from my implementation are evidence for anything beyond my implementation.

Worldbuilding is getting cheaper. What gets cheaper first is generating material. What stays hard — and what decides whether any of it is worth reading — is keeping a world consistent once it is larger than one person can hold in their head.

References

Related: on AI-assisted authorship, and the silent objective failure that started the habit of validating rather than trusting.