Knowledge needs a meta-knowledge layer

In Part 1, I argued that the core failure in mature RAG systems is not retrieval in the narrow sense. A system can retrieve authoritative, relevant text and still give the wrong answer because it never determined whether that evidence applies here, now, under these conditions. That was the applicability problem.

The next question is architectural: what would a system need in order to enforce applicability before generation begins?

It needs more than a corpus. It needs a meta-layer: a description of what each knowledge base is for, when it applies, what inputs it requires, how it should be queried, and where its boundaries are. It needs to know what it knows. I call this the meta-knowledge layer.

Meta-knowledge is needed as soon as a corpus stops being uniform. Mature organizations accumulate conditional truths: policies that branch by region, plan tier, product version, effective date, environment, or user status. The simplest RAG architecture treats all of that as one thing: index everything and let similarity search sort it out. That can work when the corpus is small and internally consistent. It breaks down when the same question has different valid answers under different conditions.

Consider our use case from part 1: A customer asks, “My toaster isn’t working—can I get a replacement?” A naive system retrieves passages about toaster replacement from the standard retail warranty, a utility-program warranty, and a newer troubleshooting requirement. Each passage is topically relevant. But they don’t belong to the same policy branch. When the system blends them, it produces a fluent answer that no real workflow will honor.

That is a franken-answer: locally relevant evidence combined across incompatible applicability frames.

There’s nothing wrong with retrieval in the narrow sense. The failure is structural. The system was asked to decide applicability at query time without being told how the organization’s truth is partitioned.

Organizations already know their knowledge is not one thing. Different teams own different documentation. Different systems of record govern different processes. Different access controls, authority levels, and freshness rules apply. The problem is not that these distinctions do not exist. The problem is that many RAG stacks fail to represent them explicitly at runtime.

A useful unit to think about here is the knowledge base: a bounded collection of documents behind a retrieval interface. One knowledge base might cover retail warranty policy, another utility-program eligibility, another troubleshooting procedures, and another fulfillment operations. This does not require physically separate vector stores - it could be separate indexes, namespaces, filtered views, or some other boundary. What matters is that the runtime knows these are distinct domains with different scope rules, required inputs, and authority structures.

With that said, partitioning is necessary, but it’s not enough. Once knowledge is partitioned, the agent has a new problem: which knowledge base should it consult for this question, and does it have enough information to consult it safely?

That is where the meta-knowledge layer comes in.

I call the concrete implementation for meta-knowledge a “manifest”. A manifest is an agent-readable contract for a knowledge base. It describes what the knowledge base is for, when it should be used, what inputs must be known first, how a query should be formed, what sources are authoritative, and where the knowledge base stops.

Think of it as an interface description for retrieval. It defines how to call the knowledge base and when that call is valid.

This is broader than ordinary metadata. Metadata can tell you a document’s date, owner, or tags. A manifest tells the runtime how to reason about applicability before retrieval begins. It turns “knowledge about knowledge” into something enforceable.

It also helps to state what this argument is not. The issue is not literally “one index bad, many indexes good.” Mature systems already use filters, namespaces, ACLs, and routing heuristics. Those help, but they do not by themselves solve applicability. The deeper requirement is that scope rules, required inputs, authority conditions, and boundary conditions be explicit and inspectable at runtime.

In practice, a manifest might include five kinds of information:

• Identity and purpose: a knowledge base ID, domain description, declared capabilities, and expected outputs.
• Input requirements: the fields that must be known first, their dependencies, validations, and which missing values require clarification.
• Routing and scope: the user goals the knowledge base serves, the cases it excludes, and examples of in-scope and out-of-scope queries.
• Query construction: templates or rules for assembling a stable, structured query once the right inputs are available.
• Authority and validity: governing sources, freshness expectations, effective dates, precedence rules, and consistency constraints.

The order matters because it mirrors the runtime decision sequence.

A system that uses manifests can handle applicability in a much more disciplined way.

1. Route and scope-check - The agent or router reads manifests to determine which knowledge base actually matches the user’s goal. If no knowledge base is in scope, it redirects early instead of retrieving vaguely related text and generating a confident wrong answer.
2. Collect required inputs - The manifest declares what must be known before retrieval is meaningful. A warranty knowledge base may require product model, purchase or enrollment date, region, and program status. If those are missing, the system asks for them rather than guessing.
3. Construct the query - Once the knowledge base is selected and the required inputs are available, the system assembles a query using the manifest’s template or rules. That keeps retrieval aligned to the knowledge base’s structure instead of relying on whatever the raw user phrasing happened to include.
4. Retrieve evidence - Only then does the system request context from the selected knowledge base.
5. Check consistency before answering - Retrieved evidence is not automatically safe to use. The runtime still needs to verify that the returned material is consistent with the user’s branch, current in time, and authoritative enough to govern the answer. If it is not, the system should ask another question, answer cautiously, or decline.

This structure addresses several common applicability failures.

• Ambiguity — When a question is linguistically clear but underspecified for the corpus's branching structure, the manifest declares which distinctions matter and which missing fields should trigger clarification before retrieval even starts.
• Implicit conditions — Sometimes applicability lives entirely outside the text — in configuration, environment, or system state. The manifest surfaces these hidden selectors (enrollment status, plan tier, release channel) so the agent knows what to resolve before trusting retrieval results.
• Compositional applicability — Individually correct chunks from incompatible branches can produce a franken-answer. The manifest declares branch variables, scope assumptions, and consistency constraints so evidence is only combined within the same applicability frame.
• Authority conditions — Topically relevant text isn't always authoritative enough to govern an answer. The manifest distinguishes governing sources from explanatory or subordinate ones, giving the runtime a basis for preferring normative material over description.
• Granularity mismatch — The agent might apply a narrow exception too broadly or give a generic answer where a specific one is needed. The manifest declares the level of detail the knowledge base supports and what additional inputs are required to safely narrow the answer.
• Path convergence — Different logic paths can converge on the same surface-level answer while carrying different constraints and downstream procedures. The manifest makes branch boundaries explicit so the agent tracks which governing path it's actually on, not just whether the answer sounds right.
• Temporal applicability — Retrieved information may have been true once but is no longer in force. The manifest declares effective dates, freshness expectations, version precedence, and supersession rules so the runtime distinguishes current truth from historical truth.

This is why high recall is not enough. Vector retrieval is good at finding related passages. But “related” is weaker than “applicable”.

A support system may retrieve the standard 24-month retail warranty because it matches “toaster replacement,” even though the user’s device came through a utility program with a 12-month window and a registration requirement. In both cases, retrieval can look excellent while the answer is still wrong. The missing piece is not more semantic similarity. It is enforceable applicability constraints.

The toaster example makes the operational difference clear.

Without a meta-layer, the agent searches across all available sources and lands on the closest-sounding guidance. It retrieves the retail warranty, the program warranty, and the post–January 1 troubleshooting requirement, then blends them into a single answer that sounds helpful but corresponds to no valid branch.

With the meta-layer in place, the behavior changes:

• routing selects among retail warranty, utility-program warranty, troubleshooting, and fulfillment.
• scope rules prevent the agent from consulting a knowledge base that does not apply to this customer.
• input gating collects missing constraints such as enrollment date, model, region, and program membership.
• query construction produces a stable, scoped query for the selected knowledge base.
• consistency checks reject mixed-branch or stale evidence before an answer is produced.

The retrieval backbone may be identical in both systems. The difference is that one treats applicability as an explicit runtime concern and the other leaves it to chance.

This matters even more for agentic systems. When an agent can call tools, mutate state, or trigger workflows, a wrong knowledge base does not just produce a wrong paragraph. It produces a wrong action.

An operations agent asked to restart the payment service should verify environment, permissions, and deployment topology before choosing a runbook. A billing agent should verify account state, plan tier, and region before changing a customer’s plan. The principle is the same as in RAG: the agent should not act merely because it found a semantically related procedure. It should act only when the applicability conditions for that procedure are satisfied.

There is a cost to this approach. Manifests have to be authored, maintained, and kept in sync with the underlying knowledge bases. They can drift. They will not remove the need for good retrieval, good source management, or good judgment. But they force a useful discipline: the same organization that curates a body of knowledge also has to declare its scope, authority, and preconditions in a form the runtime can inspect. In practice, that is often less brittle than hiding all of that logic inside prompts, heuristics, and hope.

RAG’s practical limit is not recall. It is applicability under real constraints.

A meta-knowledge layer becomes operational when it is expressed in concrete artifacts: a manifest for each knowledge base, and a planner that reads those manifests to decide what to ask, what to query, how to sequence calls, and when not to answer at all.

So when you see a fluent wrong answer, don’t just ask, “What did the system retrieve?” Ask harder questions. Did it choose the right knowledge base for the user’s goal? Did it have the required inputs, or should it have clarified first? Was the evidence authoritative, current, and internally consistent? And should the system have answered at all?

In the next part of this series, we’ll start digging into the practical side of things: we’ll create a working manifest and planner by using progressive introspection techniques and LLM based planning and routing.