Skip to main content

Architectural Safeguards Against Oracle Hallucination

The safest response to speculative reasoning is not a better disclaimer. It is an architecture in which speculation lacks the capability to impersonate canonical truth.

OntoGraph's core safeguard is separation: document tools may discover and rank candidates; the semantic core validates explicit records; governance records authority; and Promotion establishes current package visibility. No one layer is allowed to perform all four jobs.

Delivery status

StatusBoundary
ImplementedThe Rust calculus works on a private candidate copy, emits typed deterministic violations, compiles validation findings from those same values, and uses a recoverable memory/file transaction for canonical, validation, and search files.
Bounded proofThe external document adapter labels every candidate as review-required and proves exact source reconstruction without importing its parser, extractor, prompt, or candidate graph into ontograph-core.
PlannedA production prompt/extraction service, a versioned speculative-link sidecar, durable rejected-run inbox semantics, adversarial model tests, and equivalent safeguards for future storage backends.

Safeguard 1: document mechanics stay outside the semantic core

Markdown parsing, folder walking, prose extraction, prompt rendering, and source-layout reconstruction are replaceable client concerns. The core accepts format-neutral input and owns OntoUML-informed semantics, provenance/evidence rules, validation, governed mutation, and query meaning.

This means an extractor can be replaced, upgraded, or distrusted without changing the meaning of a canonical Assertion or Evidence object. Its outputs remain source-local candidates until a declared mapper submits them through the public boundary.

Safeguard 2: explicit declarations beat label inference

The bounded calculus consumes supported stereotypes, endpoints, state times, and disjointness declarations only when they are explicit. A record named Customer, Event, Role, or Permit does not acquire that ontological category from its label.

Missing interpretation produces a SemanticGap; missing ancestry produces MissingIdentityRoot; malformed mediation or provenance produces its own typed condition. The engine is designed to expose absence instead of filling it with a plausible guess.

Safeguard 3: speculation lives in rebuildable projections

Search indexes, similarity scores, snippets, clusters, and future fuzzy links are derived read models. They may point to canonical or source-local refs and carry ranking metadata, but they cannot:

  • create an Assertion;
  • count as Evidence;
  • change epistemic or lifecycle state;
  • approve a candidate;
  • select a governance alternative; or
  • establish a Promotion.

Because the projection retains its source graph digest, profile, and refs, it can be deleted and rebuilt. If deleting a fuzzy-link index would delete the only copy of a claimed fact, the design boundary has already been violated.

Safeguard 4: calculate on a private candidate state

The implemented product operation is conceptually:

Calculus(CurrentState, ProposedChangeSet)
-> Result<ValidatedState, GovernanceError>

Mutations are applied to a private in-memory candidate map. Rules evaluate the complete candidate, not a partially written workspace. On any blocking violation the supplied current state remains unchanged. An expected base digest also fails closed if the caller prepared its ChangeSet against stale state.

This protects against a particularly dangerous failure mode: writing the first half of an apparently sensible inference and discovering only afterward that the full relation violates identity, mediation, provenance, or phase rules.

Safeguard 5: one source of typed validation findings

The validation read model is compiled from the same typed violations returned by the calculus. It does not independently reinterpret free-form log messages into SemanticGap or ProvenanceGap findings.

For a rejected transition, the caller can inspect a matching in-memory report while the workspace files remain unchanged. For an accepted transition, the persisted sidecar records the ChangeSet and exact base/candidate digests. This prevents the UI from presenting a different reason than the rule engine actually returned.

Safeguard 6: canonical and sidecar publication is recoverable

After successful calculation and canonical document validation, the memory/file path prepares three byte streams:

  1. data/canonical.json — the validated canonical document;
  2. validation/last-validation.json — the derived validation result; and
  3. indexes/search-index.json — a rebuildable projection over stable IDs.

No candidate file is published before all three streams are staged. Existing files are backed up, replacements are attempted as one recoverable unit, and an injected publish failure restores every already-replaced file from its exact pre-transaction bytes. Staging and backup residue is then removed.

The transaction protects file consistency. It does not elevate the search index into truth, and it does not turn a successful semantic calculation into institutional approval.

Safeguard 7: governance cannot be synthesized by the Oracle

Even a perfectly grounded and structurally valid candidate is only a candidate at the relevant governance stage. Accepted canonical package meaning requires distinct records for the Proposal, Review, Decision, and Promotion. Generated questions and suggestions are explicitly non-assertive and unselected; they cannot authorize mutation or create Evidence on their own.

This is the critical distinction between two statements:

"The candidate satisfies the bounded rule matrix."
"An authorized Promotion made the candidate current."

OntoGraph records them with different objects because they are different kinds of fact.

Threat-to-control map

Failure modeArchitectural control
A model invents a source quoteExact Artifact digest and source-span reconstruction must verify.
Similar words are treated as the same identitySource-local refs remain distinct; identity requires an explicit proposal.
A confidence score becomes EvidenceScores are projection metadata; Evidence requires source/derivation grounding.
A label becomes an OntoUML stereotypeOnly explicit supported declarations enter the rule profile.
A failed candidate partly changes diskPure calculation precedes staging; rejection performs no workspace writes.
A sidecar publish fails mid-transactionExact backups restore already-replaced files.
A generated answer edits canonical stateAnswers re-enter as source material and a proposed ChangeSet.
Validation is presented as approvalReview, Decision, and Promotion remain separate required records.

Residual risk and the next proof

The current proof is deliberately memory/file and bounded. A future production backend must demonstrate equivalent candidate isolation, loss reporting, rollback or atomic commit semantics, stable canonical IDs, and hydration of Evidence and governance lineage. A model-backed extractor also needs prompt injection tests, source-quote verification, deterministic boundary flags, profile/version capture, and fail-closed behavior when its output is malformed.

Until those proofs exist, the safe product claim is narrow: OntoGraph has an executable semantic and transaction boundary on the memory/file path, plus an external controlled-document proof. It does not yet have an autonomous Oracle that can ingest arbitrary notes and safely govern their meaning end to end.

What this direction does not claim

No architecture can guarantee that a language model never produces false text. These safeguards instead limit what speculative output can do. They do not establish comprehensive formal conformance, production-backend parity, source authority, model certification, release approval, or automatic canonical Promotion.