Skip to main content

The Anatomy of an Unstructured Note Ingestion

A note enters OntoGraph as source material, not as truth. Its text is preserved first; structure, claim candidates, evidential grounding, interpretation, and canonical acceptance are separate stages with separate records.

Delivery status

StatusBoundary
ImplementedOntoGraph has canonical Source, Artifact, Assertion, Evidence, derivation, validation, gap, and governance semantics. The memory/file calculus can validate explicit candidate changes and publish accepted writes atomically.
Bounded proofThe standalone reference-document adapter parses three controlled UTF-8 Markdown documents into ordered blocks, source-local entity and claim candidates, exact byte-range evidence spans, and byte-exact reconstruction. Its output is not yet admitted through the OntoGraph service.
PlannedRecursive folder discovery, arbitrary-note admission, character-index aliases, adapter-to-service submission, and optional semantic extraction across a changing corpus.

Begin with bytes, not beliefs

Consider this raw text scrap:

Pump P-17 is isolated during maintenance.
Isolation Permit IP-9 authorizes the maintenance.

For this ASCII example, the first sentence occupies UTF-8 byte range [0, 41) and the second occupies [42, 91). The newline at byte 41 belongs to the source representation even if it is not part of either sentence.

That distinction matters. The current external adapter's executable contract uses UTF-8 byte offsets because they can be checked directly against the exact captured bytes. A future interface may also expose character, line, heading, page, or block selectors, but it must retain one unambiguous representation against which a quoted span can be reconstructed.

The records are different kinds of thing

RecordWhat it identifiesWhat it does not establish
SourceThe origin context: for example, a selected notes corpus or repository.That the origin is authoritative or correct.
ArtifactOne exact information-bearing representation, with media type, locator, fingerprint, and version context.That every sentence is a canonical claim.
Source spanAn addressable portion of that exact Artifact, such as a byte range and quote.Evidence quality or interpretation by itself.
source-local Assertion candidateA proposed reading of a statement found in the note.Accepted identity, ontology, or truth.
EvidenceA first-class connector between an Assertion and its Artifact/span grounding.Review, approval, or Promotion.
DerivationEventThe parser or extractor occurrence, including method profile and input/output refs.Human authority over the extracted meaning.

The Source and Artifact split prevents two common errors. A folder is not the same thing as one file, and a file's durable identity is not the same thing as one captured version of its bytes. Reimport can therefore preserve history instead of overwriting yesterday's evidence with today's note.

The structuring filter

The target ingestion path has seven explicit steps.

1. Capture

The external document tool reads the selected file without modifying it. It records the exact bytes, media type, length, stable root-relative locator, and cryptographic digest. A successful capture survives even if a later parser cannot understand the document.

2. Register origin and representation

The admission package identifies the containing Source and the exact Artifact representation. Import time, importer identity, acquisition method, and prior snapshot lineage remain provenance facts; they are not compressed into a generic createdBy field.

3. Parse losslessly enough to point back

Markdown headings, paragraphs, lists, tables, code blocks, links, and other layout units become tool-owned structural records with ordered byte bounds. Unsupported syntax is retained as a span or an explicit parse gap. It is never silently discarded merely because it is inconvenient to interpret.

4. Extract candidates, not facts

An extractor may read the example as a candidate relationship between the pump, a maintenance occurrence, and a permit. The candidate records its surface text, method, confidence, and reviewRequired=true. A sentence that contains no supported pattern remains source text; the importer is not obligated to turn every sentence into an Assertion.

5. Bind each candidate to first-class Evidence

The logical shape is:

Assertion candidate: "IP-9 authorizes the maintenance"
|
| evidenceRef
v
Evidence: supportKind = supports; method = parser-extraction
|
+--> Artifact: the exact note representation
`--> Source span: byte [42, 91), exact quoted text

The current canonical contract represents Evidence as an object with an assertionId, artifactRefs, optional addressable source-span selectors, a support kind, and a method. That is materially stronger than attaching a floating confidence number to an edge: the reader can recover what text was used, from which representation, by which extraction step.

6. Validate the proposed interpretation

If a later mapping proposes explicit OntoUML structure, the core evaluates the complete candidate state. Missing stereotypes, missing identity roots, dangling Relators, broken Evidence endpoints, inheritance cycles, and explicit disjoint-phase conflicts produce typed violations. Failure does not erase the captured Artifact.

7. Govern canonical visibility

Passing structural validation is not enough to make an interpretation current. Selected candidates still require the appropriate Proposal, Review, Decision, and Promotion lineage. Raw admission and canonical acceptance answer different questions and may happen at very different times.

Why exact spans are the load-bearing detail

Without exact spans, an extracted claim is difficult to audit, challenge, or re-extract after a parser change. With spans, OntoGraph can answer:

  • Which exact bytes prompted this claim?
  • Did a later note version change those bytes?
  • Which parser and profile produced the candidate?
  • Does another source challenge the same assertion?
  • Can a reviewer inspect the original context rather than a summary?

This is why a source quote is not merely display text. It is part of the traceability path from interpretation back to the immutable representation on which it depends.

What this direction does not claim

The current product does not recursively ingest an arbitrary notes folder into OntoGraph, and the standalone adapter's candidate graph is not an OntoGraph graph. Exact extraction does not resolve identity, determine an OntoUML stereotype, create accepted Evidence, approve a change, or promote canonical state. Those boundaries are the subject of the remaining articles.