Skip to content
Olivares AI docs

Module VIII — data, knowledge & context

Module VIII is the governed data plane: it builds knowledge bases and runs semantic RAG over a pluggable vector index, governs every retrieval by identity, classification and residency, and records append-only lineage of what crossed the perimeter and what the residency gate refused, so a residency claim is evidenced rather than asserted. It also holds the versioned prompt registry, governed agent memory and context/compaction policies as data — not as promises.

What it is

Section titled “What it is”

The module orchestrates the data plane; it does not re-implement its neighbours. It pulls content from read-only data connectors, runs every body, prompt template and memory entry through its own redactor before anything is chunked, embedded, hashed or stored, then governs retrieval against the grants the identity module declares. Embedding is delegated to a model seam — the module never calls a provider directly — and ranking is delegated to a vector-index seam, so the governance contract is identical whether retrieval runs in-process or against an external ANN backend.

The red line is non-negotiable: the product governs the customer’s data and never sells or exfiltrates it. Data crosses the perimeter only where the operator provisioned a crossing — an external embedding provider, a SIEM/webhook output — and the residency gate is deny-closed on every other destination. Three mechanisms record that in the design — redaction before indexing, the egress gate, and lineage that evidences which crossings happened.

Contract & entities

Section titled “Contract & entities”

Module VIII declares eight tenant-scoped entities in the shared data model: the knowledge base, the document (metadata and provenance, never the body), the chunk (redacted text plus an inherited classification and ACL), the prompt and its append-only immutable revisions, governed agent memory, the context/compaction policy, and the append-only lineage row. Its routes mount under the module’s own namespace, wrapped with authentication, tenant scoping and authorization; reading knowledge and lineage is a privileged, audited action.

Retrieval is the security contract, and the order is the contract: resolve the identity’s grants (fail-closed — a guard error denies, never a degraded allow), apply the residency gate, embed the query, then filter candidates by classification and ACL before ranking so a chunk the identity cannot see never enters the ranked set, then rank, then append the immutable lineage row. The egress gate is composed on top: a residency-locked knowledge base refuses ingest or retrieval with an embedder that would egress, enforced at create, update, ingest and retrieval (defence in depth). Document content travels a typed connector contract by design, not the event bus — bulk reference data must not be broadcast.

On the event bus

Section titled “On the event bus”

Module VIII produces finding.reported events: one hashed FindingReport per ingest when a secret or PII is redacted, and a finding when a residency or egress gate denies — hashed detail only, never the secret or the body. Forensics and compliance consume the lineage and these findings. It consumes nothing from the bus for content: by design content rides a typed pull contract, so minimal-data is a property of the wire, not a runtime filter applied after the fact.

Section titled “Related”