What is AI-ready data?

The five criteria of AI-ready documents

A document set is AI-ready when it meets five conditions. Each one maps to a failure mode you will otherwise see in the answers.

1. Deduplicated. One canonical version per piece of knowledge. Copies in forgotten sites, exports, “final_v3_FINAL” variants are either removed or point at the canonical document. Failure mode: the model retrieves four versions, pays for all four, and reconciles them silently.
2. Consistent. Documents that cover the same fact agree — or the disagreement is known and resolved. Failure mode: the travel policy in Finance says 0.30 per km, the copy in HR says 0.35; the assistant returns one and the user has no signal that the other exists.
3. Current. Superseded content is marked as such, archived, or excluded from retrieval. Failure mode: the 2019 file still ranks first because it is well-linked, and the model states it confidently.
4. Permission-correct. The AI sees exactly what the asking user may see, mirrored from the source system — no over-sharing through the AI, no under-sharing that hides the right answer.
5. Traceable. Every answer can cite its source document, page or row, so a human can verify it in one click. Without citations, “AI-ready” cannot be checked — it is only claimed.

Why indexing is not the same as AI-ready

Most enterprise search and assistant products answer the question “where is it?”. They index what exists and retrieve the closest match. That is necessary — and it is exactly where unready data fails: indexing does not know that two documents disagree, that a version is superseded, or that a library is a stale copy of another. Retrieval returns one of them; the model answers.

Better models do not fix this. A stronger model reconciles four contradictory sources more fluently — it does not tell you which one is authoritative. That is a property of the corpus, and it has to be established before retrieval: as findings a person or agent can act on, not as an answer a user has to trust.

How to measure AI-readiness

Measuring it is the difference between a clean-up project and a clean-up habit. Practical metrics, as TensorLot implements them in its console:

• A health score per document (0–100) and for the whole tenant, tracked over time — e.g. a tenant at 82/100, +34 points since March.
• Findings with a type (duplicate, contradiction, outdated), a severity, an owner and a status (open / resolved) — so “what is wrong” becomes a list, not a feeling.
• Contradictions as pairs: the two documents side by side, the key differences, a confidence score and a verdict (“possible update: Document A may be outdated”).
• Coverage: what share of documents has a canonical version marked; how many outdated documents still rank in retrieval.
• Cost: tokens per answer before and after curation — the number that makes the CFO care.

How to get there — four stages

Making documents AI-ready is a process, not a migration. Nothing needs to move; nothing should be rewritten behind anyone’s back. The four stages as TensorLot Core runs them:

1. Connect. Connect the sources read-only — SharePoint sites, libraries and site collections, Confluence, Jira, GitHub — with incremental sync and permissions mirrored from the source.
2. Analyse. Score every document, detect contradictions across documents and sources, find duplicates and outdated content, record each as a finding with the affected documents and a canonical candidate.
3. Clean. Every finding gets a recommended action (resolve, archive, move, ignore). Action rules define once what “archive” or “resolve” means per URL prefix, library, site or tenant — the most specific rule wins — and the person or agent acts with their own rights. The system itself never writes to the source.
4. Serve. Serve the cleaned knowledge with citations — to people (Tensor Bot) and to agents through one MCP endpoint with governed tools and an audit log per call. On any model.

The payoff: precision and cost are the same lever

AI-ready data is usually sold as an accuracy story. It is equally a cost story. Every duplicate and superseded version that enters a context window is billed on every query, then costs a second time when the unreliable answer triggers a follow-up question. Curated, deduplicated context is smaller and more precise at the same time: in TensorLot pilot deployments (2026) it meant up to 70 % fewer tokens per answer — on any model, because the saving happens before the model is called.