Keeping a knowledge assistant's answers current

A knowledge assistant that was accurate on launch day and never updated becomes actively misleading within months — and the update mechanism matters more than the initial build quality.

Why staleness is worse for an AI assistant than for a search tool

A traditional search tool that returns a stale document at least shows you the document, with its date, letting you judge its currency yourself. An AI assistant that answers confidently in natural language, drawing on stale indexed content, presents outdated information with the same fluent confidence as current information — the staleness is invisible to the person asking, which makes it more dangerous, not less, than a simple search result.

The re-indexing architecture that prevents this

We build automated re-indexing triggered by actual source changes, not on a fixed calendar schedule alone. For sources with webhook support (many modern SaaS tools, including Confluence and Google Drive, support this), a document edit triggers near-real-time re-indexing of just that document. For sources without webhook support, we poll on a defined interval — typically hourly for high-change-frequency content, daily for more static content — checking for modifications and re-indexing only what's changed, rather than a full, expensive re-index of everything on every run.

Handling deleted or superseded content correctly

An underappreciated failure mode: a document gets deleted or superseded in the source system, but the old indexed version remains in the vector database, and the assistant keeps confidently citing content that no longer exists anywhere current. We build explicit deletion propagation — when a source document is removed or replaced, its indexed content gets removed too, not left as an orphaned entry that can still surface in search results.

A concrete example

For a client's HR-focused knowledge assistant, an early version had a two-week lag on re-indexing because it was only running on a scheduled weekly batch job. During a period when the company updated its parental leave policy, the assistant continued confidently citing the outdated policy for over a week before the next scheduled re-index — a real problem for a document employees might make personal decisions based on. We rebuilt the sync to use the HR system's webhook support where available, reducing effective update lag from up to a week to typically under 15 minutes for that source, and added a "last verified" timestamp displayed alongside every answer, so even in edge cases where a source lacks webhook support, users can see how current the underlying information is rather than assuming infinite freshness.

The confidence-and-sourcing pattern that builds trust even when perfect freshness isn't guaranteed

No re-indexing system is instantaneous for every possible source. Rather than presenting false certainty, we design every answer to cite its specific source document and, where available, a last-updated date — so users can judge currency themselves for anything time-sensitive, rather than the assistant implicitly claiming perfect real-time accuracy it can't fully guarantee.

How Ndakum approaches it

Automated, source-triggered re-indexing is a standard requirement in every AI Knowledge Assistant we build — not an optional add-on, because a stale assistant is worse than no assistant.