AI knowledge systems / daily field note

AI Knowledge Freshness: Retire the Old Answer

A cited answer can still be yesterday’s policy. The business needs a way to retire knowledge, not just upload more of it.

6 minute readAI knowledge freshness

Imagine a UAE distribution group changing its warranty terms. The English policy is approved on Monday. The Arabic version follows later. Sales still has the old PDF in a shared folder, and the internal assistant confidently quotes it to a branch supervisor. Every sentence has a citation. Nobody has hallucinated anything. The business has simply failed to retire an answer. AI knowledge freshness is the operating discipline that prevents this gap between a policy changing and the assistant changing with it.

I would ask a simpler question than which model the group uses: when an authorised owner withdraws a rule, how long can an employee still receive it as current guidance? A larger context window does not answer that. Neither does uploading the replacement document. The old material may remain searchable, translated, cached or embedded in a saved conversation.

AI knowledge freshness starts with authority

Separate three dates: when a rule takes effect, when its source was last approved, and when the assistant last processed that source. A document uploaded this morning can contain an expired offer. A policy approved last year can remain valid. Treating the latest file timestamp as the truth confuses technical activity with business authority.

For a Dubai headquarters serving UAE and Saudi branches, authority also has a scope. A warranty may apply to one product family, contracting entity or market. Arabic and English copies may have different review states. The assistant needs the approved version for the question being asked, rather than whichever passage happens to match its wording most closely.

This is a content lifecycle problem within practical AI consulting. Give the person who owns the policy responsibility for its meaning, and the system owner responsibility for propagating it. Neither can finish alone. Use five decisions to join their work.

Five decisions before connecting another folder

1. Name the authoritative record

For each knowledge area, identify one approved publishing location and one accountable owner. Record the document identity, version, market, language, effective date, review date and replacement relationship. Keep drafts and historical material distinguishable from current guidance. An assistant can legitimately explain an old contract, but it must not present that contract as the default rule for a new customer.

Do not let the model settle a disagreement between two policy owners. Route the conflict to the business and suspend definitive guidance on the disputed point. Where translation approval is pending, define a permitted fallback with the owner. Silently using an older Arabic policy because it reads fluently is not an acceptable language strategy.

2. Give each answer a freshness requirement

Classify information by the consequence of being late. A general onboarding guide, a temporary delivery restriction and a customer-specific price require different update behaviour. Set the maximum acceptable delay for each class and decide what happens when that limit is exceeded. These are business choices, not universal vendor defaults.

For fast-changing operational facts, consider querying the authorised system of record at answer time instead of placing a periodically copied document in the knowledge base. Keep stable explanatory material in retrieval. The architecture should follow the required freshness, rather than forcing every kind of information through the same upload routine.

3. Trace updates and withdrawals separately

Ask the technical team to demonstrate both replacing a document and removing one without a replacement. Those are different paths. Amazon Bedrock’s S3 connector documentation describes crawling new, modified and deleted content when the data source syncs. That makes the sync event part of the operating promise. A changed source alone does not prove that the assistant is already using the change.

Deletion deserves particular attention. Microsoft’s Azure AI Search guidance explains that resetting and rerunning an indexer does not itself clean up orphaned search documents. The lesson for the buyer is specific: ask how this connector detects removal and how the team proves that obsolete fragments have disappeared. “We rebuild regularly” is not enough evidence.

Follow the document into extracted fragments, search results, answer caches and conversation history. A current search index cannot correct text already supplied in an earlier turn. Define when the application must retrieve again, flag an older answer or stop relying on previous context. Preserve historical records where required, while removing their authority to answer present-day questions.

4. Make uncertain freshness visible

Show the relevant policy version or effective date with consequential guidance. More importantly, have the application check whether the underlying collection is within its approved update window. If ingestion fails, an old green status from yesterday must not continue to imply current coverage.

Agree the degraded behaviour in advance. The assistant might direct staff to the approved source, ask them to contact the policy owner, or decline to state current terms. Choose the response according to the task’s consequence. A confident answer followed by a small warning is a poor fallback when someone will use the answer to make a customer commitment.

Keep freshness distinct from RAG access control. A person may be entitled to read a document that is no longer current. Equally, a newly approved policy may still be restricted. The release process must satisfy both conditions without treating one as evidence for the other.

5. Rehearse a policy change

Use controlled test documents with unmistakable values. Ask the same question before and after a change, in both supported languages, in a new chat and an existing conversation. Withdraw the source entirely. Introduce conflicting versions. Pause synchronisation. Check which answer appears and what the user is told when current guidance cannot be established.

Record elapsed time from approval to verified answer, plus the time until superseded guidance stops appearing. Inspect retrieved passages as well as the final prose; the model may happen to answer correctly while still receiving obsolete material. This rehearsal complements AI model evaluation by testing change over time, rather than only a fixed collection of questions.

Give the owner a small operating report

The weekly report should show overdue reviews, failed updates, unresolved version conflicts and questions blocked because freshness could not be established. Include the affected business area and the person expected to act. A document-count dashboard can grow while the usable knowledge gets worse. Count unresolved authority problems instead.

AI knowledge freshness becomes manageable when every consequential answer has an owner, an effective version and a tested withdrawal path. Before connecting another department, change one policy and prove that the previous answer loses its authority everywhere the assistant works. If the business cannot retire yesterday’s guidance, it is not ready to distribute tomorrow’s at scale.

Have a problem hiding behind a technology conversation?

Start a conversation