AI for internal knowledge search across what you already wrote

3 days ago
3 min read

Updated: 1 day ago

Introduction

In any business more than a few years old, the answer to most questions already exists somewhere. It is in an email from two years ago, a job file, a supplier specification, a method statement, a quote, or the head of somebody who is on holiday. The cost is not that the knowledge is missing; it is that finding it takes longer than asking someone, and asking someone interrupts them.

This is the application where the return is largest and least disputed. Being able to ask a question in plain language and get an answer with a reference to the source document changes how a small business operates, particularly where expertise sits with two or three long-serving people. It also has a specific and significant risk, which is that a confident wrong answer is worse than no answer.

1. AI for internal knowledge search must cite its source

The non-negotiable requirement.

Every answer should point at the document it came from, so the person asking can verify it. Without a citation you have a system producing plausible statements, which in a technical business is dangerous rather than merely unhelpful.

2. Start with the questions people actually ask

Scope by demand.

Note the questions that interrupt your experienced staff for two weeks. That list defines what the search needs to cover, and it is usually narrower than the whole document estate.

3. Curate the source material rather than indexing everything

Quality of input decides quality of output.

Superseded procedures, draft documents, old price lists and abandoned proposals will be retrieved and presented with the same confidence as current ones. Deciding what is authoritative is the substantive work.

4. Mark versions and dates clearly

The most common failure mode.

An answer drawn from a procedure replaced last year is wrong, and nothing in the answer will say so. Document dates, version numbers and a retirement process for superseded material are what prevent this.

5. Control who can see what

A real access problem.

Personnel records, pay information, client confidential material, tender pricing. A search tool that retrieves across everything will surface things people should not see, and permissions have to be designed in rather than added later.

6. Be careful with client confidential information

A professional obligation.

Documents covered by confidentiality clauses, and personal data belonging to your clients, have restrictions on where they can be processed and who can access them. This needs checking against your own obligations before anything is indexed.

7. Expect the gaps to be revealed

A useful side effect.

The questions the system cannot answer are the things your business has never written down. That list is the documentation priority, and it is more valuable than an ordered plan to document everything.

8. Keep an obvious route to a person

For the cases that matter.

Anything safety-related, contractual, or where the answer is uncertain should route to someone who knows. A system that always answers, regardless of confidence, teaches people to trust it exactly when they should not.

9. Measure it by time saved on real questions

The honest evaluation.

How long the same questions used to take, and how often the answer is now found without interrupting anyone. Usage statistics tell you about curiosity; this tells you about value.

Watch for staleness over time. A knowledge base that is not maintained degrades into a confident source of last year's answers, and the degradation is invisible because the answers still look authoritative.

Conclusion

Require a citation on every answer, because an unsourced confident statement is the main risk here.

Scope the system around the questions that actually interrupt your experienced people, curate the source material instead of indexing everything, mark versions and dates and retire superseded documents, design access permissions before you index anything sensitive, check your confidentiality obligations regarding client material, treat the unanswerable questions as your documentation priority, keep an obvious route to a person for safety and contractual matters, and measure the result in time saved on real questions.