An AI knowledge base your team can trust starts with a short list of approved documents, a named owner for each one, and a way to retire old versions. Then you test it with three kinds of questions before anyone uses it on real product work. Loading every file on the shared drive does not tell the tool which answer is the right one.
Here is how it goes wrong. A rep asks a product question. The AI gives a neat answer with a document link. The link opens. The document is real. It is also the version your team stopped using last year. That is a source problem. Better writing will not fix it.
For a building business, this comes up all the time. Product names look alike. Specs change. A familiar document gets copied into one more folder.
Give Each Source a Job
Start with one set of questions your team needs answered. Do not start with the whole shared drive.
A showroom team might need current finish options and ordering contacts for one product line. A dealer's operations team might need its approved delivery process. Those two sets of documents have different owners and different limits.
Keep customer-facing material apart from internal policies and restricted records. Just because a person may read a file does not mean every AI user should get answers drawn from it. Have the person in charge of information access approve the set of documents and the tool's permissions. A line in a prompt will not hide what the system is able to pull up.
Check permission on material made for other uses, too. An interview or a product video made to be published is not automatically cleared to go into an AI tool. It needs its own OK.
Fill Out a Source Card for Each Document
| Field | What it tells the team |
|---|---|
| Exact product or process | What this document covers, and what it does not |
| Owner | Who can confirm it is current |
| Version and date it took effect | Which edition is approved for use |
| Official location | Where the master copy lives |
| Audience | Who may use it and for what |
| Replaced material | Which older copy must not guide new answers |
| Review trigger | What event means it needs another check |
A recent upload date is not the same as a recent document. Someone may have uploaded an old file yesterday.
Keep a record of every source that was replaced. Your setup should stop retired files from quietly competing with the approved answer. How that works depends on the system, so ask the provider to show you.
Test What Happens When Two Files Disagree
Here are two made-up training sheets. Neither one has real installation advice.
- Training Sheet A: Demo Window Alpha, edition 1. Points the team to Review Checklist A.
- Training Sheet B: Demo Window Alpha, edition 2. Marked approved and replaces edition 1. Points the team to Review Checklist B.
Ask the tool which review checklist applies to Demo Window Alpha. A good answer names Checklist B, points to edition 2, and explains that it replaced edition 1. It should not blend both checklists into a new one.
Now take out the "approved" and "replaces" notes. Ask again. This time the tool should flag the conflict and send it to the source owner. It should not decide the higher edition number wins with nothing else to go on.
This is how a good setup should behave. Some AI tools will not handle it well, which is why you run the test before trusting it with real product questions.
Run Three Checks Before Your Team Uses It
- Known answer check. Ask a question with an approved answer. Does the response name the right source and keep its conditions?
- Conflict check. Give it two allowed sources that disagree. Does it show the disagreement instead of hiding it?
- No answer check. Ask for something the documents do not contain, such as a current lead time when no lead time source is loaded. Does it say it does not know instead of making up a date?
Have a product or process expert judge the results. A citation shows where to look. It does not prove the tool read the source correctly.
Save each question with the source version, the response, and the fix. That gives you a test you can rerun after every update.
Who Keeps the Knowledge Base Current?
Name who approves new documents, who retires old copies, and who reruns the test questions after a change. Give users an easy way to report an answer that looks wrong, so fixes do not live only in one person's notes.
If the same answer keeps failing, go back to the source and the setup before adding more instructions.
This is the same rule we use on custom AI projects that help your team find answers. Any project has to say who can use the information, who keeps it current, and where a person checks the work. Once your answers are reliable, you can build them into a repeatable workflow for one job. If those answers end up in customer emails, use the checklist of what a person should check before a sales draft goes out.
Start with one product line or one internal process. Fill out its source cards and run all three checks before you add anything else.
Questions and Answers
Start with one set of questions and a short list of approved documents, each with a named owner, a version, and an official location. Record which older copies were replaced so they do not compete with the approved answer. Then test the setup before anyone uses it on real product questions.
Run three checks. Ask a question with a known approved answer, give it two allowed sources that disagree, and ask for something the documents do not contain. A good setup names the right source, shows the conflict, and says it does not know instead of making up an answer.
Do not rely on it. A line in a prompt will not hide what the system is able to pull up. Keep customer-facing material apart from internal policies and restricted records, and have the person in charge of information access approve the documents and the tool's permissions.
Name who approves new documents, who retires old copies, and who reruns the test questions after a change. Give users an easy way to report an answer that looks wrong. If the same answer keeps failing, fix the source and the setup before adding more instructions.
Want Help Putting It to Work?
A free call with Stefanie Couch. About 30 minutes.
Not ready to talk? Get the Built to Win Newsletter

