Definition

Knowledge base

A knowledge base is the curated set of facts your AI actually trusts when answering customers, not every document on your shared drive.

What is a Knowledge Base?

Picture this: a customer asks your AI if a winter jacket can be returned after Christmas. The model pulls an old PDF from last year’s policy and says “final sale.” The customer escalates, your team honors the real 60-day window anyway, and you eat the cost of a bad answer. That is what happens when people confuse a knowledge base with a file dump.

A knowledge base is the structured, approved corpus your retrieval-augmented generation system is allowed to read. The trap is treating it like a bulk upload of every help article, Slack thread, and PDF you can find. Operators care because the knowledge base is where containment rate, refund risk, and customer trust are actually decided.

Approved sources for RAG

In ecommerce support, approved sources are the specific systems of record you let the model retrieve from: help center articles, policy pages, product specs in your PIM, shipping SLAs in your WMS, and order status pulled from your store platform. Not Slack threads, not a PDF from 2019, not a scraped forum. The retrieval layer should call named endpoints or indexed documents with clear owners.

If your AI can cite a source, that source must be on an allow-list and signed off by someone who understands the business rule behind it. I usually start with three sources: the public help center, an internal policy wiki, and live order data pulled through Shopify Admin API scopes or the WooCommerce REST API. Anything else needs a ticket and a named business owner.

This keeps hallucinations bounded and makes every answer traceable. If you cannot name the owner of a source, it does not belong in the knowledge base.

Ownership and accountability

Every chunk in the knowledge base needs an owner, not a department. Saying “Marketing owns the returns page” is too vague; name the person who signs off when the 30-day window changes to 60. In practice I keep a source registry: URL or endpoint, owner Slack handle, review cadence, and escalation path. When the AI gives a wrong answer, I look up the owner before I blame the model.

Most bad answers I have chased down were stale source data, not model failure. Ownership also means deciding what not to include. The owner of the shipping policy should reject a draft article that contradicts the returns policy. I run a weekly source-owner standup for high-traffic topics: returns, sizing, warranty, and international shipping. If an owner is on vacation, there must be a delegate. No owner, no index.

This is boring governance, but it is what keeps your containment rate from collapsing during a promotion.

Freshness and update cadence

Freshness is not “we re-crawl nightly.” It is the documented interval between a business fact changing and the knowledge base reflecting it. For an ecommerce store, product availability, pricing, and shipping rules change fast. I set different SLAs by source type: live order data is real-time via API; policy pages refresh within four hours; help articles within 24 hours; seasonal FAQ within one week.

You need a changelog or diff check so you can prove the update happened, not just assume it did. The trap is indexing a PDF once and forgetting it. I have seen a Black Friday banner policy live in a vector store for six months because nobody owned the cleanup job. I now require every source to have a TTL or a scheduled review.

We also version our embeddings so we can roll back a bad update. Freshness metrics should sit on your AI support dashboard next to tool calling latency and answer relevance scores.

Conflict resolution between sources

Conflicts kill trust. The AI reads “free returns within 30 days” from the help center and “final sale, no returns” from the product page. Without a resolution rule, the model will average them into a polite lie. I set a hierarchy: live order data beats product page; product page beats policy page; policy page beats general help article.

When two sources at the same level disagree, the fallback is “I can’t confirm that; let me connect you with support.” Never let the model negotiate between sources. We detect conflicts with an overlap test: run the same query against all sources and flag answers where retrieved chunks contradict. A human reviewer resolves the flag by updating the source or adding a routing rule.

This is messy work that requires product, legal, and ops input, but if you skip it, your AI will confidently refund orders it should not, or deny refunds it should. That shows up in chargebacks, not just CSAT.

Retrieval architecture and chunking

The knowledge base is not just a folder of documents; it is a retrieval system. For ecommerce, chunking matters. A 40-page shipping guide chunked at 512 tokens will drop the “no PO boxes” clause three chunks away from the “free shipping” clause, and the model will miss the restriction. I chunk by topic boundary, add metadata like product_id, region, and effective_date, and use hybrid search: dense vectors plus keyword.

Metadata filters let a UK customer see UK policy, not US policy. I also separate static knowledge from dynamic facts. Static content goes into the vector store with scheduled refreshes. Dynamic facts. Order status, inventory, tracking. Come through OpenAI function calling or similar tool calls at answer time. Mixing them in one index is a freshness and permission nightmare.

Keep the knowledge base for things that change slowly and tools for things that change every second.

Measuring knowledge base health

You cannot manage what you do not measure. I track four metrics: source freshness lag, conflict flags per week, citation accuracy, and “I don’t know” rate. Freshness lag is the time between a source edit and the embedding update. Conflict flags are manual or automated contradictions found in retrieval tests. Citation accuracy checks whether the AI’s cited source actually supports the claim.

A high “I don’t know” rate usually means coverage gaps, not model stupidity. I run a weekly adversarial test with real customer questions from the past seven days. If the AI gives a wrong answer, I trace it back to source, chunk, or conflict. That trace becomes a ticket for the source owner. No trace, no fix. Over time these tests become your regression suite.

They are more useful than any generic “AI accuracy” score because they test the actual decisions your AI makes with your actual policies.

Common questions

Frequently asked questions

What is the difference between a help center and a knowledge base?

A help center is built for humans to browse; a knowledge base is what your AI is allowed to retrieve from. Not every help article should be indexed, and some knowledge base sources may never be public, like internal policy documents or live API data.

How often should I refresh my knowledge base?

It depends on the source type. Live order data should be real-time via API; policy pages within hours; help articles within a day. Set a TTL for every source and measure freshness lag, not just crawl frequency.

Who should own the knowledge base?

No single person owns the whole thing. Each source has a named owner. A program manager coordinates the source registry, review cadence, and conflict resolution. The AI team owns the retrieval pipeline; business teams own the facts inside it.

How do I stop the AI from mixing up US and UK policies?

Add region metadata to every chunk and use filtered retrieval so a UK query only sees UK sources. If the query is ambiguous, ask the customer for their market before answering instead of guessing.

What do I do when two approved sources disagree?

Define a source hierarchy and a fallback. Live data beats static pages; specific pages beat general articles. If same-level sources conflict, have the AI escalate instead of guessing. Then fix the source so it does not happen again.