Semantic Search for Business Data: Building a Search Assistant with RAG and n8n
By Abhishek Patoliya · 2026-08-12
Search a company's internal wiki for "how do I cancel a subscription" and keyword search might return nothing useful, because the actual help article is titled "ending your plan" and never uses the word "cancel." This is the exact gap semantic search closes — it matches on meaning, not exact wording, which makes it the right tool for searching business documents where the same idea gets phrased a dozen different ways across policies, tickets, and internal notes.
How Semantic Search Actually Works
Every piece of text — a document, a paragraph, a query — can be converted into an embedding: a list of numbers that represents its meaning in a way that lets you mathematically compare how similar two pieces of text are, regardless of whether they share any of the same words. Semantic search stores embeddings for your documents in a vector database, then for any incoming query, embeds that query the same way and retrieves the stored documents whose embeddings are mathematically closest to it.
Building the Pipeline in n8n
- Chunk your documents. Split long documents into smaller pieces (a paragraph or section at a time) — embedding an entire long document as one unit loses the specificity needed to match a narrow query.
- Generate embeddings. Pass each chunk through an embedding model (via the same LLM provider you're already using, or a dedicated embedding API) and store the resulting vector alongside the original text and its source.
- Store in a vector database. n8n's Vector Store nodes support Pinecone, Weaviate, Supabase/pgvector, and others — pick based on what you're already running rather than assuming one is inherently better.
- Query-time retrieval. An incoming query gets embedded the same way, and the vector store returns the closest-matching chunks by similarity.
- Optional: generate a direct answer. Pass the retrieved chunks to an LLM along with the original query, asking it to answer using only that retrieved context — this is the RAG pattern, turning search results into a direct answer.
Keeping the Index Current
A search assistant is only as good as how current its index is. Build the re-embedding step as its own triggered workflow — whenever a source document changes (a webhook from your CMS, wiki, or document store), re-embed just that document rather than the entire corpus, and update its entry in the vector database. This keeps the index accurate without the cost and delay of a full rebuild every time one document changes.
A Concrete Example: Internal Policy Search
A common first build: indexing HR policies, IT documentation, and internal how-to guides so employees can ask a question in plain language instead of digging through a wiki's folder structure. "What's our policy on expense reports over $500" retrieves the specific policy section discussing expense thresholds, even if the employee's phrasing doesn't match the document's exact wording — and an LLM layer on top can turn that retrieved section into a direct, quoted answer rather than making the employee read the whole document themselves.
Where Semantic Search Falls Short on Its Own
- Exact-match needs. Looking up a specific invoice number or an exact product SKU is a job for a database lookup, not semantic similarity — don't replace exact-match search where exact match is actually what's needed.
- Very short or ambiguous queries. A one-word query has less semantic content to match against, and can retrieve broader results than a more specific question would.
- Freshness. Semantic search only knows what's in the index — a document created five minutes ago and not yet re-embedded won't show up, regardless of how relevant it is.
Best Practices
- Chunk thoughtfully — too large loses specificity, too small loses context; a paragraph or a defined section is a reasonable starting point for most business documents.
- Store the source reference with every chunk, so results and generated answers can point back to the original document for verification.
- Automate re-indexing on document change rather than relying on manual reruns.
- Combine with metadata filtering (department, document type, date) where your vector database supports it, to narrow retrieval beyond similarity alone.
Handling Access Control So Search Results Respect Permissions
A semantic search index built by embedding every document in a shared knowledge base has a serious default failure mode: it returns results based purely on semantic relevance, with no awareness of who's allowed to see what. A search assistant that surfaces a snippet from an HR-only compensation document to anyone who asks a vaguely related question isn't a hypothetical risk — it's the default behavior unless access control is built into the retrieval step deliberately.
The fix is to filter at retrieval time, not just at index time. Store the permission scope (a team ID, a clearance level, an explicit list of allowed roles) as metadata alongside each embedded chunk, and apply that filter as part of the vector search query itself — not as a step afterward that tries to strip out results the requester shouldn't see, which is easy to get wrong and expensive to audit.
// Vector search filtered by the requester's access scope,
// not filtered after the fact
const results = await vectorStore.query({
vector: queryEmbedding,
topK: 5,
filter: {
allowedRoles: { $in: [requesterRole, 'all-staff'] },
},
});
Test this explicitly with at least two requester roles that should see different result sets for the same query — the same class of cross-boundary test recommended for multi-tenant AI automations generally, since an access-control gap in retrieval is invisible in normal functional testing and only shows up when someone specifically checks that the wrong content never surfaces to the wrong requester.
Re-index affected documents whenever a permission change happens upstream — someone leaving a team, a document's sensitivity being reclassified — rather than only updating permissions at the source system and assuming the search index picks it up automatically. A permission change that isn't reflected in the index's own metadata means the search assistant keeps serving results based on stale access rules until the next full re-index, which can be a meaningfully long exposure window if re-indexing only happens on a slow schedule.
Where the source system supports it, trigger re-indexing directly off its own permission-change events instead of a fixed schedule — that closes the exposure window to minutes instead of whatever your regular re-index cadence happens to be.
Getting Started
Start with one well-scoped document set — a single wiki space or policy folder — get chunking and retrieval quality right on that smaller set, and confirm results are actually useful before expanding to your entire document library. A large, poorly-chunked index is harder to fix retroactively than a small one built carefully from the start.
Written by Abhishek Patoliya, Senior n8n Developer & Automation Architect, Surat, India. Verified n8n Creator.