Semantic Document Search: How It Works and How to Build It with Embeddings
How semantic document search works, plus a Python build: extract, chunk, embed, and search, with a tested keyword, semantic, and hybrid comparison.

A user types "how do I get out of the contract early" into a search box. The contract says "terminate this Agreement for convenience." A keyword search finds nothing, because the two sentences share no important words. A semantic search finds the clause, because it compares meaning instead of spelling.
That difference is the reason teams add semantic search to document collections such as contracts, policies, claims, and invoices. It matters most where the people searching do not use the words the authors used. For examples across industries, see DeepRead's overview of document processing applications.
This guide explains how semantic document search works, then builds a small version in Python. The code is tested, and the output below is from a real run. It also shows where semantic search fails, because the failures decide how you should design the system.
The Short Answer
Semantic document search turns text into vectors, called embeddings, so that passages with similar meaning sit close together. To search, you embed the query with the same model and return the nearest passages. Four stages make it work:
- Extract clean text from each document, using OCR for scans (see DeepRead's OCR software overview).
- Split the text into chunks.
- Embed each chunk and store the vectors.
- Embed the query, rank chunks by similarity, and return the top results.
Most production systems add a fifth step: combine semantic ranking with keyword ranking. That combination is called hybrid search.
How Semantic Search Differs From Keyword Search
Keyword search, usually BM25, scores a document by the query words it contains. It is fast, exact, and predictable. It is also blind to synonyms. "Bills" does not match "invoices," and "leaked" does not match "unauthorized disclosure."
Semantic search scores by closeness in vector space. At search time, the query is embedded into the same vector space and the nearest corpus embeddings are returned. It handles paraphrase well. It is weaker on exact strings such as part numbers, error codes, and names, where keyword search is the better tool.
Fields such as dates and totals are better stored as structured data and filtered exactly. For that approach, see DeepRead's guide to structured data extraction. Neither search method wins everywhere, which is why the two are often combined. The test later in this guide measures all three on the same queries.
Step 1: Get Clean Text Out of the Documents
Search quality starts before any embedding. If a scanned PDF becomes garbled text, the vectors encode the garble. Digital PDFs usually give clean text directly. Scans and photos need OCR first.
Extraction also decides what you can show next to a result. A search hit is more useful when it carries its page number and the source text, so the reader can verify it. For PDF extraction in general, see DeepRead's guide to extracting data from any PDF.
Keep the page number with each piece of text from the start. Adding it later means re-processing every document.
Step 2: Split the Text Into Chunks
An embedding represents a whole piece of text with one vector, so the size of the piece matters. A whole contract in one vector blurs every clause together. A single sentence may lose the context that makes it meaningful. Chunks of a paragraph or a few paragraphs are a common middle ground.
The function in the next section splits on blank lines and merges short paragraphs up to a word limit. It is deliberately simple. For splitting that follows topic changes instead of length, see DeepRead's guide to semantic chunking for RAG.
Step 3: Embed and Index
The example uses ten contract clauses as its corpus. These are written for this guide, not taken from a real agreement.
CHUNKS = [
"Either party may terminate this Agreement for convenience by giving the other party thirty (30) days prior written notice.",
"Supplier shall indemnify Customer against losses arising from any unauthorized disclosure of Customer Data caused by Supplier's failure to maintain reasonable safeguards.",
"Customer shall pay all undisputed invoices within forty-five (45) days of the invoice date. Late balances accrue interest at 1.5% per month.",
"This Agreement is governed by the laws of the State of Delaware, and the parties submit to the exclusive jurisdiction of its courts.",
"Supplier will deliver the Services in accordance with the schedule in Exhibit A and will notify Customer promptly of any anticipated delay.",
"Each party shall keep the other party's Confidential Information secret and use it only to perform this Agreement.",
"Supplier warrants that the Services will be performed in a professional manner by qualified personnel.",
"Neither party is liable for failure to perform caused by events beyond its reasonable control, including natural disasters and war.",
"Supplier's total liability under this Agreement shall not exceed the fees paid by Customer in the twelve (12) months preceding the claim.",
"Customer may not assign this Agreement without the prior written consent of Supplier.",
]
The search module has four parts: a chunker, an embed function, keyword search with BM25, and semantic search with cosine similarity. A hybrid method merges the two rankings with reciprocal rank fusion (RRF).
import numpy as np
import spacy
from rank_bm25 import BM25Okapi
nlp = spacy.load("en_core_web_md", disable=["ner", "parser"])
def chunk_paragraphs(text, max_words=120):
"""Split on blank lines, then merge short paragraphs up to max_words."""
chunks, current = [], []
for para in [p.strip() for p in text.split("\n\n") if p.strip()]:
if current and sum(len(p.split()) for p in current) + len(para.split()) > max_words:
chunks.append(" ".join(current))
current = []
current.append(para)
if current:
chunks.append(" ".join(current))
return chunks
def tokens(text):
return [t.lower_ for t in nlp(text) if t.is_alpha and not t.is_stop]
def embed(texts):
"""Mean of word vectors, unit length. Swap this for a sentence model."""
rows = []
for text in texts:
vecs = [t.vector for t in nlp(text) if t.is_alpha and not t.is_stop and t.has_vector]
v = np.mean(vecs, axis=0) if vecs else np.zeros(nlp.vocab.vectors.shape[1])
rows.append(v / (np.linalg.norm(v) or 1.0))
return np.vstack(rows)
class Index:
def __init__(self, chunks):
self.chunks = chunks
self.vecs = embed(chunks)
self.bm25 = BM25Okapi([tokens(c) for c in chunks])
def semantic(self, query, k=3):
scores = self.vecs @ embed([query])[0]
return [(int(i), float(scores[i])) for i in np.argsort(-scores)[:k] if scores[i] > 0]
def keyword(self, query, k=3):
scores = self.bm25.get_scores(tokens(query))
return [(int(i), float(scores[i])) for i in np.argsort(-scores)[:k] if scores[i] > 0]
def hybrid(self, query, k=3, c=60):
fused = {}
for ranking in (self.semantic(query, len(self.chunks)),
self.keyword(query, len(self.chunks))):
for rank, (i, _) in enumerate(ranking):
fused[i] = fused.get(i, 0.0) + 1.0 / (c + rank + 1)
return sorted(fused.items(), key=lambda p: -p[1])[:k]
Three design choices are worth noting.
- embed is the only model-specific code. This example averages spaCy's en_core_web_md word vectors, so it runs on a laptop with no model service. It is a simple embedding. A production system would replace the body with a call to a sentence embedding model or an embeddings API, and nothing else changes.
- Vectors are unit length. With normalized vectors, a dot product equals cosine similarity, so ranking is one matrix multiplication.
- Both searches drop zero scores. If no query word matches, or the query is empty, they return nothing instead of an arbitrary first chunk.
RRF merges by rank instead of by score. Elastic describes it as unsupervised and not requiring score calibration, which helps because BM25 scores and cosine similarities are not on the same scale. Each ranking adds 1 / (k + rank) to a chunk, with k set to 60, the value the original paper suggested.
Step 4: Search and Compare
The test runs eight queries against the ten clauses. Each query has one correct clause. Five of the eight deliberately avoid the clause's own wording.
from corpus import CHUNKS
from semsearch import Index, chunk_paragraphs
text = "\n\n".join(CHUNKS[:4])
print("chunks from 4 paragraphs at max_words=60:", len(chunk_paragraphs(text, max_words=60)))
index = Index(CHUNKS)
QUERIES = [
("how do I get out of the contract early", 0),
("who pays if customer data is leaked", 1),
("when are bills due", 2),
("what is the payment deadline for invoices", 2),
("which state law applies", 3),
("what if a hurricane stops the work", 7),
("is there a cap on damages", 8),
("can I transfer this agreement to another company", 9),
]
methods = (("keyword", index.keyword), ("semantic", index.semantic), ("hybrid", index.hybrid))
top1 = {name: 0 for name, _ in methods}
top3 = {name: 0 for name, _ in methods}
print(f"{'query':50} {'want':>4} {'keyword':>8} {'semantic':>9} {'hybrid':>7}")
for query, want in QUERIES:
cells = []
for name, fn in methods:
ranked = [i for i, _ in fn(query, k=3)]
top1[name] += bool(ranked) and ranked[0] == want
top3[name] += want in ranked
cells.append(str(ranked[0]) if ranked else "none")
print(f"{query:50} {want:>4} {cells[0]:>8} {cells[1]:>9} {cells[2]:>7}")
print("top-1 hits:", top1)
print("top-3 hits:", top3)
This is the printed output from a real run:
chunks from 4 paragraphs at max_words=60: 2
query want keyword semantic hybrid
how do I get out of the contract early 0 none 0 0
who pays if customer data is leaked 1 1 1 1
when are bills due 2 none 3 3
what is the payment deadline for invoices 2 2 2 2
which state law applies 3 3 3 3
what if a hurricane stops the work 7 none 7 7
is there a cap on damages 8 none 8 8
can I transfer this agreement to another company 9 none 0 0
top-1 hits: {'keyword': 3, 'semantic': 6, 'hybrid': 6}
top-3 hits: {'keyword': 3, 'semantic': 8, 'hybrid': 8}
What the Test Shows
Keyword search found the right clause first on 3 of 8 queries, and returned nothing at all on five. Those five queries share no meaningful word with the answer. "Hurricane" does not appear in a clause about "events beyond its reasonable control," and "cap on damages" does not appear in the liability clause.
Semantic search put the right clause first on 6 of 8 queries, and in the top three on all 8. That is the practical value: it surfaced the right clause for paraphrased questions that keyword search could not answer.
Two results deserve attention.
- Semantic search still missed twice at rank one. "When are bills due" returned the governing-law clause first, and "transfer this agreement to another company" returned the termination clause first. The correct clauses were in the top three. A simple averaged embedding is weak, and a retrieval-trained sentence model should do better, but no embedding is exact.
- Hybrid did not fix those misses. Keyword search had nothing to add on those two queries, so RRF could not correct them. Hybrid helps when each method gets some queries right. It is not a repair for a weak embedding.
These numbers come from ten clauses and eight queries. They show the behavior, not a benchmark. Measure on your own documents before you decide.
Choosing an Embedding Model
Four questions narrow the choice.
- Are queries short and documents long? That is asymmetric search, a short question matched to a longer answering passage. Sentence Transformers documentation distinguishes symmetric from asymmetric search, and the distinction affects which model suits your task.
- What languages do your documents use? Check the model's language coverage before you embed anything.
- How long are your chunks? Each model has a maximum input length, and longer text is usually truncated.
- Where will it run? A hosted embeddings API is simple. A local model keeps text on your own machines.
Use the same model for the corpus and for queries. Vectors from two models are not comparable. If you change models later, re-embed everything.
Scaling Past a Small Corpus
The example holds every vector in memory and scores them all. For small collections, that is fine. The Sentence Transformers documentation says a manual implementation like this works for corpora up to about one million entries. Beyond that, use a vector index or a search engine that supports approximate nearest neighbor search.
Scale adds three needs the example skips.
- Metadata filters. Restrict a search to one customer, one document type, or one date range before ranking.
- Reindexing. Documents change. Store a document ID and a version with each chunk so you can replace only what changed.
- A reranking step. Retrieve 20 or 50 candidates, then reorder them with a model that reads the query and each candidate together. This addresses the near-miss ranking seen in the test.
Where DeepRead Fits
DeepRead sits at the front of this pipeline. It is a developer-first OCR and document AI API that turns PDFs, scans, and images into structured data. Each extracted field comes with its page, bounding box, and source text, which are the pieces a search result needs to point back to the original page. Large batches often arrive as stacked files that need splitting and classifying first, which DeepRead's guide to automated document indexing addresses.
The Free plan gives 2,000 pages a month with no card, up to 50 pages and 15 MB per document. Standard has no monthly page limit and charges per 1,000 pages. DeepRead is hosted only and defaults to US East. It has no self-hosted option and no BAA on any plan, so check those points against your data before you send documents.
DeepRead does not embed or index your text. Pair it with an embedding model and a vector store of your choice.
Common Mistakes
- Embedding bad text. Garbled OCR output produces garbled vectors. Check extraction before you tune search.
- Dropping page numbers. Results without a page reference are hard to trust and hard to verify.
- Treating the similarity score as confidence. Nearby candidates often score close together, and the top score is not a probability.
- Using semantic search for exact identifiers. Invoice numbers and error codes need keyword matching. Use hybrid search.
- Mixing embedding models. Queries and documents must use the same model.
- Skipping evaluation. Write 20 to 50 real queries with known answers and measure top-1 and top-3 before launch.
- Chunking by length alone. A clause split in half can fail both halves.
Conclusion
Semantic document search finds passages by meaning, so it answers the questions people actually ask. A tested comparison on contract clauses showed keyword search returning nothing for five of eight paraphrased queries, while semantic search found the right clause in the top three every time. It also showed that semantic search misses at rank one, and that hybrid search helps only when the methods complement each other. Start with clean extraction and sensible chunks, keep page numbers, add keyword matching for exact terms, and measure on your own queries.
FAQ
What is semantic document search?
It is a search method that ranks document passages by similarity of meaning to the query instead of by shared words. It uses embeddings, which are vectors that place similar text close together.
How is semantic search different from keyword search?
Keyword search scores by the query words a document contains. Semantic search scores by closeness in meaning, so it can match "get out of the contract" to "terminate this Agreement."
Do I need a vector database?
Not for a small collection. You can hold the vectors in memory and compute similarity directly. A vector index becomes useful as the collection grows, or when you need filters and fast approximate search.
What is hybrid search?
Hybrid search runs keyword and semantic search together and merges the rankings, often with reciprocal rank fusion. It helps when some queries need exact terms and others need meaning.
Which embedding model should I use?
Choose a model trained for retrieval if your queries are short and your passages are longer, and check its language coverage and input length. Test two or three models on your own queries.
Can I run semantic search on scanned documents?
Yes, after OCR. Search quality depends on how clean the extracted text is, so check the OCR output first.
More articles

How to Parse a PDF File: Methods, Libraries, and Tools
Parse a PDF file in Python with tested code: check the text layer, extract text and tables, OCR scanned pages, and compare pypdf, pdfplumber, and PyMuPDF

Unstructured Data Extraction Tools: How to Choose by Input Type
Choose unstructured data extraction tools by input: PDFs, scans, emails, images, and long reports. Tested Python code, honest limits, and when an API fits.