Semantic Search from Scratch: Build Search That Understands Meaning

Traditional keyword search matches words, not meaning — so a search for “affordable notebook computer” can miss a perfect result that says “cheap laptop,” simply because no words overlap. Semantic search fixes this by comparing meaning instead of exact tokens, understanding that both phrases are asking for the same thing. In this guide you’ll build semantic search from scratch in Python, starting with the core idea — embeddings — and ending with a version you can scale to millions of documents.

How semantic search works: embeddings

The trick behind semantic search is turning text into numbers in a way that captures meaning. An embedding is a vector — a list of numbers, often a few hundred or a thousand of them — that represents a piece of text as a point in high-dimensional space.

The magic is in how those points are arranged. An embedding model is trained so that texts with similar meanings end up close together, and unrelated texts end up far apart. “Cheap laptop” and “affordable notebook computer” land near each other; “photosynthesis in plants” lands somewhere else entirely.

Once your text lives in this space, search becomes a geometry problem: embed the user’s query, then find the document vectors closest to it. No keyword overlap required — proximity is relevance.

Measuring similarity: cosine similarity

To find the “closest” vectors, we need a way to measure how similar two vectors are. The standard choice for text embeddings is cosine similarity, which measures the angle between two vectors rather than the distance between their tips.

Why the angle? Because for text embeddings, direction encodes meaning while magnitude is mostly noise. Two vectors pointing the same way are semantically similar even if one is “longer.” Cosine similarity produces a score from -1 to 1:

• 1 — the vectors point in exactly the same direction (nearly identical meaning).
• 0 — the vectors are perpendicular (unrelated).
• -1 — the vectors point in opposite directions.

The formula is the dot product of the two vectors divided by the product of their magnitudes:

cosine_similarity(A, B) = (A · B) / (‖A‖ × ‖B‖)

Setting up

We’ll use sentence-transformers to generate embeddings — it downloads a small, high-quality model that runs locally, so there’s no API key or network call once installed. We’ll use numpy for the math.

pip install sentence-transformers numpy

Load a model. all-MiniLM-L6-v2 is a great starting point: it’s small, fast, and produces 384-dimensional embeddings that perform well for general-purpose search.

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")

Building a tiny corpus

Let’s create a small set of documents to search over. In a real application these would be product descriptions, support articles, or paragraphs from your knowledge base.

documents = [
    "How to reset your account password",
    "The best budget laptops for students in 2026",
    "A beginner's guide to composting at home",
    "Troubleshooting slow Wi-Fi and connection drops",
    "Affordable notebook computers with great battery life",
    "Growing tomatoes in small urban gardens",
]

Embedding the documents

Now we turn every document into a vector. The model processes them in one batch and returns a matrix — one row per document, each row a 384-number embedding.

doc_embeddings = model.encode(documents)
print(doc_embeddings.shape)   # (6, 384)

This step is a one-time cost. In production you compute these embeddings once when a document is added or updated, store them, and reuse them for every future search.

Implementing cosine similarity from scratch

Rather than reach for a library helper, let’s implement cosine similarity directly with numpy so the mechanics are clear. The function compares one query vector against every document vector at once:

def cosine_similarity(query_vec, doc_matrix):
    # Dot product of the query with every document row.
    dots = doc_matrix @ query_vec

    # Magnitudes (L2 norms).
    query_norm = np.linalg.norm(query_vec)
    doc_norms = np.linalg.norm(doc_matrix, axis=1)

    # Divide element-wise to get a similarity score per document.
    return dots / (doc_norms * query_norm)

The key line is doc_matrix @ query_vec — a single matrix-vector multiplication that computes the dot product of the query against all documents simultaneously. Vectorizing like this is what keeps semantic search fast even over large corpora.

Putting it together: the search function

Now we can write the search itself. Embed the query, score it against every document, and return the top matches sorted by similarity:

def search(query, top_k=3):
    query_vec = model.encode(query)
    scores = cosine_similarity(query_vec, doc_embeddings)

    # Indices of the highest-scoring documents, best first.
    top_indices = np.argsort(scores)[::-1][:top_k]

    return [(documents[i], float(scores[i])) for i in top_indices]

Let’s try it with a query that shares almost no words with any document:

for doc, score in search("cheap computer for schoolwork"):
    print(f"{score:.3f}  {doc}")

The output ranks the two laptop documents at the top, even though neither contains the words “cheap,” “computer,” or “schoolwork”:

0.612  Affordable notebook computers with great battery life
0.598  The best budget laptops for students in 2026
0.104  Troubleshooting slow Wi-Fi and connection drops

That gap between the top two results and everything else is semantic search working exactly as intended. A keyword search for “cheap computer for schoolwork” would have returned nothing, because no document contains those exact words. The embedding model understood that “affordable notebook” and “budget laptops for students” mean the same thing.

A performance shortcut: normalize once

If you normalize every embedding to unit length up front, cosine similarity collapses into a plain dot product — the division disappears because every magnitude is 1. Most embedding libraries can do this for you:

doc_embeddings = model.encode(documents, normalize_embeddings=True)

def search_fast(query, top_k=3):
    query_vec = model.encode(query, normalize_embeddings=True)
    scores = doc_embeddings @ query_vec          # dot product == cosine now
    top_indices = np.argsort(scores)[::-1][:top_k]
    return [(documents[i], float(scores[i])) for i in top_indices]

For a handful of documents the difference is negligible, but at scale, replacing a division-heavy computation with a single matrix multiply is a meaningful win.

Scaling semantic search with a vector database

Our from-scratch version compares the query against every document — a linear scan. That’s perfect for learning and fine for thousands of documents. But at a million documents, scanning all of them for every search becomes too slow.

This is the problem vector databases solve. Tools like FAISS, Chroma, Qdrant, Pinecone, and pgvector (a Postgres extension) store your embeddings in specialized index structures and use approximate nearest neighbor (ANN) algorithms to find the closest vectors without comparing against all of them. You trade a tiny amount of accuracy for enormous speed — searching millions of vectors in milliseconds.

The good news: everything you’ve learned still applies. A vector database doesn’t change the concepts — you still embed your documents, embed the query, and rank by similarity. It just handles storage and the nearest-neighbor lookup efficiently. Here’s the same idea using FAISS:

import faiss

# Build an index for 384-dimensional vectors using inner product.
index = faiss.IndexFlatIP(384)
index.add(doc_embeddings)          # embeddings normalized as above

# Search: returns the top-k scores and their indices.
query_vec = model.encode("cheap computer for schoolwork",
                         normalize_embeddings=True)
scores, indices = index.search(np.array([query_vec]), k=3)

for score, idx in zip(scores[0], indices[0]):
    print(f"{score:.3f}  {documents[idx]}")

Where to go from here

You’ve built a semantic search engine from its foundations: embeddings to capture meaning, cosine similarity to measure it, and a ranked search over a real corpus — then seen how a vector database scales the exact same idea to production.

A few natural next steps once you’re comfortable with the basics:

• Chunk long documents. Embedding a whole article as one vector blurs its meaning. Split long text into passages and embed each one so searches can pinpoint the relevant section.
• Combine with keyword search. Hybrid search blends semantic scores with traditional keyword matching (BM25) to get the best of both — great for queries that include exact terms like product codes.
• Try larger models. Bigger embedding models capture more nuance at the cost of speed and storage. Benchmark a few against your own data.
• Feed it into RAG. Semantic search is the retrieval half of Retrieval-Augmented Generation — the technique that grounds an AI assistant on your own documents. For a real-world example, see how I added token and LLM cost estimation to Microsoft GraphRAG’s indexing pipeline.

The concepts don’t get more complicated as you scale — you’re always embedding text, measuring similarity, and ranking results. Master that core loop, and you can build semantic search that actually understands what people are looking for. If you enjoy building tools like this from scratch, you might also like my write-up on building and publishing a small Python tool.