Chapter 20: Capstone - Building a Semantic Search Engine for Documentation

Theoretical Foundations

The core challenge in building a semantic search engine for technical documentation is bridging the gap between human language and machine-understandable data. Traditional databases excel at exact matches, but they fail when a user asks, "How do I handle database migrations?" and the documentation uses the phrase "applying schema updates." To solve this, we must transform text into a mathematical representation that captures meaning, not just keywords. This is the domain of vector embeddings and vector databases.

The Geometry of Meaning: Vector Embeddings

At the heart of our semantic search engine lies the concept of vector embeddings. An embedding is a dense numerical vector (a list of floating-point numbers) that represents a piece of text in a high-dimensional space. Words or sentences with similar meanings are located closer to each other in this geometric space.

Analogy: Imagine a vast, multi-dimensional library. In a traditional library (keyword search), books are organized strictly by title and author. If you look for "The Art of Computer Programming," you must know the exact title. In our vector library (semantic search), books are placed on shelves based on their conceptual content. A book about "C# Memory Management" would be physically close to a book about "Garbage Collection in .NET," even if they share no common keywords. The distance between them represents their semantic similarity.

We use pre-trained models (like OpenAI's text-embedding-ada-002 or local models like all-MiniLM-L6-v2) to convert our technical documentation text into these vectors. The dimensionality (e.g., 1536 dimensions) defines the "resolution" of our semantic space. Higher dimensions allow for more nuanced distinctions but require more storage and computation.

The Retrieval-Augmented Generation (RAG) Pipeline

RAG is the architectural pattern that combines the vast knowledge of a Large Language Model (LLM) with the specific, up-to-date data stored in our vector database. It decouples the retrieval of information from the generation of answers.

The Process:

1. Ingestion: We chunk documents, generate embeddings for each chunk, and store them.
2. Querying: When a user asks a question, we generate an embedding for the query.
3. Retrieval: We calculate the cosine similarity between the query vector and all stored document vectors. The top \(K\) most similar vectors are retrieved.
4. Generation: The retrieved text chunks are injected into the LLM's context window as "grounding" data, and the LLM generates an answer based only on this provided context.

Why this matters: Without RAG, an LLM might hallucinate or rely on outdated training data. With RAG, we force the model to cite specific sections of our documentation, ensuring accuracy and traceability.

Vector Databases: The Engine of Semantic Search

A vector database is specialized software designed to store and query high-dimensional vectors efficiently. Traditional relational databases (like SQL Server) use B-trees for indexing, which are excellent for exact matches and ranges but inefficient for similarity searches in high-dimensional space.

Analogy: Searching for a similar vector in a standard database is like finding the "closest" person in a crowded city by checking every single person's height, weight, and age one by one (Brute Force). A vector database uses algorithms like HNSW (Hierarchical Navigable Small World) or IVF (Inverted File Index) to build a graph structure that allows for approximate nearest neighbor (ANN) search. This is like having a map that groups people by neighborhood; you only search within the most likely neighborhoods, making the search incredibly fast even with millions of vectors.

In our C# application, we will interact with a vector database (such as PostgreSQL with the pgvector extension, Redis, or a dedicated solution like Milvus) via EF Core or a dedicated client. The database must support:

1. Storage: Storing the vector alongside metadata (Document ID, Chunk Index, Text).
2. Indexing: Creating an ANN index (e.g., HNSW) to speed up queries.
3. Querying: Accepting a query vector and returning the nearest neighbors.

Memory Storage: Caching and Persistence

While the vector database handles the heavy lifting of semantic search, we need a lighter, faster storage layer for memory. This serves two purposes:

1. Caching: Storing the results of common queries to avoid re-computing embeddings and database lookups.
2. Session Management: Persisting user interactions (chat history) to maintain context across multiple turns in a conversation.

Analogy: Think of the vector database as a massive university library (slow to search but comprehensive). The memory storage (like Redis or a local SQLite database) is your personal notebook. When you ask a question, you first check your notebook (cache). If the answer isn't there, you go to the library (vector DB), get the answer, and write it down in your notebook for next time.

In C#, we leverage IDistributedCache or a lightweight ORM like LiteDB for this. The key is that this storage is ephemeral or session-based, whereas the vector database is the permanent source of truth for the documentation.

Architectural Flow and C# Integration

The theoretical architecture relies heavily on abstractions. We define interfaces for the Vector Database and Memory Storage. This allows us to swap implementations without changing the core logic.

Why Interfaces are Crucial:

• Vendor Independence: You might start with a local vector database (e.g., Qdrant running in Docker) for development but switch to a cloud provider (e.g., Azure Cognitive Search) for production.
• Testing: You can mock the vector database to test the RAG pipeline logic without needing a running database instance.

Concept from Previous Books: In Book 4, "Architecting Modern Web APIs with ASP.NET Core," we discussed the Repository Pattern and Dependency Injection (DI). We will apply those exact principles here. The IVectorStore interface will be injected into our SemanticSearchService, decoupling the search logic from the storage implementation.

Visualizing the Data Flow

The following diagram illustrates the lifecycle of a semantic search request, highlighting where vector storage and memory storage interact.

Hold "Ctrl" to enable pan & zoom

This diagram illustrates the dependency injection of the `IVectorStore` interface into the `SemanticSearchService`, visually mapping the data flow between vector storage and memory storage during a semantic search request.

Deep Dive: The Role of EF Core and Modern C# Features

While EF Core is traditionally associated with relational databases, in this capstone, we utilize it primarily for the Memory Storage and Metadata Management aspects.

1. EF Core for Metadata and Caching: We can use EF Core with a provider like SQLite to store the mapping between document chunks and their vector IDs. Furthermore, we can store the chat history (User prompts and AI responses) in a relational format. This allows us to leverage LINQ for complex queries on user interactions (e.g., "What did the user ask about migrations yesterday?").

2. Modern C# Features in AI Applications:

• IAsyncEnumerable<T>: When streaming responses from an LLM or processing large batches of documents for ingestion, IAsyncEnumerable is vital. It allows us to yield results as they become available, preventing blocking calls and reducing memory overhead.
• record Types: We will define our data transfer objects (DTOs) for embeddings and search results as record types. This provides immutability and value-based equality, which is crucial when comparing search results or caching data.
• Span<T> and Memory<T>: When dealing with raw vector data (arrays of floats), performance is key. Span<T> allows us to slice arrays without allocating new memory. This is critical when converting binary data from the database into the float arrays required by the ML.NET or ONNX runtime for similarity calculations.

3. Dependency Injection and Configuration: We will rely heavily on the IOptions<T> pattern (from Microsoft.Extensions.Options) to configure the vector database connection string and the embedding model settings. This keeps our codebase clean and adheres to the 12-factor app methodology.

Edge Cases and Architectural Implications

1. Chunking Strategy: The quality of semantic search is heavily dependent on how we split documents.

• Fixed-size chunking: Simple but risks cutting sentences in half, losing context.
• Semantic chunking: (e.g., splitting by paragraphs or semantic boundaries) Preserves context but requires more complex logic.
• Overlap: Adding an overlap between chunks ensures that a concept split across two chunks is still fully represented in at least one of them.

2. The "Cold Start" Problem: A vector database is useless until populated. We need a robust ingestion pipeline that can handle various file formats (PDF, DOCX, Markdown) and extract text reliably. We must also handle updates—if documentation changes, we need to update the vector store without re-indexing everything (handling "upserts").

3. Hybrid Search: Pure semantic search can sometimes be "fuzzy" and miss exact technical terms (e.g., specific error codes like HTTP 404). A robust system often implements Hybrid Search:

• Vector Search: Finds semantically similar concepts.
• Keyword Search (BM25): Finds exact matches for technical terms.
• Re-ranking: Combining both scores to produce the final result.

In C#, this might look like querying two different indices and merging the results using a weighted sum algorithm.

Theoretical Foundations

We are building a system that understands the intent behind a question, not just the keywords. By mapping text to vectors, we transform unstructured documentation into a structured geometric problem. Using a vector database allows us to solve this problem efficiently at scale. The RAG pattern ensures that the generative AI is grounded in facts, and the memory storage layer optimizes the user experience through caching. Throughout this, modern C# features like IAsyncEnumerable and Span<T> provide the performance and safety required for production-grade AI systems.

Basic Code Example

Here is a simple, self-contained "Hello World" example of a semantic search engine using EF Core and a local vector database (SQLite with the sqlite-vss extension). This example simulates a technical documentation search scenario.

// NuGet Packages Required:
// 1. Microsoft.EntityFrameworkCore.Sqlite
// 2. Microsoft.EntityFrameworkCore.Sqlite.Vss (Community fork or equivalent wrapper for Vector Search)
//    Note: For this example, we will mock the vector search logic if the extension isn't available,
//    but the code below assumes a standard EF Core setup with a hypothetical Vector property.

using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace SemanticSearchHelloWorld
{
    // 1. Define the Entity
    // Represents a chunk of technical documentation.
    public class DocumentationChunk
    {
        public int Id { get; set; }
        public string Content { get; set; } = string.Empty;

        // In a real vector database, this would be a specialized type (e.g., float[] or Vector).
        // For SQLite VSS, it's often stored as a blob or specific type.
        // We use string here for simplicity in this "Hello World" example, 
        // but we will simulate vector math in the service layer.
        public string? VectorEmbedding { get; set; } 
    }

    // 2. Define the DbContext
    public class DocsContext : DbContext
    {
        public DbSet<DocumentationChunk> DocumentationChunks { get; set; }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            // Using an in-memory database for this example to ensure it runs without file locks.
            // In production, use: optionsBuilder.UseSqlite("Data Source=docs.db");
            optionsBuilder.UseInMemoryDatabase("HelloWorldDocs");
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            // If using SQLite VSS, you would configure the vector column here:
            // modelBuilder.Entity<DocumentationChunk>()
            //     .Property(e => e.VectorEmbedding)
            //     .HasColumnType("VECTOR(3)"); // Assuming 3 dimensions for simplicity
        }
    }

    // 3. The Semantic Search Service
    // This service handles the "Intelligent" part: converting text to vectors and searching.
    public class SemanticSearchService
    {
        private readonly DocsContext _context;

        public SemanticSearchService(DocsContext context)
        {
            _context = context;
        }

        // Simulates generating a vector embedding (e.g., using Azure OpenAI or local ONNX model).
        // In a real app, this calls an LLM API.
        // Here, we calculate a simple "hash" vector for demonstration.
        private float[] GenerateEmbedding(string text)
        {
            // A real embedding would be a high-dimensional array (e.g., 1536 dimensions).
            // We will use 3 dimensions for this demo.
            // Logic: Sum of char codes modulo 10, normalized.
            var sum = text.Sum(c => (int)c);
            return new float[] 
            { 
                (sum % 10) / 10f, 
                ((sum / 10) % 10) / 10f, 
                ((sum / 100) % 10) / 10f 
            };
        }

        // Calculates Euclidean distance between two vectors.
        // In production, use optimized vector distance functions provided by the DB.
        private float CalculateDistance(float[] vecA, float[] vecB)
        {
            if (vecA.Length != vecB.Length) throw new ArgumentException("Vector dimensions must match.");

            float sumOfSquares = 0;
            for (int i = 0; i < vecA.Length; i++)
            {
                sumOfSquares += (vecA[i] - vecB[i]) * (vecA[i] - vecB[i]);
            }
            return (float)Math.Sqrt(sumOfSquares);
        }

        // Adds a document chunk to the database with its vector.
        public async Task AddDocumentAsync(string content)
        {
            var vector = GenerateEmbedding(content);

            // In a real Vector DB, we store the vector directly. 
            // Here we serialize it to string for storage in our simple entity.
            var chunk = new DocumentationChunk
            {
                Content = content,
                VectorEmbedding = string.Join(",", vector)
            };

            _context.DocumentationChunks.Add(chunk);
            await _context.SaveChangesAsync();
        }

        // Performs the semantic search.
        public async Task<List<(string Content, float Distance)>> SearchAsync(string query, int topK = 2)
        {
            // 1. Convert query to vector
            var queryVector = GenerateEmbedding(query);

            // 2. Retrieve all documents (In production, use DB-side vector search for performance)
            var allDocs = await _context.DocumentationChunks.ToListAsync();

            // 3. Calculate similarity (Distance) for each document
            var results = new List<(string Content, float Distance)>();

            foreach (var doc in allDocs)
            {
                if (doc.VectorEmbedding == null) continue;

                // Parse stored vector back to float array
                var docVector = doc.VectorEmbedding.Split(',')
                                                    .Select(float.Parse)
                                                    .ToArray();

                float distance = CalculateDistance(queryVector, docVector);

                // 4. Store result
                results.Add((doc.Content, distance));
            }

            // 5. Sort by smallest distance (closest match) and take top K
            return results.OrderBy(r => r.Distance).Take(topK).ToList();
        }
    }

    // 4. Main Program Execution
    class Program
    {
        static async Task Main(string[] args)
        {
            Console.WriteLine("=== Semantic Search Engine - Hello World ===\n");

            // Initialize Database
            using var context = new DocsContext();
            await context.Database.EnsureCreatedAsync();

            var searchService = new SemanticSearchService(context);

            // --- Populate Data ---
            Console.WriteLine("Indexing documentation chunks...");
            await searchService.AddDocumentAsync("EF Core is an object-relational mapper (O/RM).");
            await searchService.AddDocumentAsync("Vector databases store data as embeddings for semantic search.");
            await searchService.AddDocumentAsync("C# is a strongly-typed language developed by Microsoft.");
            await searchService.AddDocumentAsync("RAG stands for Retrieval-Augmented Generation.");
            Console.WriteLine("Indexing complete.\n");

            // --- Perform Search ---
            string userQuery = "What is EF Core?";
            Console.WriteLine($"Query: \"{userQuery}\"");

            var results = await searchService.SearchAsync(userQuery);

            Console.WriteLine("\nTop Relevant Results:");
            foreach (var result in results)
            {
                // Note: Lower distance means more similar
                Console.WriteLine($"[Dist: {result.Distance:F4}] {result.Content}");
            }
        }
    }
}

Detailed Explanation

1. The Entity Model (DocumentationChunk)

In a semantic search engine, data is not stored as plain text alone. It is stored as "chunks" of text paired with their vector representations.

• Id: Standard primary key for database tracking.
• Content: The actual text snippet (e.g., a paragraph from documentation).
• VectorEmbedding: A numerical representation of the text. In this example, we store it as a comma-separated string (e.g., "0.5,0.2,0.1") to simulate how a vector database stores high-dimensional arrays (blobs). In production systems using PostgreSQL pgvector or SQLite vss, this column would be typed specifically for vector operations.

2. The Database Context (DocsContext)

We use Entity Framework Core to abstract the database interactions.

• OnConfiguring: For this "Hello World" example, we use UseInMemoryDatabase. This ensures the code runs immediately without needing to install SQLite or manage file paths. However, vector search capabilities (like cosine similarity) are strictly database-engine specific. In a real scenario, you would swap this for UseSqlite (with the VSS extension loaded) or UseNpgsql (with pgvector).
• OnModelCreating: This is where you would define the schema for vector columns if using a real vector database extension.

3. The Semantic Search Service (SemanticSearchService)

This is the core logic layer. It bridges the gap between raw text and mathematical vector space.

• GenerateEmbedding:

  • Real World: You would call an API (like Azure OpenAI text-embedding-ada-002) or run a local ONNX model (like all-MiniLM-L6-v2). These models convert text into arrays of floats (e.g., 1536 dimensions).
  • This Example: To keep the code self-contained and runnable without API keys, we simulate an embedding. We calculate a deterministic "vector" based on the character sum of the text. This allows us to demonstrate the mechanism of vector comparison without external dependencies.

• CalculateDistance:

  • Real World: Vector databases perform this calculation highly optimized (often using C++ or GPU acceleration). You rarely calculate this in C# code for production search; you push the calculation to the database engine (e.g., ORDER BY embedding <-> query_embedding LIMIT 5).
  • This Example: We implement Euclidean Distance manually in C#. This measures the straight-line distance between two points in vector space. A lower distance means the texts are semantically closer.

• AddDocumentAsync:

  1. Takes a string.
  2. Generates the vector.
  3. Serializes the vector to a string for storage.
  4. Saves to EF Core.

• SearchAsync:

  1. Query Vectorization: Converts the user's natural language query into the same vector space as the documents.
  2. Retrieval: Fetches all documents (in a real system, this is the "Approximate Nearest Neighbor" search).
  3. Ranking: Calculates the distance between the query vector and every document vector.
  4. Sorting: Orders by distance (ascending) to find the closest matches.
  5. Top-K: Returns only the best results.

4. Execution (Program)

1. Initialization: Creates the database context.
2. Indexing: We add four distinct chunks of text. Notice that two are related to EF Core, one to Vector DBs, and one to C#.
3. Querying: We ask "What is EF Core?".
4. Result: The system calculates the vector distance. Even though the query string "What is EF Core?" is not identical to "EF Core is an object-relational mapper...", their simulated vectors will be mathematically closer to each other than to "C# is a strongly-typed language...", demonstrating semantic matching.

Common Pitfalls

1. Treating Vectors as Strings in Production:

   • The Mistake: Storing vectors as comma-separated strings (like we did here) in a production relational database.
   • Why it's bad: It is incredibly inefficient for calculation. You cannot use SQL WHERE or ORDER BY clauses effectively on string representations of arrays. You would have to parse strings in memory (like our loop), which is slow and doesn't scale.
   • The Fix: Use a dedicated vector database (Pinecone, Weaviate) or a SQL extension (PostgreSQL pgvector, SQLite vss) that stores vectors as native binary arrays and supports vector-specific indexing (HNSW, IVFFlat).

2. Chunk Size Mismatch:

   • The Mistake: Feeding entire 50-page PDFs into an embedding model.
   • Why it's bad: Embedding models have a "context window" (token limit). Furthermore, if a document contains one relevant fact and 100 irrelevant pages, the resulting vector will be "diluted" and lose the specific semantic signal of the fact.
   • The Fix: Implement Chunking. Split documents into smaller, overlapping windows (e.g., 256 or 512 tokens) before generating embeddings. This ensures high semantic density per vector.

3. Ignoring Distance Metrics:

   • The Mistake: Using Euclidean distance when the embedding model was trained on Cosine similarity.
   • Why it's bad: Different distance metrics yield different rankings. Most modern embedding models (like OpenAI's) are normalized (length of vector = 1). For normalized vectors, Cosine Similarity and Euclidean Distance are mathematically related, but for non-normalized vectors, the choice matters significantly.
   • The Fix: Always use the distance metric recommended by your embedding model provider. If using OpenAI, prefer Cosine Similarity.

Visualizing the Vector Space

The following diagram illustrates how the text chunks are projected into a 3-dimensional vector space (simplified for this example). The query "What is EF Core?" is plotted, and the system finds the nearest neighbor.

Hold "Ctrl" to enable pan & zoom

A diagram showing a 3D vector space where the query What is EF Core? is plotted as a point, surrounded by other points representing text chunks, with an arrow indicating the system identifying the nearest neighbor to retrieve relevant information.

The chapter continues with advanced code, exercises and solutions with analysis, you can find them on the ebook on Leanpub.com or Amazon

---

Loading knowledge check...

---

---

Code License: All code examples are released under the MIT License. Github repo.

Content Copyright: Copyright © 2026 Edgar Milvus | Privacy & Cookie Policy. All rights reserved.

All textual explanations, original diagrams, and illustrations are the intellectual property of the author. To support the maintenance of this site via AdSense, please read this content exclusively online. Copying, redistribution, or reproduction is strictly prohibited.