Case Study | Embeddable Vector Search for AI Applications

Building embeddable vector search for AI applications

AI/ML Vector Database C++ / Python Engineering Database Engineering

FiftyFive built an embeddable vector database combining a high-performance C++ HNSW index with persistent LSM storage, BM25 keyword search, and metadata filtering. The engagement covered AI/ML engineering, C++ and Python development, transactional recovery, and performance optimisation across the complete retrieval stack.

Build Your Platform With Us

The Client

Overview

The client needed a high-performance vector database that could bring semantic search and persistent data storage together in one system. Their requirement extended beyond similarity search alone: the engine also had to support keyword retrieval, metadata filtering, and reliable recovery. Critically, all of this had to run inside an application-owned environment rather than as an external hosted dependency.

The Need

Challenges

Building a vector database that pairs high-performance approximate nearest-neighbour search with durable, transactional storage meant solving retrieval quality and data integrity problems at the same time. The engineering work had to hold the HNSW graph and the persisted document store in agreement through every insert, delete, transaction, and restart.

Combining HNSW search with persistent storage required two systems with opposing design assumptions to operate as one. The HNSW graph is an in-memory structure optimised for traversal speed, while durable storage is optimised for write ordering and crash safety. The platform had to keep the vector index, the document store, and the transaction log consistent across inserts, deletes, transactions, and recovery, so that a search result always resolved to a document that genuinely existed in the persisted state.
Deleted and rolled-back documents posed a structural risk to the graph itself. HNSW does not support clean node removal without degrading connectivity, so naive deletion would have left orphaned edges, broken traversal paths, or corrupted neighbour lists. The engine needed a way to retire nodes and reverse aborted transactions without damaging the graph or returning results that pointed to documents no longer present.
Semantic similarity alone could not satisfy the retrieval requirement. Vector search captures meaning but misses exact terms, identifiers, and rare tokens, while lexical search captures precision but misses paraphrase. The platform had to run both retrieval paths over the same corpus and reconcile two scoring systems with entirely different numeric scales into a single, defensible ranked result.
Metadata filtering had to be applied efficiently rather than as an afterthought. Scoring vectors is the most expensive operation in the query path, so filters covering exact match, ranges, OR, NOT, and multi-key conditions needed to narrow the candidate set before vector scoring rather than discarding results after the cost had already been paid.
Advanced retrieval features had to coexist with architectural restraint. Reranking, query expansion, result diversification, vector compression, and optional IVF indexing each add configuration surface and failure modes. The system had to remain configurable and extensible without becoming rigid, and without forcing every deployment to carry capabilities it did not need.
C++ performance optimisation could not assume a uniform deployment environment. AVX2 and FMA instruction sets accelerate vector distance computation substantially, but they are not universally available. The engine needed to exploit them where present and fall back cleanly where absent, without maintaining divergent builds.
Recovery had to be safe under imperfect conditions. Index files can be left partially written or corrupted by an interrupted process, and the engine needed to detect that state, recover deterministically, and restart without silently serving a damaged index.

Our Approach

Solution

FiftyFive built the platform as a single embeddable engine rather than a service the application calls across a network boundary. The architecture layers a C++ HNSW vector index over an LSM-based persistent storage engine, with a hybrid query layer that runs semantic and lexical retrieval side by side. Transactional persistence, WAL-based recovery, and deterministic ID mapping keep the index and the document store aligned through failure and restart. Advanced retrieval — reranking, query expansion, diversification, compression, and optional IVF indexing — sits above this core as configurable capability rather than fixed behaviour.

Persistent LSM storage engine

FiftyFive built a log-structured merge storage engine to give the database durability and predictable write performance beneath the vector index.

The storage layer provides:

Write-ahead logging for durability, ordering guarantees, and crash recovery.
Memtables for in-memory write buffering ahead of flush.
SSTables for immutable on-disk segment storage.
Background compaction to reclaim space and control read amplification over time.
Transactional persistence so document writes commit or roll back as a unit.

C++ HNSW vector index

The HNSW index is the core search capability of the platform. FiftyFive implemented it in C++ as a graph-based structure for efficient traversal of high-dimensional vectors.

The index layer provides:

Configurable index construction parameters to tune build-time graph quality.
Configurable search parameters to trade recall against query latency per workload.
Deterministic HNSW node-to-document ID mapping, so every graph hit resolves reliably to its persisted document.
Ghost-node handling for documents that have been deleted or rolled back, preventing graph corruption.
Index rebuild mechanisms to restore graph health after accumulated churn.

Hybrid retrieval with metadata filter pushdown

FiftyFive combined semantic and lexical retrieval so that queries are answered on both meaning and exact terms, with filtering applied before the expensive scoring stage.

The query layer supports:

Okapi BM25 keyword search backed by a positional inverted index.
Hybrid retrieval that merges HNSW vector similarity with BM25 lexical scoring.
Metadata filter pushdown for exact, range, OR, NOT, and multi-key filters.
Multi-stage ranking across the combined candidate set.

Advanced retrieval and compression

FiftyFive engineered a configurable optimisation layer above core retrieval so applications can tune result quality and memory footprint independently.

The optimisation layer includes:

Maximal Marginal Relevance (MMR) for result diversification.
Query expansion to broaden recall on sparse or ambiguous queries.
Score calibration to reconcile scoring scales across retrieval methods.
Configurable reranking as a final ordering stage.
Product Quantization for compressed vector storage.
Optional IVF indexing for alternative retrieval and performance profiles.

Recovery, portability, and production controls

FiftyFive built the operational layer that allows the database to be embedded directly inside a production application.

This layer covers:

WAL-based recovery to restore committed state after interruption.
Corruption detection for partially written or damaged index files.
Three-tier index restoration for graduated recovery rather than all-or-nothing failure.
Portable and AVX2/FMA-optimised vector distance computation selected by runtime dispatch.
Telemetry for visibility into engine behaviour in production.
Thread-safety controls for concurrent application integration.

Technology

Tech Stack

Languages

C++Python

Vector Indexing

HNSWIVF

Storage Engine

LSMWALMemtablesSSTables

Lexical Retrieval

Okapi BM25Inverted Index

Retrieval Optimisation

Hybrid SearchMMRReranking

Compression

Product Quantization

Performance

AVX2FMARuntime Dispatch

Reliability

Transactional PersistenceCorruption Detection

Operations

TelemetryThread Safety

Experts

Team

AI
Engineer

Backend
Engineer

The Impact

Results

Project DurationApril 2026 - Ongoing

FiftyFive delivered one embeddable architecture for persistent vector search and hybrid information retrieval, providing semantic search, keyword retrieval, durable storage, and recovery without an external hosted dependency.

Embeddable Architecture

FiftyFive delivered vector search that runs inside the application environment, removing reliance on an externally hosted vector database service.

Hybrid Retrieval

The platform answers queries through combined HNSW semantic similarity and BM25 keyword matching, improving relevance across both conceptual and exact-term searches.

Durable Persistence

Transactional LSM storage with WAL-based recovery keeps vector and document data consistent through inserts, deletes, rollbacks, and restarts.

Configurable Performance

Tunable index parameters, vector compression, and optional IVF indexing let teams balance recall, latency, and memory per deployment.

Support

FAQs

What is a vector database and how does it work? +

A vector database stores numerical embeddings and retrieves results by similarity, using approximate nearest-neighbour indexes such as HNSW for semantic search, recommendations, and RAG.

What is HNSW indexing and why is it used for vector search? +

HNSW is a layered graph index that finds nearby high-dimensional vectors efficiently, with configurable construction and search parameters balancing recall, build time, and latency.

Should you use an embeddable vector database or a hosted one? +

Embeddable databases remove network hops and external dependencies while keeping data inside the application environment. Hosted databases reduce operational setup but add service dependency and latency.

What is hybrid search and why does it outperform vector-only search? +

Hybrid search combines semantic vector similarity with keyword retrieval such as BM25, improving relevance for both concepts and exact identifiers, codes, or rare terms.

How does metadata filtering improve vector search performance? +

Filter pushdown narrows candidates before expensive vector scoring, avoiding computation on results that would later be removed by exact, range, OR, NOT, or multi-key conditions.

What infrastructure does a RAG application need? +

Production RAG requires persistent document storage, vector indexing, keyword search, metadata filters, ranking, consistency guarantees, and crash recovery in addition to embeddings and an LLM.

How do vector databases handle deleted or rolled-back documents? +

HNSW deletions can use ghost-node marking to preserve graph connectivity, exclude deleted records from results, and later rebuild the index to reclaim structure safely.

What is vector compression and when should you use it? +

Product Quantization compresses embeddings into smaller representations, allowing larger indexes within the same memory budget with a controlled trade-off in retrieval precision.

How do you make vector search durable and crash-recoverable? +

Write-ahead logging records changes before application, while corruption detection and tiered restoration recover committed state and keep the index aligned with document storage.

Which industries use custom vector database development? +

Custom vector databases support AI, analytics, LegalTech, healthcare, financial services, and enterprise knowledge applications with proprietary data, residency, or latency requirements.

How much does it cost to build a custom vector database? +

Cost and timeline depend on index types, persistence guarantees, hybrid retrieval, filtering, compression, recovery behaviour, operational controls, and performance requirements.

How do you hire engineers for vector database and AI retrieval work? +

Vector database work needs systems engineers experienced in C++, index structures, persistence, and performance, alongside AI engineers who understand embeddings, ranking, retrieval, and evaluation.