Vector Search Doesn't Need the Cloud
Your data doesn't need someone else's server to be searchable.
Most vector search today ships your notes to a cloud service. Your journal entries, medical records, research notes - they leave your device to be embedded, indexed, and queried elsewhere. It works, but there's a dependency: your ability to search your own information requires someone else's infrastructure.
That feels backwards.
Local-First AI Infrastructure#
There's a growing movement around local-first software - applications where your data lives on your device, works offline, and doesn't require server permission to function. CRDTs, sync engines, collaborative editors that work without central servers.
But AI features? Still cloud-first. Want semantic search? Call an API. Want vector similarity? Ship to a database service. Want embeddings? Send your text to a cloud provider.
I wanted to test whether this had to be true. Could you build AI-powered semantic search that follows local-first principles? Data on device, works offline, no server dependencies, optional sync?
Mostly yes, with caveats I got wrong the first time I wrote this post. The storage layer works: IndexedDB will hold your vectors and compute cosine similarity over them without anything leaving the machine. The part I oversold was the indexing.
Why This Matters#
Privacy by default. Your notes stay on your device — not "encrypted in transit," not "we don't look at your data." They never leave.
The application works offline. On planes, in basements, when your internet dies. It doesn't degrade because there's no server to lose connection to.
No vendor lock-in. No API keys, no usage limits, no pricing changes, no service shutdowns. Your data, your search, your infrastructure.
And speed. A network round-trip costs 50-200ms; a local query over a few thousand vectors came back in single-digit milliseconds in the demo. I never wrote a proper benchmark, so treat that as an observation from the browser console rather than a measurement you should plan against.
This aligns with what Ink & Switch calls the seven ideals of local-first software: no spinners, work doesn't require permission, fast operations, longevity, privacy, user control.
The Technical Reality#
IndexedDB wasn't designed for vector similarity search.
Vector databases use specialized indices (HNSW, IVF) and hardware acceleration (SIMD, GPU). IndexedDB gives you document storage and B-tree indices. Not the same thing.
Brute force doesn't scale. Comparing every query against 10,000 vectors in JavaScript? Too slow. You need to narrow the search space before computing similarity.
My plan was two filtering layers: a magnitude pre-filter to eliminate obvious non-matches, then LSH bucketing to group similar vectors, then exact cosine similarity on whatever survived. Locality-Sensitive Hashing projects vectors onto random hyperplanes and turns the signs into a binary hash, so unlike normal hashing, similar items land in the same bucket.
That's what I described the first time I wrote this post. It isn't what the code does, and two of the three layers don't work the way I claimed.
The layers never compose. QueryStrategy in store.ts is a switch, not a pipeline: you query by one index (magnitude, hash, text, vector, or timestamp), then run cosine over everything it returns. Magnitude and LSH are alternatives. They never run in sequence, so the "10,000 down to ~200 candidates" figure describes an architecture I sketched and didn't build.
The magnitude filter is a no-op on normalized embeddings. Cosine similarity divides out both magnitudes, so length carries no similarity signal to begin with. Worse, the default embedding model is text-embedding-ada-002, which returns unit-normalized vectors — every magnitude is 1.0. With the default tolerance of 1e-7, the filter's range is [1 - 1e-7, 1 + 1e-7], which either matches the entire store or drops valid neighbors to float32 rounding. That tolerance value should have told me something: nobody puts a ±0.00001% band around a quantity they believe is discriminative.
The LSH bucketing is too fine-grained. The default k is 50, so computeHash produces a 50-bit string. Across 10,000 vectors that's effectively a unique bucket per vector, and an exact-match bucket lookup returns near-nothing rather than a useful candidate set. LSH needs either far fewer bits per hash or multiple hash tables queried in parallel. Mine had one table and 50 bits.
What actually ran, then, was closer to brute force over whatever a single index returned. That works fine at small scale, which is why the demo felt fast and why I didn't catch any of this.
What I Built#
A VectorStore that wraps IndexedDB and handles the complexity - storing vectors with pre-computed metadata, querying by different indices, caching frequent searches, extracting top-K results.
For developers, a React hook that abstracts everything:
const { insertVectors, fetchVectors } = useVectorStoreHook()
You don't think about IndexedDB transactions or LSH parameters. Just insert and search.
What I Got Wrong About the Browser#
The browser is more capable than we assume. We're still treating it like a thin client when it's a legitimate compute platform. IndexedDB, Float32Array, Web APIs - the primitives are there.
JavaScript wasn't the bottleneck. The IndexedDB transaction dominated each query; the cosine math barely registered. Whatever slowness I hit was structural, not a property of the language.
Local-first doesn't mean solo. You could extend this with background sync to a cloud vector DB. Local-first with optional collaboration. The data lives on your device but can sync when you want it to.
Developer experience compounds. Hiding IndexedDB's awkward transaction model behind a clean hook made the difference between "possible" and "practical." Good abstractions multiply adoption.
The Trade-offs#
This isn't replacing cloud vector databases for production systems. It can't handle millions of vectors, doesn't support real-time multi-user updates, and until the index layer is fixed it's doing linear work per query.
For personal applications, privacy-sensitive data, and offline-first tools at a few thousand vectors, that's an acceptable trade.
The local-first philosophy accepts different trade-offs than cloud-first architecture. You optimize for user control and offline capability over massive scale and real-time collaboration. Both models are valid. They solve different problems.
Local-First AI Apps#
This opens up new application patterns.
Personal semantic search. Index your notes, documents, and bookmarks locally, and search without sending anything to a server.
Privacy-preserving AI features. Medical apps, therapy journals, financial tools - anything where data sensitivity matters. The AI works, but nothing leaves the device.
Offline-first knowledge bases. Documentation, research libraries, reference materials. Available everywhere, no connectivity required.
Prototypes and experiments. Build AI features without backend infrastructure. Deploy a static site, everything works client-side.
The browser becomes viable infrastructure for semantic search when you accept approximate results, pre-compute expensive operations, and cache aggressively. Getting the approximate-index layer right is the hard part, and it's the part I underestimated.
What's Left to Build#
There are natural extensions here. You could add CRDTs for sync - multiple devices, eventual consistency, conflict resolution. Better indexing for larger datasets - PCA for dimensionality reduction, quantization for memory efficiency. Integration with local LLMs running in the browser for embeddings and hardware acceleration. A fully local AI stack.
The honest status is that the storage and privacy properties hold up, and the indexing needs the rework described above before this is useful past a few thousand vectors.
We've accepted that AI features require cloud infrastructure because that's how the first wave of AI products worked. But the browser is more capable than we give it credit for. The primitives exist — IndexedDB, Float32Array, Web Workers. What's missing is the approximate-index work that cloud vector databases spent years on, and there's no reason it can't be done client-side.
Check out the source code, try the React hook, or play with the demo.
If you're building local-first applications and need semantic search, this might help. If you extend it with sync or scale it further, I'd love to hear about it.