← All projects
Local SQLite + Ollama · 0 cloud calls active

Document Search

The local-only counterpart to Library: a document search server running on SQLite and Ollama so private drafts and NDA materials never leave the laptop.

Some documents don’t belong in my cloud library: private drafts, client material, and personal notes. I built Document Search to search those files locally, with the extraction, embedding, and retrieval all running on my laptop.

It answers the same question as Library, with a different trade-off. I give up the cloud service’s richer retrieval pipeline in exchange for keeping the documents on my machine and searching offline.

How a file becomes searchable

The ingestion script extracts text from PDF, DOCX, TXT, and EPUB files, splits it into short overlapping passages, and sends them to an embedding model running in Ollama. ChromaDB stores the vectors and text locally.

expertise-reversal-2003.pdf a file in documents/ on your laptop Extract text pypdf pulls out the text layer of every page Chunk into passages 128 tokens each (25-token overlap) so ideas span seams Embed each chunk locally Ollama runs mxbai-embed-large on your CPU/GPU Each chunk becomes a vector of 1,024 numbers OLLAMA · LOCAL Persist to ChromaDB Vectors + chunk text + source filename → a SQLite file on disk filename logged to ingest manifest Later, at query time "what does expertise reversal mean?" Embed query (same model), ChromaDB returns top 5

A query goes through the same embedding model. Chroma finds similar passages and returns the top five with their filenames. The AI client talks to the server through a local process pipe, so the search service needs no listening network port.

A manifest tracks ingested files so rerunning the script skips documents already indexed.

Local or cloud?

Document Search Local-first Library Cloud-first RUNTIME Python process on your laptop AWS Lambda EMBEDDINGS Ollama · mxbai-embed-large Voyage AI · voyage-4 VECTOR STORE ChromaDB · SQLite file Pinecone · hybrid index RETRIEVAL One stage · cosine top-k Two stage · hybrid + rerank TRANSPORT MCP over stdio · no network MCP over HTTPS · OAuth/JWT COST $0 — electricity only ~$1-2/mo at this scale

For private material or offline work, I use the local version. For my published research collection, I use Library’s hybrid search, reranking, and larger surrounding sections.

Document Search uses short, fixed-size chunks and one retrieval pass. It has less document structure and metadata to work with. It also leaves backups and maintenance to me.

Rough edges

Scanned PDFs need manual OCR before ingestion. PDFs with broken text layers can also fail; checking the extracted text is the first debugging step. Automatic OCR and better handling of malformed text remain planned work.

Building both versions gave me a concrete way to think about the local-versus-cloud decision: which documents can leave the machine, and what search features are worth the extra infrastructure?

Stack

PartTool
Runtime and interfacePython, MCP over stdio
Local embeddingsOllama, mxbai-embed-large
Search storeChromaDB, persisted in SQLite
Text extractionpypdf, python-docx, EbookLib