Build a local document index with Ollama embeddings
Use one consistent embedding model for indexing and queries, and keep bulk document work from crowding out interactive inference.
In brief
- Ollama’s embedding endpoint converts text into vectors and accepts batches of strings.
- Start with a small set of questions whose relevant passages you already know.
What to check
Ollama’s embedding endpoint converts text into vectors and accepts batches of strings. Store each vector with a document reference and the model identity used to create it. Use the same model and preprocessing for indexing and querying. Changing models can change the vector space or dimensions, so plan to rebuild the index when switching.
Put it to work
Start with a small set of questions whose relevant passages you already know. Inspect the retrieved passages before asking a chat model to summarize them. Tune chunk size and batch size independently. Schedule large indexing jobs away from interactive peaks if both workloads share a GPU, and treat stored vectors and their source mappings as part of the private document collection.
Make indexing failures visible
The embedding endpoint offers a truncation setting. During initial indexing, consider disabling truncation so oversized chunks fail visibly and can be fixed. Preserve document identifiers and source offsets with each vector. Run a small batch first and check that every returned vector maps to its intended input. Record the embedding model digest before building the production index.
Measure retrieval independently from answers
Use questions with known matching passages and inspect the top results. Track missed relevant passages and misleading near matches. Change chunk size, overlap or retrieval depth one at a time. When switching embedding models, build a separate index and compare the results before replacing the old one. The generation model should cite retrieved passages, so a reader can verify that the answer rests on the right evidence.
Sources and verification
Based on primary documentation reviewed on 3 November 2026. Practical test procedures are editorial guidance. Performance depends on the model, hardware and configuration.
- Embeddings Ollama · Primary document
- Generate embeddings Ollama · Primary document
- List models Ollama · Primary document
Prepared with AI assistance from the linked sources. This is not a claim of independent field reporting or human review. How we publish · Report a correction
Keep up with homelab
Explore the rest of the local Ollama guide series.
RSS requires an email newsletter opt-in.
