Self-hostable · frontier-grade IR · zero hosted-API deps

Search,
research,
on your own box.

Crawl the web. Index it (BM25 + HNSW dense). Search with hybrid retrieval, MMR diversification, time-decay. Answer and research with cited sources via any OpenAI-compatible LLM. Everything in one Go binary — cosift.

Indexed docs
Vector passages
Retrievers live
Docs / min
% of Common Crawl
The pipeline

Five orthogonal knobs. One label.

retriever

bm25 (lexical) · dense (HNSW cosine over per-passage embeddings) · hybrid (RRF-fuse the two)

expand

hyde appends an LLM-hypothesized passage to the query · paraphrase fans out N rewrites and RRF-fuses

rerank

Cross-encoder reorder of the candidate pool. HTTP (Cohere / Voyage / Jina / TEI) or LLM listwise.

mmr

Maximal Marginal Relevance — λ ∈ [0,1] trades relevance against diversity from already-selected sources.

decay

Exponential time-decay half-life in days — exp(-ln 2 · age / H). Hits without a publish date are unchanged.

The retrieval label is the composed pipeline, e.g. bm25+dense:rrf+rerank:cohere+mmr:0.70+decay:30d. Each knob composes on every retrieval/synth endpoint (/search, /answer, /research, /find_similar).

This server, right now

Live state.

Backend
Uptime
BM25 k1 / b
Avg doc length
HNSW
Embedder · Chat
PQ compression
Crawl rate
Retrievers available
Crawl frontier

What's in the queue right now.

Queued
In flight
Done
Errored
loading…
Right now in the index

Sources by doc count.

loading…
Try it now

Search the live corpus.

Direct API access: /docs · /openapi.json · curl '/search?q=…&retriever=hybrid&mmr=0.7&decay=30'

Chat with the corpus

Ask anything.

Multi-turn chat with streaming answers and cited sources. RAG over the live index.

Open chat →