Stage 8 of 12 · about 3.2 h

RAG in production

Index at scale, keep it fresh, sharpen the query, and measure every stage.

You take Pocket's retrieval from a working prototype to something you can run for years. You build an IVF index by hand and measure the recall it trades for speed, then decide when brute force is still right. You attach metadata to chunks, see why a post-filter returns fewer than k results, and keep the index fresh with content hashes that re-embed only what changed and rebuild when the model changes. You rewrite follow-up questions, fuse several phrasings, and rerank a short list, reading every gain per question. Last, you assemble context under a budget and score it with context precision and recall, so you can tell a retrieval failure from a generation failure.

Before you start

You can chunk and embed notes, search them by cosine, measure recall@k and MRR on a golden set, and grade answers with deterministic checks and a judge, as built in stages 5 and 7.

When you finish

pocket/stage8/ holds ivf_index.py (an IVF index with a recall-versus-nprobe table), incremental_index.py (a hash-keyed index that re-embeds only changed chunks), query_and_rerank.py (rewriting, multi-query fusion and reranking scored on the golden questions) and assemble_context.py (a budgeted context assembler with context precision and recall).

1. Lesson 1 · 45 min

   Vector indexes: exact, IVF and HNSW

   Build an IVF index by hand with k-means cells and nprobe, measure its recall@10 against brute force as you trade vectors scanned for accuracy, and decide when an approximate index is worth running at all.

2. Lesson 2 · 45 min

   Metadata, filters and a fresh index

   Attach metadata to every chunk, explain why a post-filter can return fewer than k results, and keep the index fresh with a sha256 manifest that re-embeds only new or changed chunks, drops deleted ones, and rebuilds when the embedding model changes.

3. Lesson 3 · 50 min

   Query rewriting, fusion and reranking

   Rewrite follow-up questions into standalone queries, fuse several phrasings with reciprocal rank fusion, explain HyDE, and build a retrieve-then-rerank pipeline whose gains you measure per question with recall@3 and MRR.

4. Lesson 4 · 50 min

   Context assembly and end-to-end RAG evals

   Assemble retrieved chunks into a prompt under a word budget with neighbor merging and edge ordering, score the context with context precision and context recall, and tell a retrieval failure from a generation failure using faithfulness and answer relevance.