VEL ORENDISVel Orendis
The Observatory / Search and RAGQuest: The Missing Entry
0 StandingLV 10 XP

The RAG Pipeline

Watch one question travel the pipeline.

Tidewater's assistant now searches its FAQ, and still gets answers wrong. Find the part that breaks.

Tidewater's assistant no longer answers from memory alone. Each question now runs through three parts, in order.1

Search compares the question with Tidewater's FAQ and returns the entries that match best. The prompt puts those entries in front of the model, with the question and a rule for using them. The model writes the answer from what the prompt gives it.

This is retrieval-augmented generation, or RAG. The facts live in the index, so a change to the FAQ reaches the next answer without retraining the model.2 It also means the answer can be no better than what search finds.

Watch one question go through.

References

  1. 1. Generative AI for Beginners, Lesson 15, Retrieval Augmented Generation (RAG) and Vector Databases: retrieval supplies the context, generation writes from it. Microsoft, MIT, commit 3a9d9ef.
  2. 2. OpenAI Cookbook, Question answering using embeddings-based search: search, then ask, and why search is better than fine-tuning for facts. OpenAI, MIT, commit 5bd6cb4.

Numbered ideas come from these sources, rewritten in our own words. Everything else, including the simulation and its numbers, was written for Vel Orendis.

  1. A student asks

    When does the spring cohort start?

  2. Search, current FAQ, top 3

    Not yet run.

  3. Prompt, with the rule

    Not yet built.

  4. Small model

    Not yet written.

A student asks when the spring cohort starts. Alone, the model gave last year's date. This time the question goes to search first.

When you have watched the question go through, open the pipeline.