VEL ORENDISVel Orendis

Evaluation and monitoring

LLM as a judge, logging, feedback, dashboards, ingestion pipelines.

7 lessonsquests, about 1.2 h. None is open yet: they open here as they are written. Take them in any order.

Introductory2 lessonsquests

  1. Offline and online evaluationBeing written8 min
  2. Logging every LLM callBeing written10 min

Intermediate5 lessonsquests

  1. LLM as a judgeBeing written12 min
  2. Comparing prompts and modelsBeing written10 min
  3. Feedback and online judgesBeing written10 min
  4. Dashboards and traces for LLM appsBeing written12 min
  5. Ingestion pipelinesBeing written12 min