Learning path · Evaluation & Quality · 54
RAGAS
Reference-free RAG evaluation suite measuring faithfulness, answer relevance, context precision, and context recall with LLM-assisted scoring.
Why it matters
- Turns RAG quality into CI-friendly metrics without expensive gold labels for every claim.
- Separates retrieval failures from generation hallucinations.
- Pairs well with Phoenix-style tracing for production samples.
Key ideas
- Faithfulness
- Answer relevance
- Context precision/recall
Top resources
- 01DocsExploding Gradients
RAGAS
Why this resource. Primary documentation for RAGAS metrics and datasets.
Covers in this concept
- faithfulness
- answer relevancy
- context recall
- 02PaperLiu et al.
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Why this resource. Related judge-based scoring when you leave the RAGAS defaults.
Covers in this concept
- LLM-as-judge
RAGAS scores RAG traces end-to-end: did retrieved context contain the answer, did the reply stick to that context, and did it address the question. Use it on golden sets and sampled production logs—not as a single vanity number, but sliced by corpus and language. Treat dips after index rebuilds or prompt edits as release blockers. Combine RAGAS with human spot-checks on high-risk domains; automated scores miss subtle policy misreads. Wire failures into prompt and retrieval tickets with the offending chunks attached so engineers fix causes, not symptoms.
Updated 2026-08-09 · Full learning path