官网: Ragas
Ragas(Retrieval-Augmented Generation, RAG)是一个基于简单手写提示的评估框架,通过这些提示全自动地衡量答案的准确性、 相关性和上下文相关性。这种评估方法不需要访问人工注释的数据集或参考答案,使得评估过程更为简便和高效。
跟着官方的入门案例来了解该框架
安装
pip install ragas
加载数据
从Hugging face中加载数据集
from datasets import load_dataset
dataset = load_dataset(
"explodinggradients/amnesty_qa",
"english_v3",
trust_remote_code=True
)
# 若无法使用代理也可以直接加载
dataset = {'question': 'Which private companies in the Americas are the largest GHG emitters according to the Carbon Majors database?',
'ground_truths': ['The largest private companies in the Americas that are the largest GHG emitters according to the Carbon Majors databa