DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models

本文是LLM系列文章,针对《DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models》的翻译。

DiaHalu:大型语言模型的对话级幻觉评价基准

摘要

由于近年来大型语言模型(LLM)取得了显著的成功,幻觉问题仍然是一个挑战,因此提出了许多检测幻觉的基准。尽管如此,其中一些基准并不是LLM自然生成的,而是有意诱导的。此外,许多人只关注真实性幻觉,而忽视了忠实性幻觉。此外,尽管对话模式在LLM时代得到了更广泛的应用,但目前的基准只集中在句子层面和段落层面的幻觉。在这项研究中,我们提出了DiaHalu,这是我们所知的第一个对话级幻觉评估基准。最初,我们将收集的主题集成到系统提示中,并促进两个ChatGPT3.5之间的对话。随后,我们手动修改不符合人类语言约定的内容,然后重新生成LLM,模拟真实的人机交互场景。最后,专业学者对数据集中的所有样本进行注释。DiaHalu涵盖了四个常见的多回合对话领域和五种幻觉亚型,从真实性和忠实性幻觉扩展而来。在数据集上通过一些著名的LLM和检测方法进行的实验表明,DiaHalu是一个具有挑战性的基准,对进一步研究具有重要价值。

1 引言

2 相关工作

3 DiaHalu概述

4 DiaHalu的构造

5 实验

6 结论

船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

AI 应用开发:四维架构落地实战——以 HR Agent 为例拆解感知·推理·记忆·执行【意图路由、规划编排、长期记忆、工具执行、面试题】

本文基于四维架构(感知·推理·记忆·执行),以HR Agent为例,系统拆解大模型应用从“聊天”到“生产可用”的落地路径。通过Spring AI实现意图路由、规划编排、RAG检索与工具调用,结合真实踩坑复盘,揭示架构化设计对安全、成本与可维护性的关键价值,为AI Agent工程化提供可复用的实战框架。

王大师企业官方博客 1325

COMSOL解的继承

COMSOL解的继承1. 应用场景2. 实现方法2.1. 添加研究步骤2.2. 引用研究 根据参考文献整理而来,原文见: http://blog.sina.com.cn/s/blog_4a0a8b5d010126w5.html https://www.zhihu.com/question/371867999/answer/1309194094 个人学习笔记,如有疏漏,欢迎补充 1. 应用场景 在COMSOL模拟中,一次求解之后往往遇到以下几种情况需要用到解的继承: 1)瞬态求解的继续运算。例如初始设定求解1

jessica0307的博客 2万+

10X Genomics单细胞数据分析实战:从FASTQ到UMAP可视化的完整流程

本文详细介绍了10X Genomics单细胞数据分析的完整流程,从FASTQ原始数据处理到UMAP可视化。通过Seurat工具包实现数据质量控制、标准化、降维聚类和细胞注释,特别适合生物信息学新手学习。文章包含可复现的代码示例和常见问题解决方案,帮助研究者快速掌握单细胞测序数据分析的核心技术。

weixin_30952535的博客 539

RTKLIB模糊度固定实战:三种模式对比与部分模糊度固定代码解析

本文深入解析RTKLIB中模糊度固定的三种核心模式(continuous、instantaneous、fix and hold),对比其性能与适用场景,并详细拆解了部分模糊度固定技术的源码实现。通过实战案例与代码级解析,为GNSS高精度定位开发者提供了在不同数据特性下提升固定成功率的优化思路与配置建议。

efc1234567的博客 473

《从0到1:CTFer成长之路》书籍配套题目-[第二章 web进阶]文件上传

[第二章 web进阶]文件上传开始行动绕过第1个限制绕过第2个限制 介绍:记录一下笔记,方便以后迅速回忆使用。 《从0到1:CTFer成长之路》书籍配套题目,来源网站:《从0到1:CTFer成长之路》 先看一下题目源码: <?php header("Content-Type:text/html; charset=utf-8"); // 每5分钟会清除一次目录下上传的文件 require_once('pclzip.lib.php'); if(!$_FILES){ echo '

weixin_46703850的博客 3412

STM32F103C8T6+HX711:手把手教你打造高精度智能体重秤(附完整代码)

本文详细介绍了如何使用STM32F103C8T6微控制器和HX711称重模块构建高精度智能体重秤,涵盖硬件选型、电路设计、软件开发到无线数据传输的全过程。通过完整的代码示例和校准方法,帮助开发者快速实现毕业设计或商业项目,打造具备物联网功能的智能健康设备。

h6i7j8的博客 1032

开源ISP实战(1)——基于Zynq7020的Vivado工程移植与调试

本文详细介绍了将开源ISP项目移植到正点原子Zynq7020开发板的实战过程。重点阐述了在Vivado中进行工程创建、IP核导入、Block Design框架搭建、时钟配置与管脚约束修改等关键步骤,并提供了系统联调与常见问题的排查思路,为FPGA图像处理开发者提供了完整的移植与调试指南。

weixin_29191081的博客 204

【WRF模拟】土地利用类型的确定:最大覆盖法/瓦片法

土地利用类型的确定:最大覆盖法/瓦片法

WW、forever的博客 1530

orchestrator常见问题汇总

默认的取集群名称的规则为参数决定 ,默认为取主实例 主机名 ,然后按分隔符"."之前的第一段字符为集群别名,这样在集群主库宕机后,新提升的从库的服务器名称为了新的集群名,宕机的主库成为单独的一个集群。1 默认配置文件中设置了匹配规则 ,只有被匹配的主机名才能进行故障恢复,所以该参数需要设置为 "*"3 另外配置文件中还有许多发现,恢复的忽略规则(Ignore) 这些参数也是需要注意的。由于LVS的后端的RS为IP地址,如果直接显示主机名,LVS的接口方法将无法工作。只传入脚本中用到的参数,不要多传入。

DBA之路的博客 829

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

ET2012新版免狗免安装绿色软件

ET2012新版免狗免安祖昂绿色软件。,新界面,新视觉,

汉书HE8,好用的汇编代码编写工具

汉书HE8,好用的汇编代码编写工具早期编写汇编代码所用功能强大

假人有限元模型

假人模型,车辆碰撞假人有限元模型,适合做乘员安全的研究者利用

上一篇: Reading Subtext: Evaluating Large Language Models on Short Story Summarization with Writers
下一篇: EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
UnknownBody
博客等级 码龄16年 5601粉丝 4754原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值