Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator

本文是LLM系列文章,针对《Automatic Interactive Evaluation for Large Language Models
with State Aware Patient Simulator》的翻译。

具有状态感知患者模拟器的大型语言模型的自动交互评估

摘要

大型语言模型(LLM)在人类互动方面表现出了非凡的熟练程度,但其在医学领域的应用仍有待充分探索。以往的工作主要集中在医学知识与检查的表现上,与现实情况相去甚远,无法评估LLM在临床任务中的能力。为了增强大型语言模型(LLM)在医疗保健中的应用,本文引入了自动交互评估(AIE)框架和状态感知患者模拟器(SAPS),针对传统LLM评估与临床实践细微需求之间的差距。与以前依赖静态医学知识评估的方法不同,AIE和SAPS为通过多轮医患模拟评估LLM提供了一个动态、现实的平台。这种方法提供了更接近真实临床场景的方法,并允许对LLM行为进行详细分析,以应对复杂的患者互动。我们广泛的实验验证证明了AIE框架的有效性,其结果与人类评估非常一致,突显了其在彻底改变医疗LLM测试以改善医疗服务方面的潜力。

1 引言

2 结果

3 讨论

4 方法

船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

AI 应用开发:四维架构落地实战——以 HR Agent 为例拆解感知·推理·记忆·执行【意图路由、规划编排、长期记忆、工具执行、面试题】

本文基于四维架构(感知·推理·记忆·执行),以HR Agent为例,系统拆解大模型应用从“聊天”到“生产可用”的落地路径。通过Spring AI实现意图路由、规划编排、RAG检索与工具调用,结合真实踩坑复盘,揭示架构化设计对安全、成本与可维护性的关键价值,为AI Agent工程化提供可复用的实战框架。

王大师企业官方博客 1119

血管分割中的直径平衡难题:从clDice到cbDice的演进与实践

本文探讨了血管分割中的直径平衡难题,详细分析了从clDice到cbDice的技术演进与实践。通过引入半径感知权重和边界对齐惩罚,cbDice在保持拓扑完整性的同时提升了几何敏感度,显著改善了细小血管的分割效果。文章还分享了在nnU-Net中实现cbDice的工程细节,包括GPU加速骨架提取和多任务损失组合策略,为医学影像分析提供了重要参考。

weixin_29170327的博客 244

【实用运维脚本】Windows 批量 TCP 端口检测(tcping 一键巡检)

摘要:本文介绍一款高效的Windows批量TCP端口检测脚本,适配tcpingv0.39工具,实现多服务器一键巡检。脚本具有进度可视化、双日志输出、自动容错等优势,支持注释行过滤和UTF-8编码。 使用仅需三步: 1)下载tcping工具 2)编辑目标列表 3)运行脚本即可自动检测并生成带编号的检测报告。 该方案解决了手动检测效率低、易遗漏的问题,特别适合日常运维中的端口连通性检查,实测稳定可靠,开箱即用。

蕲士|D365 & 服务器运维,心有东坡,行稳致远。 803

clock-gating整理

文章目录clock gating 理解1、latch双稳态器件常见锁存器结构SR锁存器D锁存器2、clock gating(clock low-active) latch + AND gating 电路(clock high-active)latch + OR gating 电路 clock gating 理解 1、latch 参考:https://www.cnblogs.com/IClearner/p/6443539.html 双稳态器件 稳定状态为0或1两种 举例:交叉耦合反相器 假设初始Q为1,则可

d_b_的博客 1万+

Fabric学习笔记(一)- Centos 7下的Fabric 2.3.2环境搭建

一、 环境准备 (一)、在VMware Workstation 15 安装一个Centos 7的虚拟环境,切换到图形化界面: 参考文章链接:Centos 7切换图形化界面 其中可能在yum groupinstall "X Window System"时会报错:Cannot find a valid baseurl for repo: base/7/x86_6,参考这篇文章即可:安装图形化界面报错解决 (二)、打开终端,安装所需的库: 切换root用户,输入命令:su root yum更新,输入命令:y

茕夜 2611

华为MetaERP PTP采购到应付全流程解析

摘要:本文系统解析华为MetaERP中的PTP(采购到应付)全流程管理方案,涵盖从采购申请到付款执行的完整业务流程。文章详细分解了PR、PO、GR、IV、AP等关键环节的财务核算规则,并提供了预算校验、库存更新等Python代码示例。同时阐述了IT落地架构设计,包括会计引擎配置、系统集成方案及银企直连支付接口实现。该方案通过三单匹配自动化和智能会计引擎,将处理效率提升10倍(200笔/人天→2000笔/人天),实现98%的自动化率,为集团企业提供了高效的数字化财务管理样板。文中所有技术方案均需根据实际场景调

金牌架构师 1743

【AI Agent】Harness Engineering实践

Harness(运行宿主)是围绕 LLM构建的一套确定性工程系统,其核心职责是:承认模型的固有局限,用可控的工程手段逐一弥补,使"强大但不可靠的推理引擎"变成"可信赖的生产级Agent"模型 是发动机——提供原始动力Agent 是驾驶员——做出决策Harness 是道路、地图、护栏、仪表盘、维修站和质检线——确保行驶安全有序没有Harness,模型越强,可能只是更快地制造混乱。

sysu_lluozh 195

(LDA新闻文本主题分析)基于 gensim训练LDA模型,进行新闻文本主题分析 完整代码+数据 可直接运行

(LDA新闻文本主题分析)基于 gensim训练LDA模型,进行新闻文本主题分析 完整代码+数据 可直接运行

机器学习深度学习业余选手 1498

Dify 第 13 篇 Dify 工作流 详解

工作流通过将复杂任务分解为节点降低复杂度,提升LLM应用的性能和稳定性。Dify工作流分为两类:Chatflow面向对话场景(如客服),支持多轮对话;Workflow面向批处理场景(如翻译),提供逻辑节点实现自动化。核心概念包括节点、变量(系统/会话/环境变量)及应用类型差异:Chatflow支持中间输出和对话历史,Workflow仅支持结束节点输出。系统变量以"sys"开头,用于全局数据传递,不同应用类型变量不同,如Chatflow的sys.query存储用户输入,Workflow的s

甘蓝的专栏 3250

Statistics and Machine Learning Toolbox

Statistics and Machine Learning Toolbox(带书签),MATLAB统计学习工具

Outlook Express 6 (OE6) for Win7/Win8/Win10

英文版 Outlook Express6.0, 兼容Win7/Win10/WinXP,解压就可用。

VC中sqlite的使用

在VC中对sqlite的使用,实现数据库大批数据的快速插入,读取数据库中的数据在ListCtrl中显示

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

上一篇: Rich Semantic Knowledge Enhanced Large Language Models for Few-shot Chinese Spell Checking
下一篇: Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey
UnknownBody
博客等级 码龄16年 5601粉丝 4744原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值