2025_NIPS_Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifti

在这里插入图片描述

文章核心总结与翻译

一、主要内容

本文针对大型语言模型(LLMs)的机器遗忘问题,提出了一种新型注意力转移(Attention-Shifting, AS)框架,以解决现有遗忘方法中“激进遗忘损害模型效用、保守遗忘易产生幻觉”的核心矛盾。

  1. 研究背景:LLMs在实际应用中存在记忆敏感数据的隐私风险,且GDPR等法规要求支持“被遗忘权”,但现有方法难以平衡遗忘效果、模型效用与抗幻觉能力。
  2. 核心目标:
    • 保留语境的抑制:削弱对事实性tokens的注意力,同时不破坏语言结构;
    • 抗幻觉的响应塑造:避免查询遗忘内容时生成虚构答案。
  3. 技术方案:
    • 重要性感知抑制:对遗忘数据集降低记忆知识的依赖;
    • 注意力引导保留增强:对保留数据集强化语义关键tokens的注意力;
    • 双损失目标联合优化:在表征叠加下形成软边界,局部化遗忘过程并保护无关知识。
  4. 实验验证:在ToFU和TDEC基准测试中,AS框架相比现有方法,在ToFU上准确率提升15%,TDEC上提升10%,同时实现近零知识泄露和0
船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

AI 应用开发:四维架构落地实战——以 HR Agent 为例拆解感知·推理·记忆·执行【意图路由、规划编排、长期记忆、工具执行、面试题】

本文基于四维架构(感知·推理·记忆·执行),以HR Agent为例,系统拆解大模型应用从“聊天”到“生产可用”的落地路径。通过Spring AI实现意图路由、规划编排、RAG检索与工具调用,结合真实踩坑复盘,揭示架构化设计对安全、成本与可维护性的关键价值,为AI Agent工程化提供可复用的实战框架。

王大师企业官方博客 1325

语音识别(ASR)-- Kaldi 学习笔记

语音识别(ASR)-- Kaldi学习笔记语音识别简介GMM-HMM基本原理DNN-HMM(基于神经网络)KaldiKaldi架构简介Kaldi在Windows下的安装工具准备编译OpenFST配置PortAudio配置KaldiKaldi目录说明 语音识别简介 语音识别(Speech Recognition)技术,也被称为自动语音识别(Automatic Speech Recognition, ASR)、计算机语音识别(Computer Speech Recognition)或是语音转文本识别(Spee

qq_43318971的博客 4849

livox mid360 + fast-livo

更改livox_ros_driver2/launch_ROS1/rviz_MID360.launch文件内参数。更改livox_ros_driver2/launch_ROS1/msg_MID360.launch文件内参数。更改livox_ros_driver2/config/MID360_config.json文件内参数。ip改为雷达的ip,192.168.1.1** 最后两位为雷达S/N码最后两位数字。雷达的广播码在雷达后面,可以在雷达机身的QR 码下面查看,共15位数。

z1872385的博客 4899

(MIUI)小米手机录音文件深度寻踪:从系统缓存到数据目录的实战解析

本文深入解析了MIUI系统中录音文件丢失的常见原因及解决方案,包括系统升级兼容性问题、应用缓存机制异常和存储权限变动。通过实战操作指南,详细介绍了如何从系统目录、数据库层面恢复录音文件,并提供了预防措施和自动化备份方案,帮助小米手机用户有效找回重要录音数据。

weixin_42699679的博客 522

最详细的 SAP ABAP Web Service 创建和消费步骤讲解

本文原版发表于 SAP 官方社区,当时用的英文版写作,标题为:Step by step to create, consume and trace web service in ABAP system https://blogs.sap.com/2014/05/20/step-by-step-to-create-consume-and-trace-web-service-in-abap-system/ 总共阅读量超过 18 万。 How to create web service provider in A

2007 年 ~ 2025 年,深耕 SAP 技术 18 年 3154

机器学习实战5-天气预测系列:利用数据集可视化分析数据,并预测某个城市的天气情况

大家好,我是微学AI,最近天气真的是多变啊,忽冷忽热,今天再次给大家带来天气的话题,机器学习实战5-天气预测系列,我们将探讨一个城市的气象数据集,并利用机器学习来预测该城市的天气状况。该数据集包含年平均温度和湿度等信息。

微学AI的博客 1万+

广东省道路工程检测练习试题(单选题)真题

1.一般项目的合格点率应达到( ),且不合格点的最大偏差值不得大于规定允许偏差值的1.5倍。A.不低于75%B.不低于80%C.不低于85%D.不低于90%答案:B2.进行土路基平整度检测时,当路宽小于9m,检测点数应为( )A.1B.2C.3D.4答案:A3.根据《城镇道路工程施工与质量验收规范》,在分部工程中,根据工种、材料、施工工艺等划分的是( )A.单位工程B.分项工程C.分部工程D.工序工程答案:B4.根据《城镇道路工程施工与质量验收规范》,土方路基压实度的检测频率为每1000m每压实层抽检( )

ZShy001238的博客 686

免杀思路总结

免杀思路总结 1.免杀技术简介 1.免杀的重要性 2.免杀的难度 3.杀毒软件的排行 国外 国内 1 、Bitdefender 1、360 2 、Norton 360 2、智量安全(没测过,据说很厉害) 3 、Kaspersky 3、火绒安全 4 、Webroot 4、金山独霸 5 、Trend Micro 5

LiBai'S BLOG 1万+

VC中sqlite的使用

在VC中对sqlite的使用,实现数据库大批数据的快速插入,读取数据库中的数据在ListCtrl中显示

假人有限元模型

假人模型,车辆碰撞假人有限元模型,适合做乘员安全的研究者利用

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

visio安装包

Microsoft Office Visio 是一款专业的制图软件。利用经过更新的形状、协作工具和数据链接图表,创建专业级图表以简化复杂的信息。

上一篇: 2025_NIPS_Reverse Diffusion Sequential Monte Carlo Samplers
下一篇: 2025_NIPS_HAWAII: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models
UnknownBody
博客等级 码龄16年 5601粉丝 4754原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值