BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models

在这里插入图片描述

文章主要内容总结

核心贡献
  1. 首次提出Token级后门攻击:针对多模态大语言模型(MLLMs)设计了两种新型攻击行为:

    • Token替换:将输出中的特定源token(如"red")替换为目标token(如"green")
    • Token插入:在输出末尾插入恶意token序列(如钓鱼链接)
  2. 优化框架:

    • 提出结合有效性损失(最大化攻击成功率)和实用性损失(保持模型性能)的优化目标
    • 通过影子数据集训练实现触发模式与攻击行为的绑定
  3. 防御验证:

    • 测试了微调(Fine-tuning)和输入净化(Zero-shot Image Purification)等防御方法的局限性
    • 发现Token替换攻击对微调具有较强抗性
关键发现
  • 在LLaVA和MiniGPT-4等模型上实现了95%以上的
【论文笔记】POISONPROMPT_ BACKDOOR ATTACK ON PROMPT-BASED LARGE LANGUAGE MODELS Prompts显着提高了预训练大型语言模型(LLM)在各种下游任务上的性能,使其对于各种 LLM 应用场景越来越不可或缺。进行了广泛的实验,评估了的有效性、保真度和鲁棒性。:我们的研究结果强调了对,并强调了在这一领域进一步研究的必要性。 阅读详情

相关推荐

VS2015+Qt+OpenCV+open62541开发过程(05_VS2015中使用OPCUA open62541通过Kepware读写PLC数据)

此外还需要所需lib包:WS2_32.Lib(opcua通讯用到的库,opcua使用socket通讯,需使用此库),添加#pragma comment(lib,"ws2_32.lib") 语句,有的文章说需要添加ws2_32.lib文件或ws2_32.dll到项目资源中,貌似影响不大。2.在VS中选择x64平台会报错,选择x86平台可以用,应该是ws2_32.lib相关的库不支持64位,目前没有解决,继续后面的实验,看看用Qt后是否可以解决。6.VS中建立的变量,与读取的OPC点数据类型注意对应。

zzh17的博客 1862

MATLAB信号处理实战:VMD分解参数调优全指南(附避坑技巧)

本文深入探讨了MATLAB中变分模态分解(VMD)的核心参数调优实战技巧。针对模态数K和惩罚系数α的选择难题,提供了从经验估计、可视化观察到量化指标辅助的渐进式调优框架,并分享了避免过分解与欠分解的实用避坑指南,旨在帮助工程师高效处理非平稳信号,提取清晰的物理特征。

weixin_29079671的博客 145

一文搞懂雷达脉冲压缩和匹配滤波器

为了解决传统单频脉冲雷达面临的作用距离和空间分辨力之间的矛盾,脉冲压缩理论被提出。在接收端设计一个和发射信号能够“共轭匹配”的网络来实现脉冲压缩。接收到的回波信号通过匹配滤波器时,滤波器的增益在信号越强的频率点越大;在信号越弱的频率点,增益越小,通过这样的方式使信号在时域更加集中。脉冲压缩技术大致可分为线性调频(LFM)和相位编码。其中LFM信号易于通过硬件或各种技术产生,但是进行匹配滤波后主旁瓣比高达13.2dB,在检测多个目标时,强目标的回波旁瓣可能会把弱小信号覆盖,导致弱小信号的检测受到影响。

FPGA入门到跑路 9938

Xilinx-复旦微-FPGA-眼图BlockDesign系统制作-针对PCIE系统

XIlinx-复旦微电子-FPGA-眼图BlockDesign系统制作-针对PCIE系统,手把手教你如何建立眼图测试

weixin_43444334的博客 2598

智能枕头

睡觉和吃饭是一样重要的,一个人想要健康的体魄,不仅要饮食得当,更需要有一个好的睡眠质量。随着人们生活品质的提高,人们对于睡眠的要求越来越高,那么就非常需要一款舒适的枕头。为了适应市场的需求,我爱物联网推出了智能枕头。因为智能枕头可以给头部及颈椎一个良好的承托力,舒适、柔软、放松,让人更好的进入睡眠。 目录 一、智能枕头方案介绍 二、智能枕头的优点 三、智能枕头功能介绍 四、智能枕头APP开发 五、...

6108我爱物联网方案开发优快云博客 3660

kettle实验十一:没有数据流入时停止操作

【实验目的】 1.利用Kettle的“阻塞数据直到步骤都完成”,“检测空流”,“中止”等组件,实现Kettle转换流程的控制。 2.熟练掌握“阻塞数据直到步骤都完成”,“检测空流”,“中止”等组件的使用,实现Kettle转换流程的控制。 【实验原理】 “检测空流”组件可以检测所在转换的输入数据,如果整个转换没有输入数据,通过“中止”组件终止整个转换的流程。在存在数据输入的情况,为了能让数据输出,需要和“阻塞数据直到步骤都完成”配合使用。当所有数据都已经从上个步骤“字段选择”读取完毕后,通过“阻塞数据直到步骤

weixin_44727274的博客 3848

【nvidia-B200】实际生产报错NVRM: nvAssertFailedNoLog: Assertion failed: pIOVAS != NULL @ io_vaspace.c:601

升级 NVIDIA 驱动到 R545/R550 数据中心版本(解决 IOVA bug,这是性能差的根本原因);重启服务器(清理驱动残留资源,消除日志中的 NVRM 错误);验证 NVLink/PCIe 链路(确保 8 卡互联使用高速链路,而非单卡 PCIe 共享带宽)。完成这 3 步后,ncccl-test的all_reduce带宽应能提升至100GB/s 以上(取决于服务器的 NVLink/NVSwitch 配置),恢复正常性能。

jundao1997的专栏 652

【2026年阿里巴巴集团暑期实习- 4月25日-算法岗-第二题- 矩阵计数】(题目+思路+Java&C++&Python解析+在线测试)

摘要: 给定两个非负整数序列a和b,构造n×m矩阵C,其中C[i][j]=a[i]×b[j]。统计矩阵中元素≥k的数量。解法是先对数组b排序,然后对于每个a[i],若k=0则直接累加m;若a[i]=0且k>0则跳过;否则计算需要的最小b[j]值,在排序后的b中使用二分查找确定满足条件的元素数量。时间复杂度为O((n+m)log m),空间复杂度O(1)。代码实现了排序和二分查找的高效组合。

m0_73659489的博客 65

SIMATIC S7-1200, S7-1500 PID 控制[手册]

SIMATIC S7-1200, S7-1500 PID 控制[手册]

STM32F407中文手册

STM32f407中文手册 本参考手册面向应用开发人员,提供有关使用 STM32F405xx/07xx、STM32F415xx/17xx、 STM32F42xxx 和 STM32F43xxx 微控制器存储器与外设的完整信息。STM32F405xx/07xx、STM32F415xx/17xx、STM32F42xxx 和 STM32F43xxx 构成一个微 控制器系列,各产品具有不同的存储器大小、封装和外设。有关订购信息以及器件的机械与电气特性,请参见数据手册。有关 ARM Cortex™-M4F 内核的信息,请参见《Cortex™-M4F 技术参考手册》。

Statistics and Machine Learning Toolbox

Statistics and Machine Learning Toolbox(带书签),MATLAB统计学习工具

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

上一篇: Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning
下一篇: Investigating Retrieval-Augmented Generation in Quranic Studies: A Study of 13 Open-Source LLM
UnknownBody
博客等级 码龄16年 5601粉丝 4741原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值