2025_NIPS_Spotlight Attention: Towards Efficient LLM Generation via Non-linear Hashing-based KV Cach

文章总结与翻译

一、主要内容在这里插入图片描述

该研究聚焦于大型语言模型(LLMs)推理阶段的KV缓存优化问题,针对解码阶段因KV缓存频繁内存交互导致的效率瓶颈,提出了一种名为Spotlight Attention的高效推理方法。

  1. 核心背景:LLMs推理分为预填充和解码两个阶段,解码阶段因KV缓存的内存交换成为主要瓶颈(如LLaMA2-7B在A100 GPU上解码阶段GPU利用率不足10%)。现有KV缓存优化方法存在缺陷:静态剪枝不适应长序列任务,动态永久淘汰可能过早移除关键令牌,而基于线性哈希的令牌级检索(如MagicPIG)需冗长哈希码,存储和计算成本高。

  2. 方法设计:

    • 采用非线性MLP哈希函数替代传统线性哈希,解决LLMs中查询和键在嵌入空间呈正交锥分布导致的线性哈希编码效率低问题,哈希码长度缩短至少5倍。
    • 提出基于Bradley-Terry排序损失的轻量训练框架,仅需少量校准数据,在16GB内存GPU上8小时即可完成训练,且不冻结LLM主干网络。
    • 实现专用CUDA内核,通过位运算优化哈希检索,单A100 GPU上512K令牌的哈希检索耗时低于100
船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

AI 应用开发:四维架构落地实战——以 HR Agent 为例拆解感知·推理·记忆·执行【意图路由、规划编排、长期记忆、工具执行、面试题】

本文基于四维架构(感知·推理·记忆·执行),以HR Agent为例,系统拆解大模型应用从“聊天”到“生产可用”的落地路径。通过Spring AI实现意图路由、规划编排、RAG检索与工具调用,结合真实踩坑复盘,揭示架构化设计对安全、成本与可维护性的关键价值,为AI Agent工程化提供可复用的实战框架。

王大师企业官方博客 714

跨阻放大器(TIA)的输入输出阻抗的定义、分析和影响因素

跨阻放大器(Transimpedance Amplifier,简称TIA)是一种常用于将电流信号转换为电压信号的放大器。在TIA电路中,输入阻抗是指输入端对电流信号的响应能力,输出阻抗是指输出端对负载电阻的驱动能力。本文将分别介绍TIA的输入和输出阻抗的定义、分析以及影响因素。

weixin_44378594的博客 1万+

DAC8760高精度数模转换器原理与工业级嵌入式应用

数模转换器(DAC)是工业测控系统中实现数字指令到模拟执行的关键接口,其核心性能由分辨率、线性度、温漂及系统级TUE(总未调整误差)共同决定。DAC8760作为一款16位四象限精密DAC,集成了可编程增益放大器(PGA)、内部基准源与SPI高速接口,支持电压/电流双模输出,显著提升闭环控制精度与硬件适配灵活性。其技术价值在于将高精度模拟前端与数字控制深度耦合,满足PLC、ATE及激光偏置等严苛场景对长期稳定性与多通道同步性的要求。本文围绕DAC8760的寄存器配置、HAL/LL驱动实践、PCB布局规范、现场

weixin_42315569的博客 583

PostgreSQL数据库全攻略:从入门到精通

本文是一份PostgreSQL数据库综合指南,涵盖安装配置、基础操作和高级管理。主要内容包括:1) PostgreSQL简介与各平台安装方法;2) 数据库基本操作,如创建数据库/表、CRUD操作、用户权限管理;3) 高级功能包括索引优化、事务控制、视图、存储过程、触发器和扩展模块;4) 数据库管理维护,涉及配置调优、备份恢复、性能监控和日常维护;5) 安全设置与高可用方案。指南提供了从入门到进阶的实用知识,强调结合官方文档进行深入学习,帮助用户高效使用这一功能强大的开源关系型数据库系统。

索大神的博客 1970

具身智能数据采集的工程化指南:从时间同步到质量校验

具身智能的模型训练离不开高质量的多模态轨迹数据,而数据采集并非简单的传感器记录。真实场景中,相机图像、机械臂关节状态与力觉信息必须在时间轴上精确对齐,标定的漂移和格式混乱会让看似正常的数据无法用于训练。时间同步与标定一致性,是决定数据质量的核心技术环节。在机器人抓取、操作等工程应用中,一套包含多传感器同步、轻量记录脚本、自动化质量校验与版本管理的数据采集流程,能大幅降低数据生产的失败率。本文从工程视角拆解具身数据采集中的典型问题,并给出可落地的流程设计与校验方法,帮助团队构建可持续迭代的高质量数据集。

weixin_30603633的博客 511

拉普拉斯矩阵

1、预备知识 拉普拉斯矩阵定义为:L=D-W。其中,D为度矩阵,W为权重矩阵。 (1)D是对角矩阵,对角线上的元素时节点的度,即所有与该节点相关联的边的权重之和;对角线以外的元素为0。有向图的对角矩阵分为 出度矩阵 和 入度矩阵。 图1.1 无向图中的度矩阵 (2)W在无向图中是对称矩阵,在有向图中是非对称矩阵。 ...

zzhe 1万+

YOLO26改进策略【Neck】| ICCV2025 反向卷积 Converse2D 数学逆运算提重构 + 特征域建模降伪影,提升多尺度目标检测精度

本文记录的是利用Converse2D 算子改进 YOLO26 的上采样。(Converse 2D Convolution)通过正则化最小二乘闭式解与深度化空间建模结合,实现YOLO26特征提取与上采样阶段的卷积逆运算及特征精准重构。本文利用算子,通过闭式解的非迭代计算还原卷积下采样后的原始特征,再通过可学习核与自适应正则化参数稳定逆问题求解,同时依托圆形填充与插值初始估计保留特征高频细节,对YOLO26中不同尺度检测目标的特征进行针对性重构强化、抑制卷积/转置卷积带来的伪影与信息损失干扰,在特征融合与上采样

Limiiiing的博客 656

2、探索谷歌云平台(GCP):入门指南

本文全面介绍了谷歌云平台(GCP)的入门知识,涵盖其全球基础设施、核心服务和使用方法。内容包括GCP区域与可用区结构、创建免费账户、配置计算资源(如GCE虚拟机)、存储类型(PD、Filestore、GCS)、网络架构(VPC)、组织与项目管理、身份访问控制(IAM),以及计算、存储、数据库、大数据和人工智能等各类云服务。文章还提供了资源规划、安全管理和成本控制等实用建议,帮助用户快速上手并高效利用GCP构建可扩展的云解决方案。

uber9的博客 255

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

efm32 入门.pdf

本PDF共计22页,介绍的是操作驱动API,下载与否自己斟酌。

假人有限元模型

假人模型,车辆碰撞假人有限元模型,适合做乘员安全的研究者利用

NTscan扫描工具

ntscan是一个扫描器,可以扫描没有密码和弱密码的共享,先到配置文件中填写猜测的用户名和密码,扫描过程中右侧出现的是扫描日志。

上一篇: 2025_NIPS_FANS: A Flatness-Aware Network Structure for Generalization in Offline Reinforcement Learn
下一篇: 2025_NIPS_Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts
UnknownBody
博客等级 码龄16年 5601粉丝 4739原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值