2025_NIPS_Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models

在这里插入图片描述

文章总结与翻译

一、主要内容

本文聚焦多模态大语言模型(MLLMs)在3D空间推理能力上的不足,核心目标是激活其空间推理潜能,以满足机器人技术、自主系统等领域对3D场景理解的需求。

核心背景

现有MLLMs虽在2D视觉理解上取得显著进展,但难以捕捉3D推理所需的跨视角一致性等精细空间信息,且受限于2D数据训练,易依赖表层像素特征而非真正的3D空间关系。

核心方法

  1. 提出视角学习(Viewpoint Learning):作为评估和提升MLLMs空间推理能力的基础任务,通过理解图像对的视角变化,帮助模型感知3D一致性。
  2. 构建Viewpoint-100K数据集:包含10万组以物体为中心的真实图像对及对应的问答对,涵盖相机水平平移、旋转等场景,支持自我中心和物体中心两种参考框架的推理。
  3. 两阶段微调策略:
    • 第一阶段(基础知识注入):基于Viewpoint-100K数据集进行有监督微调(SFT),结合混合冷启动初始化(融合思维链模板),让模型学习视角表示并保持连贯推理。
    • 第二阶段(泛化能力增强):在SAT数据集上采用组相对策略优化(GRPO
船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

AI 应用开发:四维架构落地实战——以 HR Agent 为例拆解感知·推理·记忆·执行【意图路由、规划编排、长期记忆、工具执行、面试题】

本文基于四维架构(感知·推理·记忆·执行),以HR Agent为例,系统拆解大模型应用从“聊天”到“生产可用”的落地路径。通过Spring AI实现意图路由、规划编排、RAG检索与工具调用,结合真实踩坑复盘,揭示架构化设计对安全、成本与可维护性的关键价值,为AI Agent工程化提供可复用的实战框架。

王大师企业官方博客 719

Pixhawk变砖自救指南:从DFU模式到固件重刷全流程(附2.4.8实测)

本文详细介绍了Pixhawk飞控变砖后的自救方法,从DFU模式进入、固件重刷到高级恢复技巧全流程。特别针对国产版本boot0引脚未引出的特殊情况提供了解决方案,帮助用户通过QGC或Mission Planner地面站工具恢复飞控功能,适用于2.4.8等常见固件版本。

weixin_29168813的博客 339

DAC8568实战指南:如何用STM32实现8通道DAC精准控制(附代码)

本文提供DAC8568与STM32结合的实战指南,详细解析8通道16位DAC的精准控制方法。内容涵盖硬件连接、SPI配置、驱动层构建、时序调试及多通道同步输出等关键步骤,并附有经过验证的HAL库驱动代码,帮助工程师快速解决实际项目中的通信与输出稳定性问题。

weixin_29175469的博客 746

单细胞分析(五)——使用Harmony进行数据整合和去批次

为了保证单细胞RNA测序数据的准确性和可靠性,需要对数据进行样本去批次处理。去批次的方法包括Harmony,SeuratV3等,这些方法可以有效地消除批次效应和技术噪声,并提高单细胞RNA测序数据的质量,为后续的数据分析提供更加可靠的基础。

阅读文献,记录学习笔记 3万+

AMD KFD的SDMA Packet 类型和定义解析

本文详细分析了AMD GPU架构中的SDMA(System Direct Memory Access)数据传输机制,重点介绍了其核心数据结构——SDMA Packet。SDMA通过8种主要Packet类型(包括内存拷贝、数据写入、屏障同步、中断触发等)实现高效主机-设备通信。每种Packet采用位域结构和联合体封装设计,在保证功能完整性的同时实现空间优化。这些Packet支持丰富的访问属性和扩展字段,适用于高性能数据搬运、内存初始化、任务同步、性能分析等场景,为GPU计算提供了灵活高效的数据传输解决方案。

图形/计算GPU的驱动与虚拟化技术分享园地 1254

第38节:cesium 风场效果(含源码+视频)

cesium 风场效果

qq_29384639的博客 2331

Prometheus+Grafana+alertmanager+PrometheusAlert监控告警 实现飞书告警

以上 是以grafana做的邮件告警,但是感觉邮件很少人看。下面是以webhook 可以接入。启动alertmanager (设置 开机在启动)下载prometheusalter 配置文件。这里的告警是基于grafana上搭建的。我们启动PrometheusAlert。写完之后进行重启Prometheus。prometheus 的告警规则。测试 基于我们生产环境做的文档。然后在grafana上导模板。上导入模板就可以 直接引用。访问 grafana页面。重启一下grafana。

墨痕诉清风的博客 859

将GroundDINO、EdgeMTAM和FoundationPose的ONNX模型转换为TensorRT格式并部署到NVIDIA Jetson Orin平台的完整技术指南

TensorRT模型部署指南:从ONNX到Jetson Orin的优化实践 本文详细介绍了将GroundDINO、EdgeMTAM和FoundationPose模型从ONNX格式转换为TensorRT引擎并在Jetson Orin平台部署的全流程。主要内容包括: 环境配置:JetPack 5.1.2+环境搭建,TensorRT工具链安装,存储与交换空间优化 模型转换:动态尺寸处理、自定义算子实现、层融合策略及INT8量化校准 部署优化:引擎序列化加密、多模型并行推理架构设计、CUDA流异步执行 实战技巧。

huanghm88的专栏 438

15、虚拟环境中的数据保护与恢复方案解析

本文深入解析了虚拟环境中两种重要的数据保护与恢复解决方案:RecoverPoint for Virtual Machines(RP4VM)在VxRail上的设计,以及Veeam Backup & Replication(VBR)与VxRail的集成方案。文章详细介绍了RP4VM的网络布局、故障转移场景及其相关问题,以及VBR的架构设计、备份与复制流程、许可证选项及其多平台支持特性。此外,还探讨了RP4VM与VBR结合使用的综合解决方案,以及未来数据保护领域的发展趋势,如智能化管理、云原生集成和零信任架构,为

t4y5u6i7o的博客 96

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

文档:亲测可用解决vnc灰屏,黑屏,鼠标大黑叉情况

主要解决vnc-viewer灰屏,黑屏,鼠标大黑叉等现象,详细见文档。

VC中sqlite的使用

在VC中对sqlite的使用,实现数据库大批数据的快速插入,读取数据库中的数据在ListCtrl中显示

IDEA的使用指南

IDEA基于java的使用手册,可以最快速的上手,在使用中不断的丰富自己的使用技巧.

上一篇: 2025_NIPS_Guiding Cross-Modal Representations with MLLM Priors via Preference Alignment
下一篇: 2025_NIPS_SparseDiT: Token Sparsification for Efficient Diffusion Transformer
UnknownBody
博客等级 码龄16年 5601粉丝 4739原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值