Corvid : Improving Multimodal Large Language Models Towards Chain-of-Thought Reasoning

在这里插入图片描述

文章主要内容总结

本文提出了一种增强链式思维(Chain-of-Thought, CoT)推理能力的多模态大语言模型(MLLM)——Corvid,旨在解决现有MLLMs在复杂结构化推理中的局限性。

  1. 核心挑战:现有MLLMs存在三大问题:高质量多模态CoT数据短缺、视觉表征与跨模态对齐不足、推理时易出现过度推理或推理不足。
  2. 模型架构:
    • 采用混合视觉编码器(整合SigLIP ViT和ConvNeXt-XXL),增强视觉信息提取;
    • 设计新型连接器GateMixer,通过门控机制和选择性注意促进跨模态交互与对齐。
  3. 数据集与训练:
    • 构建高质量多模态CoT指令数据集MCoT-Instruct-287K,标准化并优化了多种推理数据;
    • 采用三阶段训练:多粒度对齐预训练、CoT增强监督微调、纯CoT指令调优,逐步提升分步推理能力。
  4. 推理策略:提出自验证策略,通过跨模态相似度和模型置信度的加权评估,动态选择直接推理或CoT推理,缓解过度/不足推理。
  5. 实验结果:Corvid在数学推理、科学问题解决
船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

SSE消息推送避坑指南:从Chrome兼容性到Spring超时配置

本文深入探讨了在Java Spring Boot项目中实现SSE消息推送时,从浏览器兼容性到后端配置的完整避坑指南。针对Chrome并发连接限制、Safari自动断开、Nginx代理缓冲以及Spring SseEmitter内存泄漏与超时管理等生产环境常见问题,提供了具体的解决方案和最佳实践配置,帮助开发者构建稳定可靠的实时推送系统。

work3的博客 451

别再手动填序号了!用Python脚本在Arcgis属性表里实现分组自动编号(附完整代码)

本文详细介绍了如何利用Python脚本在Arcgis属性表中实现分组自动编号,告别低效的手工操作。通过核心代码解析和实战教程,帮助GIS工程师快速掌握自动化编号技术,提升工作效率并降低错误率。特别适合城市规划、国土调查等需要处理大量分组数据的场景。

weixin_42532473的博客 309

疾病负担研究(GBD)——如何使用joinpoint软件计算AAPC

疾病负担研究(GBD)——如何使用joinpoint软件计算AAPC

weixin_44240228的博客 1万+

慕课软件质量保证与测试(习题集)

慕课金陵科技学院.软件质量保证与测试.期末复习.习题汇总0 总目录1 绪论1.1 软件质量保证与测试的产生与发展1.2 软件缺陷,软件错误,软件故障1.3 软件质量保证与测试的意义,原则和挑战1.4 单元测试1.5 课后作业2 软件测试策略2.1 软件测试的模型,过程和生命周期2.2 软件测试的方法和技术2.3 单元测试2.4 课后作业3 黑盒测试3.1 黑盒测试概述3.2 等价类划分测试3.3 边界值测试3.4 错误推测法3.5 判定表驱动法3.6 因果图法3.7 场景法3.8 正交实验法3.9 单元测试

weixin_42473228的博客 5万+

腾讯云运维工程师认证TCA--真题(最新2022.11)

解析:云原生应用的12要素,1.基准代码:一份基准代码,多份部署;解析:腾讯云主账号具备对其名下资源(包括存储桶)的最大权限,您虽然可以在存储桶策略中限制几乎所有操作,但主账号始终具备 PUT Bucket Policy 操作的权限,主账号调用该操作不检查存储桶策略。解析:瀑布模型将软件生命周期划分为制定计划、需求分析、软件设计、程序编写、软件测试和运行维护等六个基本活动,并且规定了它们自上而下、相互衔接的固定次序,如同瀑布流水,逐级下落。解答:云原生五大关键技术:微服务、容器化、DevOps、持续交付。

e1210101的博客 9915

AI工程化之生成式UI & A2UI(一)

本文系统介绍了生成式UI和Google A2UI技术。生成式UI通过AI直接生成前端代码(完全生成式)或DSL(混合式),实现千人千面的动态界面和研发提效。Google A2UI提出组件级和消息级协议,采用JSON规范实现渲染框架解耦,支持多端渲染。文章详细阐述了A2UI的架构设计,包括协议生成、服务端处理和渲染器实现,并对比了与传统前端框架的差异,指出A2UI本质是UI=F(AI(UI)+AI(state))的创新模式。最后通过playground项目展示了A2UI的实践应用。

qq_34306228的博客 591

PyTorch深度学习实战(3)——使用PyTorch构建神经网络

PyTorch 是一个用于构建深度神经网络的库,具有灵活性和可扩展性,可以轻松自定义模型。在本节中,我们将使用 PyTorch 库构建神经网络,利用张量对象操作和梯度值计算更新网络权重,并利用 Sequential 类简化网络构建过程,最后还介绍了如何使用 save、load 方法保存和加载模型,以节省模型训练时间。

盼小辉丶的博客 8万+

SAP Fiori Launchpad 自定义层文本翻译实战:精准修正 Tile、Catalog 与 Group 标题

SAP Fiori 项目中,入口层(tile/catalog/group)的翻译问题常被忽视,但其直接影响用户体验。与前端i18n不同,这些文本属于平台配置数据,需通过ABAP标准翻译流程(SE63)处理。关键步骤包括:使用/UI2/FLC定位Text ID和Configuration ID,根据前缀区分Main/Property WD Config Key,在SE63中锁定WDY_CONF_USERT2表对象进行翻译。操作时需注意使用系统源语言环境,翻译完成后可能需要清理缓存。

2007 年 ~ 2025 年,深耕 SAP 技术 18 年 173

C# WinForm 基于DirectShow.dll编写的程序,可边预览边保存视频,音频和图片

基于微软的DirectShow.dll的二次开发1.自动识别连接的摄像头跟麦克风且可以对其进行选择(默认选中第一个)2.可录制wmv和avi两种格式的视频并且都带有音频,avi可以录制的分辨率和帧数进行选择,wmv默认进行了淸晰度优化如不喜欢可去除代码中有相关提示3.可在预览点开之后进行拍照,也可边录制视频边拍照4.添加了视频和音频的压缩功能(酌情使用)这次二次开发发现C#对DirectShow的相关信息太少,这次做完也不敢藏私共享出来,还望大家指正

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

VC中sqlite的使用

在VC中对sqlite的使用,实现数据库大批数据的快速插入,读取数据库中的数据在ListCtrl中显示

迈瑞BS系列全自动生化分析仪LIS协议接口手册(内含HL7和ASTM)

迈瑞BS系列全自动生化分析仪LIS协议接口手册,内含HL7和ASTM)

上一篇: Elite Polarization in European Parliamentary Speeches: a Novel Measurement Approach
下一篇: LLM-based Realistic Safety-Critical Driving Video Generation
UnknownBody
博客等级 码龄16年 5601粉丝 4739原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值