【大模型精调-LORA实战】大模型精调LoRA 原理（二）

墨衍会员 · AI 创作全网分发 墨衍智能分发 本文由作者通过墨衍一键同步至各平台 1分发平台 100+累计阅读 112平均单平台已同步平台优快云微信公众号微博知乎掘金百家号博客园抖音小红书 你的文章也可以这样分发 墨衍会员 ¥399起/年，写一次发全网我也要 → 码龄18年

关注

原创 2026-11-08 22:51:12 112 阅读 AI 写同款· GEO 优化 ›

大模型入门核心技术专栏收录该内容

¥49.90 ¥99.00

139篇文章

订阅专栏

超级会员免费看

一、为什么需要 LoRA？—— 从“直觉”开始

参数高效精调（PEFT）是当前最实用的路线，而 LoRA 是其中的绝对主角。这一篇我们就把它彻底拆开：从直觉 → 数学本质 → 实战流程 → 避坑指南，争取让你读完就能动手跑起来。

一、为什么需要 LoRA？—— 从“直觉”开始

1. 全参数精调的痛点

假设我们有一个 7B（70亿参数）的模型，用 FP16 精度存储：

了解本专栏

订阅专栏解锁全文

超级会员免费看

收录于

当前文章被以下社区和专栏收录：

话题

#AtomGit「码动四季·开源同行」秋季征稿活动

标签

#人工智能 #机器学习 #LORA精调 #大模型精调

用 LoRA 精调 Qwen2.5 模型：从训练数据到权重产出全流程实战 LoRA 微调到底怎么做？这篇文章带你从 0 到 1 跑通一套完整的 Qwen2.5 微调流程：训练数据怎么构、模型怎么挂 LoRA、如何跑出第一轮训练结果、怎么保存 + 加载权重、如何用 LoRA 产物搭建 API 服务。阅读详情

相关推荐

Qwen2.5 模型精调实战：使用 QLoRA + 自定义指令微调

手把手教你如何使用轻量化技术 QLoRA 对 Qwen2.5 模型进行本地微调，包括准备指令数据集、搭建训练环境、训练参数配置、结果推理与评估部署，帮助你快速打造自己的专属国产大模型。

努力分享一些人工智能、计算机视觉、影像等相关的知识干货！ 1791

QLoRA 精调模型如何部署上线？FastAPI 封装 × Docker 打包 × 多模型热切换实战指南

很多人做到这一步已经训练出了一个挺不错的国产大模型微调版本，但随之而来的问题是：“我怎么把它做成一个 API？“怎么上线一套本地服务供团队调用？“要不要上 vLLM？用 Docker 好不好？我们先快速了解几种常见的部署方式，然后再进入实战。

努力分享一些人工智能、计算机视觉、影像等相关的知识干货！ 2328

TP4054充电电路发热严重？手把手教你调整电阻值降低温度（附实测数据）

本文针对TP4054锂电池充电电路发热严重的问题，提供了详细的排错与解决方案。通过分析发热原理，指出核心在于PROG引脚电阻值设定，并手把手指导如何计算和选择正确的电阻以降低充电电流与温度。文末附有不同电阻下的实测温度与性能数据，为电子爱好者和工程师提供了实用的调试参考。

p4q5r6s7t的博客 406

OpenAI Realtime API实时语音交互实战指南

实时语音交互是人机自然对话的核心能力，其本质在于构建低延迟、双向、流式协同的音频-语义闭环。它突破传统REST API的请求-响应范式，依托WebSocket实现毫秒级音频输入流（audio.delta）、模型生成流（text/audio.delta）与动态控制流（session.update）三重并行。该技术显著降低端到端延迟至300ms以内，支撑语音助手、智能客服、对话式教育等强实时场景。本文聚焦OpenAI Realtime API的工程落地，详解PCM音频流处理、VAD语音活动检测、AudioCon

weixin_32285357的博客 390

用Bus Hound和MIDIOX抓包分析USB MIDI数据流：从硬件接口到软件解析的完整实战

本文详细介绍了如何使用Bus Hound和MIDIOX工具抓包分析USB MIDI数据流，从硬件接口到软件解析的完整实战过程。通过解析MIDI协议、捕获Note On/Off事件、控制变更消息等实例，帮助开发者深入理解音乐数字接口的底层通信机制，并提供了常见问题的解决方案和调试技巧。

weixin_42692935的博客 404

虚拟机器人平台CoppeliaSim(Vrep)的逆运动学教学

相信看这篇博客的人都知道CoppeliaSim是做什么的，因此这里不多赘述。CoppeliaSim从最早的Vrep到现在已经更新了N个版本，从过去Vrep3.x开始使用的我这里小小地吐槽一下这个软件。CoppeliaSim每一个版本都有更新，问题是这些更新有时候不能向前兼容，而且时不时更新地让人不知所措（对于有紧急任务的时候真的很火大啊）。以本文所展示地逆运动学来说，CoppeliaSim在4.0.0之后又大变，推翻了过往所有的拖动模块链接之类的构建方式，转而采用直接通过代码来实现逆运动学。简单版逆运动

qq_39243015的博客 6698

AWS RDS 介绍三）创建 RDS 实例

AWS RDS 介绍共分为 7 个部分，其中第 3部分重点介绍，如何创建一个RDS 实例。三） AWS RDS 试用MySQL http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.

XuYongshi02的专栏 7645

卸载Office 16 Click-to-Run Extensibility Component详细步骤

卸载Office 16 Click-to-Run Extensibility Component详细步骤第一步：按住键盘上的“Win+R”，打开运行窗口在运行窗口输入“Installer”回车，打开Installer文件夹第二步：在Installer文件夹红框最右侧，即“大小”后面，右击选择“其他”，找到“主题”，勾选，点击“确定”，添加主题列成功第三步：在“主题”列找到“Office 16 Click-to-Run Extensibility Component

M_try的博客 3万+

教你搞懂RT-Thread的移植和配置ENV工具环境

RT-Thread的移植和配置ENV工具环境

Troubadour~的博客 8259

乐鑫科技 EchoEar 喵伴修改自定义唤醒词的方法 EchoEar喵伴、唤醒词修改、ESP32、AI萌宠、智能音箱、开发者指南

标签：乐鑫科技、EchoEar喵伴、唤醒词修改、ESP32、AI萌宠、智能音箱、开发者指南大家好，我是一名前端开发工程师，最近在玩乐鑫科技推出的时，发现它不仅萌趣十足，还能进行全双工语音交互、多模态识别，甚至能识别你的情绪和与你互动。EchoEar 喵伴是由乐鑫科技和火山引擎扣子大模型团队合作开发的，它使用乐鑫的芯片方案（ESP32 系列）+ 火山引擎扣子大模型，支持 WiFi 连接和本地处理，长记忆能力可以让它记住用户的偏好与习惯。

代码简单说 Vue、JAVA、PHP、Node.js 熟练运用，接口、架构、性能全搞定。 1876

WINCC7.3破解文件

WINCC7.3硬件够破解文件，有需要的下载！WINCC7.3硬件够破解文件，有需要的下载！

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")