墨衍会员
·
AI 创作全网分发
墨衍智能分发
本文由作者通过墨衍一键同步至各平台
1分发平台
100+累计阅读
180平均单平台
已同步平台
优快云
微信公众号
微博
知乎
掘金
百家号
博客园
抖音
小红书
你的文章也可以这样分发
墨衍会员 ¥399起/年，写一次发全网
我也要
本文聚焦“数据受限但计算资源无限”的语言模型预训练场景（因预训练计算量年增长4倍，而网络文本数据年增长仅1.03倍），核心研究如何在固定数据下通过优化算法实现高效预训练，具体内容如下：
LLM Training
当前文章被收录于：
LLM Training
主要是LLM训练相关的paper，针对训练加速，微调等相关内容。涉及的关键词包括finetuning以及training。
损失函数与优化器：从「梯度是怎么被造出来的」到「梯度是怎么被用掉的」
本文揭示损失函数与优化器本质一体：优化器仅依赖损失产生的梯度，而损失决定梯度的方向、尺度与噪声特性。错误的损失选择（如分类用MSE）会导致梯度失衡，使优化器失效；不同损失（如Huber、Focal）通过调控梯度分布影响优化过程。实测表明，归约方式、标签结构与损失配对直接决定梯度行为，进而影响收敛性与泛化。核心观点：选对损失，才是让优化器“看得懂”的前提。
LingBot-World 2.0：Infinite Worlds with Versatile Interactions
† 项目负责人Abstract摘要我们提出 LingBot-World 2.0(也称为 LingBot-World-Infinity)，这是 LingBot-World 的一个高级迭代版本，具有四项显著升级。(1)我们的模型在保持稳定输出质量的同时，实现了无界的交互时域，这得益于精心设计的因果Pretraining 范式。(2)通过从基础模型中蒸馏出一个实时变体，我们的系统可保证快速响应时间，足以驱动720p、60fps的视频流。
【信息科学与工程学】【安全领域】第八十三篇 云安全解决方案02
企业中“搞人”与“搞事情”常见问题的分析框架。我将选取4个典型场景，每个场景都从实际管理现象出发，逐步推导出数学模型、参数设定及关联知识。
从零构建RK3568视频处理管线：C++ V4L2捕获与RKMPP硬编解码实战
本文详细介绍了在RK3568平台上，使用C++语言从零构建高性能视频处理管线的实战过程。核心内容包括通过V4L2接口高效捕获摄像头视频流，并利用RKMPP硬件编码器实现零拷贝的H.264/H.265硬编码。文章深入讲解了设备初始化、DMA缓冲区管理、编码器参数配置以及主循环协同工作等关键步骤，旨在帮助开发者构建低延迟、高效率的嵌入式视频应用。
毫米波雷达dd-mimo中发射天线辐射方向图变化
本文详细介绍了dd-mimo的相位调制原理，以及分离天线的办法，最后演示了在dd-mimo中因为相位调制带来的辐射方向图的变化。
AD9026芯片开发实战指南：从硬件配置到射频校准
本文为AD9026芯片开发实战指南，详细解析了从硬件设计、电源时钟配置、SPI初始化到多芯片同步与JESD204高速链路建立的完整流程。重点探讨了射频校准技术，包括初始化校准与跟踪校准，以保障5G、MIMO等现代通信系统所需的射频性能。旨在帮助开发者系统掌握这颗高性能射频收发器的应用，规避常见设计陷阱。
Flowable多数据源实战：从配置到指定Schema的避坑指南
本文详细介绍了Flowable多数据源配置的实战指南，包括基础配置、数据源Bean创建、指定Schema的进阶配置以及常见问题排查。通过具体代码示例和避坑建议，帮助开发者高效实现多租户系统中的工作流引擎集成，特别强调了schema设置和事务管理的注意事项。
零基础教程：用Python给试卷照片自动切题（有道OCR+PIL完整流程）
本文为零基础用户提供了一套完整的Python自动化切题方案。通过调用有道OCR API识别试卷照片中的题目坐标，并利用PIL库进行精准裁剪，将杂乱的试卷图片自动分割为独立的题目图像。教程详细涵盖了环境搭建、图像预处理、API调用、坐标解析与批量处理全流程，帮助教育工作者和技术爱好者高效处理大量试卷，实现工作流程自动化。
【AUTOSAR】MCU Port的配置（MCAL）【EB】
熟悉Classic AUTOSAR架构的朋友应该知道，经常需要用到MCU中的Port这个模块。那么如何通过工具快速配置一个Port呢?
结构光三维重建2——多频外差解包裹
本文深入解析了结构光三维重建中的多频外差解包裹技术。针对四步相移法产生的相位包裹问题，该方法通过投影多组不同频率的条纹光，利用外差原理逐级解算，最终获得精确、连续的绝对相位值，是实现高精度、稳健三维重建的关键步骤。
【DeepSeek开发入门】DeepSeek API 开发初体验
在人工智能蓬勃发展的时代，DeepSeek作为一款功能强大的AI大模型，为开发者提供了丰富的接口和便捷的开发体验。本文将基于DeepSeek的几个关键接口，带你快速入门，开启DeepSeek开发的探索之旅。通过本文，你已经初步了解了DeepSeek开发中几个重要接口的使用方法，包括列出模型、对话补全和查询余额接口，以及开发过程中的常见问题及解决办法。这是你踏入DeepSeek开发领域的第一步。在后续的学习中，你可以深入研究模型的参数调优，探索不同模型在各种场景下的应用，如利用模型实现更复杂的推理对话功能。
深度学习工具包 DeepLearningToolbox
深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[](https://bitdeli.com/free "Bitdeli Badge")
ET2012新版免狗免安装绿色软件
ET2012新版免狗免安祖昂绿色软件。，新界面，新视觉，
VC中sqlite的使用
在VC中对sqlite的使用，实现数据库大批数据的快速插入，读取数据库中的数据在ListCtrl中显示
android 蓝牙通讯demo
android 下蓝牙通讯demo，实现了发现蓝牙，使能蓝牙，通过蓝牙发送消息。
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
超级会员免费看
订阅专栏 解锁全文
332
扫一扫
