墨衍会员
·
AI 创作全网分发
墨衍智能分发
本文由作者通过墨衍一键同步至各平台
1分发平台
2.3k累计阅读
2.4k平均单平台
已同步平台
优快云
微信公众号
微博
知乎
掘金
百家号
博客园
抖音
小红书
你的文章也可以这样分发
墨衍会员 ¥399起/年，写一次发全网
我也要
大语言模型（LLM）领域，如 GPT、LLaMA、GLM 等，虽然它们效果惊艳， 但动辄10 Bilion庞大的模型参数个人设备显存远不够训练，甚至推理困难。 几乎所有人都不会只满足于用Lora等方案fine-tuing大模型学会一些新的指令， 这约等于在教牛顿玩21世纪的智能手机，然而，这远远脱离了学习物理本身的奥妙。 此外，卖课付费订阅的营销号漏洞百出的一知半解讲解AI的教程遍地， 让理解LLM的优质内容雪上加霜，严重阻碍了学习者。
因此，本项目的目标是把上手LLM的门槛无限降低， 直接从0开始训练一个极其轻量的语言模型。
7200 Stars 750 Forks 32 Issues 4 贡献者 Apache-2.0 License Python 语言
主页: MiniMind Project
更多AI开源软件：AI开源 - 小众AI
Tip
（截至2024-9-17）MiniMind系列已完成了3个型号模型的预训练，最小仅需26M（0.02B），即可具备流畅的对话能力！
| 模型 (大小) | tokenizer长度 | 推理占用 | release | 主观评分（ |
|---|
从零训练一个小型LLM：minimind完整实战指南
大语言模型并非只能停留在API调用或超大规模训练的极端认知中。理解LLM训练的核心原理，实际上可以从一个更基础的起点开始——用桌面级硬件训练一个参数量在千万到亿级的Decoder-only Transformer。这类小型模型完整复现了从分词器训练、预训练、指令微调到偏好对齐的全流程，让开发者以极低成本亲手构建一个能对话的模型。在自然语言处理与AI工程实践中，掌握BPE分词、因果语言建模、学习率调度等关键技术，比单纯阅读论文更能建立扎实的技能树。无论是学生入门、工程师做数据策略验证，还是产品团队快速生成领域
比刷剧还爽！从零开始动手实践小模型训练指南
此外，小模型还能够在保持相对较高性能的同时，提供高效的解决方案，为广泛的应用场景带来了灵活性和可行性选择。：通过结合检索和生成的优势，创建一个RAG（Retrieval-Augmented Generation）系统，可以提升你的技术水平，并为你的项目增添实用价值。从零训练1B以下的小模型，对于资源有限的开发者来说，从零开始训练一个小模型是一个可行且有益的实践项目。最近，我在网上搜集了不少资料，主要是GitHub上的仓库和Arxiv上的论文，并将其记录在这里，供大家参考和学习。，参数最小达到0.5B。
NLP DAY1: 文本数据读取
txt 文本数据的读写CSV 及 Excel 文本数据的读写更多的 DataFrame 操作数据规模太大怎么办相信通过本次实验的学习，同学们可以游刃有余地面对各种数据文件了。在接下来的实验中，我们将开始真正地开启自然语言处理之旅，开始认识并熟悉一些常见的针对文本本身的预处理操作。
使用 CFD 仿真进行阀门性能分析：第一部分
了解如何使用 Ansys Discovery 评估阀门性能，使工程师能够设计出更好的产品。
vnpy_ctp源码下载后转变为python可用的处理过程
vnpy_ctp源码下载后在window中编译使用
从CLIP到AnomalyCLIP：零样本异常检测的跨领域进化之路
本文探讨了从CLIP到AnomalyCLIP的技术演进，重点介绍了零样本异常检测在跨领域应用中的突破。AnomalyCLIP通过对象无关提示学习和多尺度特征提取，显著提升了工业质检、医疗影像等领域的检测精度，实现了高达96.4%的金属螺母划痕检测精度和85.7%的皮肤癌敏感度。这一技术为多模态模型在异常检测中的应用开辟了新路径。
迈创cameralink采集卡安装操作说明
迈创cameralink采集卡安装操作说明MIL软件安装MIL软件下载获取MIL软件安装MIL软件安装常见问题 MIL软件安装 MIL软件下载获取 MIL软件安装 解压MIL安装包，点击MilSetup.exe开始安装，如下图所示； 备注：64位系统MIL软件可以同时安装32位和64位客户端，本文档默认安装64位客户端,64位系统安装32位MIL方法后续再展开讲（不建议这样用，很麻烦） 点击Launch Installation开始安装，64位系统，默认只安装64位MIL客户端 根据采集卡驱动，选择M
11、iOS设备逻辑数据分析全解析
本文深入解析iOS设备中的逻辑数据，涵盖图像相关数据库（Photos.sqlite和PhotosAux.sqlite）、录音文件、iPhoto照片存储结构、多媒体文件、第三方应用程序、社交网络应用（如Twitter、Skype、Facebook等）以及苹果相关应用（如Google Mobile、Bing）的数据存储结构和取证方法。此外，还介绍了文档类应用（如Pages、Numbers、Documents To Go）的文件存储结构及恢复方法。适用于iOS取证、数据挖掘和数字调查领域。
【推荐100个unity插件】unity易于使用模块化设计的天空、体积云和天气系统——Enviro 3
Enviro 3是一款功能强大的Unity天空与天气系统插件，提供动态天空、体积云、天气模拟等丰富功能。安装时需将预制体拖入场景并配置主相机，URP支持需激活深度纹理和添加渲染特性。模块化设计允许自由添加/移除时间、光照、雾效等模块，并通过API控制参数。推荐配合色调映射（如ACES模式）获得最佳视觉效果。插件支持季节、温湿度模拟及闪电风暴等特效，适用于需要逼真环境渲染的Unity项目。
深度学习工具包 DeepLearningToolbox
深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[](https://bitdeli.com/free "Bitdeli Badge")
复分析 阿尔福斯(Ahlfors) 第三版答案 英文
答案分为十章，有部分习题没有答案，全英文复分析阿尔福斯(Ahlfors) 第三版答案
ET2012新版免狗免安装绿色软件
ET2012新版免狗免安祖昂绿色软件。，新界面，新视觉，
精雕软件(JDpaint) 5.5 绿色版
精雕软件(JDpaint) 5.5 绿色版精雕软件(JDpaint) 5.5 绿色版精雕软件(JDpaint) 5.5 绿色版
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
308
扫一扫
