现代机器学习研究进展 第一部分 数学与概率工具箱 第8章 随机近似与退火方法(原理推导版)

本章从**随机梯度下降(SGD)作为随机动力学(Langevin)**的视角出发,严格推导相关 SDE 极限、Fokker–Planck 描述、不随 Metropolis 的离散化偏差、温度退火(simulated annealing)收敛条件、以及 Markov 链蒙特卡洛(MH/HMC)基础与在深度学习中的变体(SGLD、pSGLD、U/Langevin 等)。


8.1 从 SGD 到 Langevin:扩散近似与极限 SDE

8.1.1 问题设定与记号

8.1.2 离散 SGD 与受噪 Langevin 迭代

8.1.3 精确到常数的离散→连续推导(中心极限定理风格)


模拟退火算法(Simulated Annealing) 原理与代码实例讲解 模拟退火算法(Simulated Annealing) - 原理与代码实例讲解 作者:禅与计算机程序设计艺术 1. 背景介绍 1.1 优化问题与启发式算法 在计算机科学和运筹学领 阅读详情

相关推荐

AI学习与模拟退火算法

作者:禅与计算机程序设计艺术 1.简介 随着计算机的发展和普及,人们发现了使用计算机进行各种各样的任务的可能性。然而,现实生活中的问题却并非都可以用计算机解决。许多时候,计算机需要处理的数据量过于庞大、需求极其复杂,无法直接在计算机上运行的算法和模型就显得尤为重要。于是,人工智能(Artificial

AI天才研究院 1694

模拟退火算法在TSP问题中的C++实现与优化

本文还有配套的精品资源,点击获取 简介:模拟退火算法是一种模仿物质退火过程的启发式搜索技术,用于寻找复杂问题的近似最优解。在旅行商问题(TSP)中,该算法尝试找到访问一系列城市一次并返回起点的最短路径。本文详细介绍了模拟退火算法的C++编程实现,包括算法流程和优化策略,并通过C++程序代码解决TSP问题,展示了算法如何处理城市数据和优化搜索过程。 1. 模拟退火算法原...

weixin_36001279的博客 1216

PCB中的CAF深度剖析与对策研究

摘要:本文系统研究了PCB板的导电阳极丝(CAF)问题,阐明了CAF是在电场作用下铜离子沿玻纤微裂通道迁移形成导电丝的现象,其形成需铜、水汽、电解质、偏压和迁移通道五个条件。CAF会导致PCB绝缘性能下降和信号传输异常,严重影响电子产品可靠性。研究提出了板材选择、钻孔工艺控制、叠板结构优化等预防措施,以及增大孔距、错位排列等设计注意事项,并介绍了IPC-TM-650-2.6.25测试标准。未来应在耐CAF材料、测试技术和制造工艺方面深入研究,以提升PCB可靠性。(150字)

2401_84691632的博客 1704

【Linux】Petalinux使用QSPI FLASH引导启动

1. 预分配Flash空间使用Flash引导Petalinux做的系统,也有进阶的做法。部分图片和经验来源于网络,若有侵权麻烦联系我删除,主要是做笔记的时候忘记写来源了,做完笔记很久才写博客。1. 预分配Flash空间根据生成的bit文件大小估算BOOT.BIN文件的大小,为Flash大概分配四个分区:boot、bootenv、kernel、bootscr,分别存放BOOT.BIN文件、u-boot的环境变量、内核文件(image.ub/zImage/uImage等)、启动脚本boot.scr。

Tesseract_9527的博客 3375

Qt上位机实战:5步搞定STM32 IAP固件升级(附拖拽文件处理技巧)

本文详细介绍了如何使用Qt开发STM32 IAP固件升级上位机。通过五个核心步骤,包括理解IAP机制、搭建Qt环境、实现拖拽文件加载、封装串口通信以及设计可靠通信协议,帮助开发者快速构建一个稳定、易用且支持图形化操作的固件升级工具,显著提升嵌入式产品开发与维护效率。

ss78901的博客 1078

N5105核显直通PVE:从黑屏到HDMI输出的完整指南

PCI直通能让虚拟机独占物理设备,但核显直通远没有独显那么轻松。Intel核显与CPU、显示输出链路的强耦合使UEFI引导、GOP驱动、驱动签名等环节都成了黑屏陷阱。在PVE虚拟化环境中,通过IOMMU开启、vfio-pci绑定、OVMF固件和Q35机型配合,可以绕过KVM检测,让Windows虚拟机正确驱动核显,从HDMI口输出画面,并解锁Intel Quick Sync硬解能力。这种方案尤其适合N5105等低功耗迷你主机,可作为客厅HTPC、本地视频硬解或依赖硬件加速的轻量工作台。本文基于N5105平台

weixin_31082403的博客 375

MySQL安装配置教程(超级详细)

一、 下载MySQL Mysql官网下载地址:MySQL :: Download MySQL Installer (Archived Versions) 1. 选择要安装的版本,本篇文章选择的是5.7.31版本,点击Download下载 正在上传…重新上传取消 二、 安装MySQL 1. 选择设置类型双击运行mysql-installer-community-5.7.31.0.msi 这里选择是自定义安装,所以直接选择“Custom”,点击“Next” “Developer Default”是

m0_67322837的博客 5万+

Ubuntu断网环境下的系统更新:apt-offline保姆级教程(附常见问题排查)

本文详细介绍了在Ubuntu断网环境下使用apt-offline进行系统更新的完整教程,包括环境准备、签名文件生成、数据传输方案对比及常见问题排查。特别适用于企业级开发环境或安全敏感场景,帮助运维人员高效完成离线系统更新,确保系统安全与稳定。

weixin_29214691的博客 359

Unity游戏开发实战:用柏林噪声打造3D随机地形(附完整C#代码)

本文详细介绍了如何在Unity游戏开发中使用柏林噪声算法生成3D随机地形,包含完整的C#代码实现。从基础柏林噪声原理到高级参数控制,再到性能优化和常见问题解决,帮助开发者掌握程序化地形生成技术,打造丰富多变的游戏世界。

weixin_29281941的博客 101

告别编译失败:详解Keil MDK中ARM编译器V5与V6的区别及项目迁移选择

本文深入解析Keil MDK中ARM编译器V5与V6的核心差异,提供STM32项目迁移的完整指南。从代码生成策略、优化重点到语法支持,详细对比两版本特性,并给出项目兼容性评估框架和双版本共存配置方案,帮助开发者高效解决编译器版本冲突问题。

weixin_28684757的博客 516

lwip-2.0.3

lwip是瑞典计算机科学院(SICS)的Adam Dunkels 开发的一个小型开源的TCP/IP协议栈。实现的重点是在保持TCP协议主要功能的基础上减少对RAM 的占用。

jre9下载 jdk9

(Java SE Runtime Environment 9.0) 9.0u175 官方最新版 【x86|x64】

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

Camera2Demo

在Camera2 的基础上进行封装,使得Camera2一堆初始化逻辑与Activity分离,降低耦合度,完成预览拍照并保存的操作

上一篇: 现代机器学习研究进展 第一部分 数学与概率工具箱 第七章 — 自动微分、数值稳定性与微分方程数值解法(理论推导版)
下一篇: 现代机器学习研究进展 第二部分 模型与架构 第9章 — 卷积网络与现代变体(原理推导)
VectorShift
博客等级 码龄1年 1829粉丝 1103原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

VectorShift

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值