HsinglukLiu的博客

这里需要画个图需要把之前的东西拿过来本来是要吧全部样本拿过来学习后来是随机梯度，就是学一个样本更新一遍之后是batch，学一部分，更新一下这几个需要解释一下。Experience replay：transiiton库存储以及batch训练的详细解析和loss计算的理解我们在replay buffer中存储的transitions的形式均为(st,at,rt,st+1)(s_t, a_t, r_t, s_{t+1})(st​,at​,rt​,st+1​)，因此，我们可以用一个数组或者Dat.

【强化学习实战-06】Policy based RL保姆级教程：以Cart Pole为例强化学习的Policy based RL和Value based RLPolicy based RL价值函数：value functionPolicy based RL: 详细理论介绍Actor-CriticActor-Critic莫凡的完整代码：作者：刘兴禄，清华大学博士在读本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?f

【Pytorch】| Pytorch中softmax的dim的详细总结关于softmax的理解一维向量：dim=0和dim=-1结果相同，dim=1和dim=2会报错二维张量：dim=1和dim=-1结果相同，dim=2会报错最终结论作者：刘兴禄，清华大学博士在读欢迎关注我们的微信公众号 运小筹关于softmax的理解Softmax的公式为softmax(xi)=exi∑iexi\text{softmax} (x_i) = \frac{e^{x_i}}{\sum_{i}{e^{x_i}}}s

【强化学习】DQN、Double DQN、Dueling DQN的总结DQNDouble DQN处理过高估计问题：Double DQNDueling DQNDQN(图片来自Wang Shusen的教学视频,网址：https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0)综上，我们来总结一下用TD-learning的DQN版本的算

【强化学习实战-05】Dueling DQN保姆级教程：以Cart Pole为例Dueling DQNAdvantage function (优势函数)Dueling DQN参考文献：Dueling Network Architectures for Deep Reinforcement Learning https://arxiv.org/abs/1511.06581Wang Z, Schaul T, Hessel M, et al. Dueling network architectures

【强化学习实战】DQN和Double DQN保姆级教程（2）：以MountainCar-v0为什么用Deep Q Network (DQN)实战：用Double DQN求解MountainCar问题MountainCar问题详解MountainCar问题的源代码解释cartpole.pyMountainCar的状态(Observation)MountainCar的动作MountainCar的目的DQN 求解MountainCar问题：完整代码详解定义神经网络Q(w)Q(\mathbf{w})Q(w)神经网络

【强化学习】概念梳理：强化学习、马尔科夫决策过程与动态规划动态规划(Dynamic programming)马尔科夫链（Markov Chain）马尔科夫决策过程和强化学习马尔科夫决策过程和动态规划强化学习的基本概念状态(State)和动作(Action)策略(Policy) π\piπState transition`reward`和`return`Action-Value function最优动作-价值函数(Optimal Action-Value function)状态-价值函数(State-Valu

【强化学习实战】Q-learning:向右走的小例子向右走：Q-learning本笔记根据【莫凡】的强化学习视频整理得来向右走：Q-learning算法伪代码"""A simple example for Reinforcement Learning using table lookup Q-learning method.An agent "o" is on the left of a 1 dimensional world, the treasure is on the rightmos

pytorch或tensorflow报错：ImportError: TensorBoard logging requires TensorBoard version 1.15 or above报错信息：pytorch或tensorflow报错：ImportError: TensorBoard logging requires TensorBoard version 1.15 or above查看pycharm的settings发现我安装的是1.14版本，我尝试在Pycharm里面的settings

离散控制与连续控制本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0注意：不能直接把DQN应用在连续控制问题上

确定策略梯度+Deterministic+Policy+Gradient本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0DDPG: deep Deterministic Policy Gradient...

随机策略做连续控制本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0

REINFORCE vs.A2C本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0

Advantage Actor-critic: A2C本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0

REINFORCE with baseline本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0

Policy gradient with baselinebaseline本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0baseline

Multi-agent reinforcement learning第3种架构本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0第3种架构

Multi-Agent+Reinforcement learning术语Convergence本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0术语这里，a1,a2,⋯ ,ana^1, a^2, \cdots, a^na1,a2,⋯,an表示nnn个age

Dueling netrwork -可以再DQN及其他地方Advantage function (优势函数)本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Advantage function (优势函数)A∗(s,a)=Q∗(s,a)−V∗(s)A^{*}(s,

Target network & Double DQNTarget network & Double DQNTarget NetworkDouble DQN本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Target network & Double D

Experience replay：经验回放 -DQN的高级技巧Experience replay本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Experience replay用优先经验抽样代替随机抽样 右边的场景更加重要

Multi-step TD target本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Multi-step TD target是对TD算法的一种改进。注意：上面的Sarsa和Q-learning都是只包含了一个奖励rtr_trt​，如果包含多个奖励，就是multi

@[TOC](Q-learning (off-policy): 用来训练最优动作价值函数)本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0价值网络Qπ(s,a;θ)Q_{\pi}(s, a; \theta)Qπ​(s,a;θ)是动作价值函数Qπ(s,a)Q_{\pi}(

Policy-based reinforcement learningPolicy function approximationState-value function approximation本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Policy function a

SarsaDerive TD Target本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Derive TD Target

Monte Carlo AlgorithmCalculating π\piπ本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Monte Carlo Algorithm: Monte Carlo是一大类随机算法，通过用随机样本来估算真实值 。Calculating π\piπ

Policy-based reinforcement learningPolicy NetworksBehavior CloningTrain policy network using Policy gradientTrain the value networkMente Carlo Tree Search本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=1827226606

Actor-critic algorithmActor-critic algorithmValue network and policy network训练神经网络Summary本笔记整理自 (作者: Shusen Wang):https://www.bilibili.com/video/BV1rv41167yx?from=search&seid=18272266068137655483&spm_id_from=333.337.0.0Actor-critic algorithmVa

Value-based reinforcement learningValue-based reinforcement learningAction-value functionsDeep Q Network (DQN)训练神经网络的算法：`Temporal difference algorithm`一个例子Apply TD learning to DQNSummary参考文献本文整理自教学视频 (作者: Shusen Wang)：https://www.bilibili.com/video/BV1rv4

强化学习基本概念

文章目录Reinforcement learning + OR的论文Reinforcement learning + OR的论文Learning to Search in Branch-and-Bound Algorithms代码：https://github.com/hhexiy/scip-dagger