wang2008start的专栏

Value Function Approximation

Policy Gradient Methods

Exploration and Exploitation

Integrating Learning and Planning

Introduction Agent and Environment,History and state, Agent state, Environment state, Information stat, Fully observable enviroments, Partially observable enviroments 环境完全可观测 环境部分可观测 Policy:

Markov Process Markov Reward Process Markov Decision Process (Markov reward process with decisions) a policy is a distribution over actions given states. GIven an MDP and policy,

Policy Evaluation Policy Iteration Value Itera 2. Policy Iteration(Any optimal policy can be subdivided into two components:An optimal first action A,Followed by an optimal poli

前面的动态规划主要用来解决model已知的MDP问题，这里主要解决model/环境未知时的MDP预估价值函数问题，方法主要有： MC方法：不需要知道转移矩阵或回报矩阵，在非马尔科夫环境中高效 时序差分方法：Monte-Carlo Learning 直接从experience的episode中学习 不需要MDP的transition、rewards 主要思想：value = mean return

(Optimise the value function of an unknown MDP)On-policy learning —— Learn about policy π from experience sampled from πOff-policy learning —— Learn about policy π from experience sampled from u On-Po