Reinforcement Learning Exercise 5.5

最新推荐文章于 2019-10-03 12:31:59 发布

YeXiang\^-^/

最新推荐文章于 2019-10-03 12:31:59 发布

阅读量1.1k

点赞数

CC 4.0 BY-SA版权

分类专栏： reinforcement learning 文章标签： reinforcement learning

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接：https://blog.youkuaiyun.com/ballade2012/article/details/98348253

在含有单一非终态状态和单一行动的MDP中，概率p返回非终态，1-p进入终态，奖励始终为1，γ=1。在一个持续10步并得到回报10的实验中，首次访问和每次访问状态下价值估计分别为p和1-p/(p^10-1)。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Exercise 5.5 Consider an MDP with a single nonterminal state and a single action that transitions back to the nonterminal state with probability $p$ and transitions to the terminal state with probability $1 - p$ . Let the reward be $+ 1$ on all transitions, and let $γ=1\gamma=1$ . Suppose you observe one episode that lasts 10 steps, with a return of 10. What are the first-visit and every-visit estimators of the value of the nonterminal state?

For the first-visit estimator, only the first visit of a state is considered. So:
$\begin{aligned} V(S_{nonterminal}) &= G(S_0) \\ &= 1 \cdot p + 0 \cdot (1-p) \\ &= p \end{aligned}$

最低0.47元/天解锁文章

200万优质内容无限畅学

博客等级

码龄14年

37
原创

12
点赞

11
收藏

11
粉丝

关注

私信

热门文章

分类专栏

reinforcement learning 37篇

上一篇：: Reinforcement Learning Exercise 5.4

下一篇：: Reinforcement Learning Exercise 5.6

最新评论

Reinforcement Learning Exercises 6.1
程序猿学长: mark
Reinforcement Learning Exercise 5.5
mymeromance: 请问这道题不用考虑probability吗？
Reinforcement Learning Exercise 5.5
Alisonyao: 我恐怕更赞同这里的做法：https://github.com/LyWangPX/Reinforcement-Learning-2nd-Edition-by-Sutton-Exercise-Solutions/blob/master/Chapter%205/Solutions_to_Reinforcement_Learning_by_Sutton_Chapter_5_r3.pdf
Reinforcement Learning Exercise 4.4
YeXiang\^-^/ 回复 Nahida_nora: Ah, thanks a lot :D. Your solution is much better. I will modify the pseudocode later. Too late for today.
Reinforcement Learning Exercise 4.9
YeXiang\^-^/ 回复 Nahida_nora: Sorry, I don't know why the policy looks like this. If you get the answer, please tell me. There is something wrong in the old code in function "policy_improvement" although it didn't affect the result. I have fixed it now so that the code can be easier to be understood. As the ties is broken between maximum status value and action, the optimal policy is not unique. You can modify the line 67 to select another action, and run the program, you will see a totally different policy. That's why I can not explain the resulting policy looks like this.

大家在看

最新文章

目录

展开全部

收起

评论 2

被折叠的条评论为什么被折叠?

到【灌水乐园】发言

查看更多评论

添加红包

成就一亿技术人!

hope_wisdom

发出的红包

实付元

使用余额支付

点击重新获取

扫码支付

钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。