POLICY GENERALIZATION IN CAPACITY-LIMITED REINFORCEMENT LEARNING

最新推荐文章于 2024-02-26 14:27:51 发布

Adam婷

最新推荐文章于 2024-02-26 14:27:51 发布

阅读量602

点赞数

分类专栏：机器学习算法论文研读深度强化学习 ICLR 强化学习

本文链接：https://blog.youkuaiyun.com/weixin_41697507/article/details/94306580

版权

机器学习同时被 3 个专栏收录

161 篇文章 ¥19.90 ¥99.00

订阅专栏

超级会员免费看

强化学习

26 篇文章 ¥19.90 ¥99.00

订阅专栏

超级会员免费看

算法

161 篇文章

订阅专栏

本文研究了在信息理论约束下强化学习（RL）中的策略泛化。通过将RL与速率失真（RD）理论相结合，开发了一种名为Capacity-Limited Actor-Critic（CL-AC）的在线RL算法，该算法在优化效用和信息速率之间找到平衡。实验证明，CL-AC在高信息速率下比标准表格Actor-Critic算法学习更快，策略更优；同时，CL-AC在容量有限的策略表示下能更好地泛化到新环境，避免过度拟合。该工作为理解RL中的泛化提供了理论基础，并为计算合理的RL代理开发提供了新视角。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

能力有限的强化学习中的策略一般化

ABSTRACT

Motivated by the study of generalization in biological intelligence, we examine reinforcement learning (RL) in settings where there are information-theoretic constraints placed on the learner’s ability to represent a behavioral policy. We first show that the problem of optimizing expected utility within capacity-limited learning agents maps naturally to the mathematical field of rate-distortion (RD) theory. Applying the RD framework to the RL setting, we develop a new online RL algorithm, Capacity-Limited Actor-Critic, that learns a policy that optimizes a tradeoff between utility maximization and information processing costs. Using this algorithm in a 2D gridworld environment, we demonstrate two novel e