<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[feifeikon的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/feifeikon</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; feifeikon]]></copyright><item><title><![CDATA[RNN/LSTM/GRU/Transformer]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151402088</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151402088</guid><author>feifeikon</author><pubDate>Wed, 10 Sep 2025 14:04:43 +0800</pubDate><description><![CDATA[但是同样会有一些更加复杂的场景。比如我们试着去预测“I grew up in France...I speak fluent French”最后的词“French”。当前的信息建议下一个词可能是一种语言的名字，但是如果我们需要弄清楚是什么语言，我们是需要先前提到的离当前位置很远的“France”的上下文。这说明相关信息和当前预测位置之间的间隔就肯定变得相当的大。不幸的是，在这个间隔不断增大时，RNN会丧失学习到连接如此远的信息的能力。]]></description><category></category></item><item><title><![CDATA[常用优化器及其区别]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151371465</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151371465</guid><author>feifeikon</author><pubDate>Tue, 09 Sep 2025 22:32:26 +0800</pubDate><description><![CDATA[优化器即优化算法是用来求取模型的最优解的，通过比较神经网络自己预测的输出与真实标签的差距，也就是Loss函数。为了找到最小的loss（也就是在神经网络训练的反向传播中，求得局部的最优解），通常采用的是梯度下降(Gradient Descent)的方法，而梯度下降，便是优化算法中的一种。根据损失函数计算得到的误差结果，需要对模型参数（即权重和偏差）进行很小的更改，以期减少预测错误。简单来说，优化器可以优化损失函数，优化器的工作是以使损失函数最小化的方式更改可训练参数，损失函数指导优化器朝正确的方向移动。]]></description><category></category></item><item><title><![CDATA[针对不同任务如何选择损失函数?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151364167</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151364167</guid><author>feifeikon</author><pubDate>Tue, 09 Sep 2025 16:44:14 +0800</pubDate><description><![CDATA[乘一个人工设定的常数权重。]]></description><category></category></item><item><title><![CDATA[手撕LRU（最近最少使用）]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151358797</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151358797</guid><author>feifeikon</author><pubDate>Tue, 09 Sep 2025 13:01:37 +0800</pubDate><description><![CDATA[【代码】手撕LRU（最近最少使用）]]></description><category></category></item><item><title><![CDATA[手撕交叉熵（二分类简化版）]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151333217</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151333217</guid><author>feifeikon</author><pubDate>Mon, 08 Sep 2025 22:27:50 +0800</pubDate><description><![CDATA[【代码】手撕交叉熵（二分类简化版）]]></description><category></category></item><item><title><![CDATA[手撕交叉熵]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151326977</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151326977</guid><author>feifeikon</author><pubDate>Mon, 08 Sep 2025 16:49:59 +0800</pubDate><description><![CDATA[【代码】手撕交叉熵。]]></description><category></category></item><item><title><![CDATA[全网最全 and 最易理解 手撕softmax函数]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151321196</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151321196</guid><author>feifeikon</author><pubDate>Mon, 08 Sep 2025 14:04:33 +0800</pubDate><description><![CDATA[【代码】全网最全 and 最易理解 手撕softmax函数。]]></description><category></category></item><item><title><![CDATA[首创 全网最全+最易理解 手撕多头注意力（MHA）]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/151296489</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/151296489</guid><author>feifeikon</author><pubDate>Mon, 08 Sep 2025 11:34:53 +0800</pubDate><description><![CDATA[【代码】手撕多头注意力（MHA）]]></description><category></category></item><item><title><![CDATA[训练一个7b模型要占用多少显存，不同zero阶段能节省多少显存?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150860630</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150860630</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 16:36:37 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[PPO算法中马尔可夫决策过程起到什么作用?是否是必要的?可以用什么方法替代?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150857468</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150857468</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 15:37:16 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[什么是旋转位置编码?解决了什么问题?为什么?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150855827</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150855827</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 14:43:01 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[计算attention时候为什么要除以根号dk，还有哪些scale处理?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150849682</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150849682</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 11:40:39 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[项目中怎么做的多任务学习，有用到经验回放吗?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150848694</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150848694</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 11:18:38 +0800</pubDate><description><![CDATA[1 前言（Multi-task learning）是和单任务学习（single-task learning）相对的一种机器学习方法。在机器学习领域，标准的算法理论是一次学习一个任务，也就是系统的输出为实数的情况。复杂的学习问题先被分解成理论上独立的子问题，然后分别对每个子问题进行学习，最后通过对子问题学习结果的组合建立复杂问题的数学模型。]]></description><category></category></item><item><title><![CDATA[大模型灾难性遗忘是什么?怎么解决的?]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150847943</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150847943</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 11:01:07 +0800</pubDate><description><![CDATA[3）[7]从另一个角度来约束参数的变化，文中认为参数是一个概率分布，只要在这个分布的核心地带，对于该任务就是可行的，不同的任务对应不同的概率分布，如果能找到两个分布重叠的部分，并将参数约束到这个区域，那么这一参数不就可以对这些任务都有效吗，这类方法被称之为Transfer Techniques法[7]。我想强调的是，深度强化学习披上了强化学习利用机器人与环境交互的数据进行学习的外衣，让我们有种深度学习具备持续学习的能力，不存在灾难性遗忘的问题的错觉。你的每一个生物神经元都是生物组织和化学物质的有机结合。]]></description><category></category></item><item><title><![CDATA[GRPO改进]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150843987</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150843987</guid><author>feifeikon</author><pubDate>Tue, 26 Aug 2025 09:06:16 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[DPO（Direct preference optimization）]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150769913</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150769913</guid><author>feifeikon</author><pubDate>Mon, 25 Aug 2025 16:42:07 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[KL-散度 & 优势函数]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150768684</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150768684</guid><author>feifeikon</author><pubDate>Mon, 25 Aug 2025 14:44:29 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[PPO改进：Group Relative Policy Optimization（GRPO）]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150763874</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150763874</guid><author>feifeikon</author><pubDate>Mon, 25 Aug 2025 11:48:12 +0800</pubDate><description><![CDATA[这是一种类似于PPO的强化学习算法，核心思想是通过限制策略更新的幅度，确保新策略与旧策略的差异不会过大，从而稳定训练）。]]></description><category></category></item><item><title><![CDATA[NLP 场景下的强化学习]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150610274</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150610274</guid><author>feifeikon</author><pubDate>Fri, 22 Aug 2025 17:28:27 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[RLHF的定义]]></title><link>https://blog.youkuaiyun.com/feifeikon/article/details/150610061</link><guid>https://blog.youkuaiyun.com/feifeikon/article/details/150610061</guid><author>feifeikon</author><pubDate>Fri, 22 Aug 2025 14:34:43 +0800</pubDate><description><![CDATA[如下图所示，利用人类偏好来拟合奖励函数，同时利用RL算法优化当前预测的奖励函数。在某些领域中，人类更擅长比较agent的性能而不是给出绝对的评分，这种比较也能学习到人类偏好。RLHF的定义首先由《Deep Reinforcement Learning from Human Preferences》提出，解决了强化学习中奖励函数怎么设计的问题。：依赖于人类设计的奖励函数，这在实际应用中往往非常困难且不够灵活，因为面临涉及复杂、定义不明确或难以指定的目标的任务时，很难定义奖励函数。用户给出反馈的成本不高。]]></description><category></category></item></channel></rss>