<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[trb331617的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/trb331617</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; trb331617]]></copyright><item><title><![CDATA[windows笔记本双系统ubuntu设置]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/165322996</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/165322996</guid><author>trb331617</author><pubDate>Mon, 14 Sep 2026 15:45:38 +0800</pubDate><description><![CDATA[ubuntu 一直运行、不休眠（即使合上盖子也不休眠）]]></description><category></category></item><item><title><![CDATA[[学习笔记]Transformer技术纵深]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/162245821</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/162245821</guid><author>trb331617</author><pubDate>Tue, 23 Jun 2026 21:35:59 +0800</pubDate><description><![CDATA[除主要模块外，Transformer 模型还应用了 LayerNorm (Layer Normalization/LN， 层归一化),Dropout 和残差连接(Residual Connection)等设计方法，这些辅助模块对提升模型性能至关重要。LayerNorm有助于稳定训练并提高收敛性，其通过归一化输入特征来确保激活的均值和方差一致，从而缓解内部协变量偏移问题，使模型学习更高效，降低对初始权重的敏感性。这促使模型学习更强大的特征，减少对特定神经元的依赖，从而更好地泛化到新数据。]]></description><category></category></item><item><title><![CDATA[torch.autograd.Function.apply()]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/161896862</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/161896862</guid><author>trb331617</author><pubDate>Thu, 11 Jun 2026 16:15:50 +0800</pubDate><description><![CDATA[apply()是 PyTorch C++ 层的入口。它会创建一个节点（也就是 autograd 图上的反向节点），然后在该节点的上下文ctx下调用你定义的forward，并把输出张量的grad_fn指向这个节点。apply= "建反向节点 → 在该节点 ctx 下以 no_grad 调用 forward → 把输出的 grad_fn 指向该节点"。所以 forward 是被它调用的，但调用之外它还干了一堆把张量挂进 autograd 图的事，从而自动注册反向图。]]></description><category></category></item><item><title><![CDATA[OPD Reverse KL]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/161800317</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/161800317</guid><author>trb331617</author><pubDate>Mon, 08 Jun 2026 15:07:02 +0800</pubDate><description><![CDATA[在线策略蒸馏（On-Policy Distillation, 简称逆向 KL 散度（Reverse KL, RKL）的公式为。与前向 KL（Forward KL,）相比，逆向 KL 具有 模式寻求（Mode-seeking）的特性。它会让学生模型倾向于只在教师模型概率较高的地方生成文本，从而减少模型产生幻觉（Hallucination）或语无伦次的概率，使生成的文本更加确定和精准。]]></description><category></category></item><item><title><![CDATA[loss.backward() 和 梯度累积]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/161664012</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/161664012</guid><author>trb331617</author><pubDate>Wed, 03 Jun 2026 21:25:18 +0800</pubDate><description><![CDATA[核心思想：把一个"大 batch"拆成 N 个"小 batch（micro_batch）"，连续做 N 次前向 + 反向但不更新参数，让梯度在p.grad上自然累加；累加 N 次后再做一次。loss = loss / self.accumulated_gradient # ← 先除以 Nloss.backward() # ← 梯度累加到 p.gradif self.step == 0: # ← 累满 N 次才真正更新...累积 N 次 backward → 1 次 optimizer.step。]]></description><category></category></item><item><title><![CDATA[读取wandb本地缓存文件，并绘制曲线图]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/161521771</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/161521771</guid><author>trb331617</author><pubDate>Fri, 29 May 2026 17:59:59 +0800</pubDate><description><![CDATA[【代码】读取wandb本地缓存文件，并绘制曲线图。]]></description><category></category></item><item><title><![CDATA[lm_head]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/161457795</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/161457795</guid><author>trb331617</author><pubDate>Wed, 27 May 2026 16:33:53 +0800</pubDate><description><![CDATA[【代码】lm_head。]]></description><category></category></item><item><title><![CDATA[【学习笔记】大模型]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/156132408</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/156132408</guid><author>trb331617</author><pubDate>Sun, 21 Dec 2025 19:15:23 +0800</pubDate><description><![CDATA[书，全解深度学习]]></description><category></category></item><item><title><![CDATA[vllm 0.9.2笔记]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/155096558</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/155096558</guid><author>trb331617</author><pubDate>Fri, 21 Nov 2025 10:42:07 +0800</pubDate><description><![CDATA[【代码】vllm笔记。]]></description><category></category></item><item><title><![CDATA[深度强化学习]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/154952532</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/154952532</guid><author>trb331617</author><pubDate>Mon, 17 Nov 2025 17:19:20 +0800</pubDate><description><![CDATA[]]></description><category></category></item><item><title><![CDATA[【学习笔记】强化学习从原理到实践]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/154187468</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/154187468</guid><author>trb331617</author><pubDate>Fri, 31 Oct 2025 15:12:20 +0800</pubDate><description><![CDATA[强化学习从原理到实践，李福林]]></description><category></category></item><item><title><![CDATA[Golang交叉编译到Android上运行]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/153965483</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/153965483</guid><author>trb331617</author><pubDate>Mon, 27 Oct 2025 11:39:59 +0800</pubDate><description><![CDATA[使用adb传输到android手机上运行。android手机上运行结果。go程序：hello.go。查看go编译支持的目标平台。]]></description><category></category></item><item><title><![CDATA[【学习笔记】图解大模型]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/153962995</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/153962995</guid><author>trb331617</author><pubDate>Mon, 27 Oct 2025 11:03:37 +0800</pubDate><description><![CDATA[词嵌入使我们能够衡量两个词的语义相似度。使用各种距离度量方法， 可以判断一个词与另一个词的接近程度，含义相似的词往往会更接近。词嵌入可以用多种属性来表示一个词的含义。这些属性组合在一起，是将人类语言转换为计算机语言行之有效的方式。可以为不同类型的输入创建嵌入，如词嵌入和句子嵌入，它们用于表示不同层次的抽象 (词与句子)。自回归（auto-regressive），在生成下一个词时，需要使用先前生成的词作为输入。神经网络由处理信息的多层互连节点组成。每个连接都有一定的权重，这些权重通常被称为模型的参数。]]></description><category></category></item><item><title><![CDATA[【学习笔记】大模型]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/153828738</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/153828738</guid><author>trb331617</author><pubDate>Fri, 24 Oct 2025 15:03:24 +0800</pubDate><description><![CDATA[多头注意力（MHA）、多查询注意力(MQA)、分组查询注意力(GQA)、多头潜在注意力(MLA)激活函数（例如 SwiGLU），以提高模型的非线性拟合能力，捕捉更复杂的模式。缓解深层网络中的梯度消失问题，提升模型训练效率。建模序列中不同词元之间的语义关系。提升训练稳定性并加速模型收敛。提升模型的容量和计算效率。进一步提取特征和语义。]]></description><category></category></item><item><title><![CDATA[【学习笔记】从零构建大模型]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/152664376</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/152664376</guid><author>trb331617</author><pubDate>Tue, 07 Oct 2025 16:43:44 +0800</pubDate><description><![CDATA[【学习笔记】从零构建大模型]]></description><category></category></item><item><title><![CDATA[python使用transformer库推理]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/151224143</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/151224143</guid><author>trb331617</author><pubDate>Fri, 05 Sep 2025 15:23:40 +0800</pubDate><description><![CDATA[【代码】python使用transformer库推理。]]></description><category></category></item><item><title><![CDATA[qemu/kvm学习笔记]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/132686093</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/132686093</guid><author>trb331617</author><pubDate>Tue, 05 Sep 2023 10:21:14 +0800</pubDate><description><![CDATA[qemu/kvm架构。]]></description><category></category></item><item><title><![CDATA[overlayfs]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/132611954</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/132611954</guid><author>trb331617</author><pubDate>Thu, 31 Aug 2023 21:29:53 +0800</pubDate><description><![CDATA[How containers work: overlayfs]]></description><category></category></item><item><title><![CDATA[linux /proc/{$pid}/status /proc/meminfo]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/131500902</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/131500902</guid><author>trb331617</author><pubDate>Sun, 02 Jul 2023 14:50:41 +0800</pubDate><description><![CDATA[/proc/pid/status & /proc/meminfo]]></description><category></category></item><item><title><![CDATA[shell学习笔记]]></title><link>https://blog.youkuaiyun.com/trb331617/article/details/131472075</link><guid>https://blog.youkuaiyun.com/trb331617/article/details/131472075</guid><author>trb331617</author><pubDate>Fri, 30 Jun 2023 11:29:23 +0800</pubDate><description><![CDATA[man test。]]></description><category></category></item></channel></rss>