<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[yyyang_的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/yyyang_</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; yyyang_]]></copyright><item><title><![CDATA[Swin Transformer, SwinIR, SwinFIR]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/127746712</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/127746712</guid><author>yyyang_</author><pubDate>Tue, 08 Nov 2022 11:02:19 +0800</pubDate><description><![CDATA[痛点：尺度变化 – 提出Hieracical.
attention的变化： sliding window attention，减少了attention的计算复杂度，同时通过sliding增强connection，实现全局attention
和ViT的区别：ViT16*下采样，尺寸单一；Swin Transformer多尺度]]></description><category></category></item><item><title><![CDATA[Transformer, DETR, ViT, SwinIR]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/127746507</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/127746507</guid><author>yyyang_</author><pubDate>Tue, 08 Nov 2022 10:55:53 +0800</pubDate><description><![CDATA[Transformer实现的意义和RNN是一样的，将时序信息传递给输入进行计算，实现时序信息的传递。RNN中是前一时刻传递给后一时刻，Transformer中是通过attention在全局的序列中提取信息。]]></description><category></category></item><item><title><![CDATA[【经典论文回顾】ResNet: Deep Residual Learning for IR]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/127746335</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/127746335</guid><author>yyyang_</author><pubDate>Tue, 08 Nov 2022 10:49:43 +0800</pubDate><description><![CDATA[ResNet目的：显示表示Identity Mapping, 有效减少了更深的网络的FLOP。]]></description><category></category></item><item><title><![CDATA[从RGB到多通道图像的迁移学习]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/124640323</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/124640323</guid><author>yyyang_</author><pubDate>Sun, 08 May 2022 02:30:00 +0800</pubDate><description><![CDATA[基于Segmentation Models PyTorch (smp) 实现]]></description><category></category></item><item><title><![CDATA[Data Augmentation: 基于albumentations库的图像数据增强]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/124511619</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/124511619</guid><author>yyyang_</author><pubDate>Sat, 30 Apr 2022 13:19:18 +0800</pubDate><description><![CDATA[​	Albumentations 是一个用于图像增强的 Python 库。图像增强用于深度学习和计算机视觉任务，以提高训练模型的质量。图像增强的目的是从现有数据中创建新的训练样本。
]]></description><category></category></item><item><title><![CDATA[【機器學習2021】9 Generative Adversarial Network (GAN)]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/124431203</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/124431203</guid><author>yyyang_</author><pubDate>Tue, 26 Apr 2022 16:36:57 +0800</pubDate><description><![CDATA[【機器學習2021】生成式對抗網路 (Generative Adversarial Network, GAN) (一) – 基本概念介紹
Generation的理由：why distribution?

训练资料中相同的input，可能需要对应不同的output
Generative Adversarial Network (GAN)
Basic Idea of GAN
Generator / Discriminatior
GNN Algorithm ( in each iteration, do step]]></description><category></category></item><item><title><![CDATA[skip-gram推荐模型笔记]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/124268402</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/124268402</guid><author>yyyang_</author><pubDate>Tue, 19 Apr 2022 11:03:03 +0800</pubDate><description><![CDATA[item-item推荐算法模型分析。用到了word2vec, CF, random walk, skip-gram, embedding]]></description><category></category></item><item><title><![CDATA[【機器學習2021】8.3 Transformer Training]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/124085341</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/124085341</guid><author>yyyang_</author><pubDate>Sun, 10 Apr 2022 20:43:10 +0800</pubDate><description><![CDATA[【機器學習2021】Transformer (下)
8.1/8.2是train好的模型的运作方式，接下来关注如何进行training和testing
Teacher Forcing: use the GT as input

训练目标：minimize cross entropy（见4.）
Traning Tips (train seq2seq model)
1. Copy Mechanism

2. Guide Attention
客制化attention，通过对任务的理解，强制要求attention的顺]]></description><category></category></item><item><title><![CDATA[【機器學習2021】8.2 Transformer Decoder]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/124085250</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/124085250</guid><author>yyyang_</author><pubDate>Sun, 10 Apr 2022 20:39:27 +0800</pubDate><description><![CDATA[【機器學習2021】Transformer (下)
以Autoregressive Decoder - Speech Recognition为例

Error Propagation: 一步错步步错
产生Exposure bias：解决方法 --scheduled sampling
https://medium.com/analytics-vidhya/seq2seq-model-and-the-exposure-bias-problem-962bb5607097
​	原始论文结构图：

总体Input: ]]></description><category></category></item><item><title><![CDATA[【機器學習2021】8.1 Transformer Encoder]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833993</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833993</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:18:51 +0800</pubDate><description><![CDATA[【機器學習2021】Transformer（上）
Seq2seq的问题定义…
Encoder
​	有很多个block，每个block是好几个layer

​	block的图解结构，对应下面的各个步骤：


取self-attention


把self-attention得到的向量加上原向量
（residual connection思路，一种常用的网络架构）


取norm，这里是Layer Norm，类似batch norm的另一种norm。

​		Batch Norm是对于不同example的同一维度]]></description><category></category></item><item><title><![CDATA[【機器學習2021】7.5 Positional Encoding]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833934</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833934</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:15:24 +0800</pubDate><description><![CDATA[Transformer的input没有位置概念，需要
Positional Encoding / Positional Embedding

其实就是在aia^iai加上一个eie^iei，这个eie^iei是根据某种embedding算法求出来的值，想让网络learn出位置信息
对比不同的positional embedding算法的论文：
g算法的论文：


...]]></description><category></category></item><item><title><![CDATA[【機器學習2021】7 Self-attention]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833807</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833807</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:13:23 +0800</pubDate><description><![CDATA[【機器學習2021】自注意力機制 (Self-attention)
Transformer: Attention is all you need
Self-attention input可以是hidden layer的output
1. Find the relevant vector in a sequence
计算相关度(relevant) α\alphaα的方法:  Dot-product &amp; Additive

每一个 α\alphaα 乘上不同的矩阵得到 q,k,vq, k, vq,k,v]]></description><category></category></item><item><title><![CDATA[【機器學習2021】6 Convolutional Neural Networks]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833781</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833781</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:09:14 +0800</pubDate><description><![CDATA[卷積神經網路 (Convolutional Neural Networks, CNN)
Model Bias:
https://stackoverflow.com/questions/2480650/what-is-the-role-of-the-bias-in-neural-networks
I think that biases are almost always helpful. In effect, a bias value allows you to shift the activation fu]]></description><category></category></item><item><title><![CDATA[【機器學習2021】5 Batch Normalization]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833734</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833734</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:07:35 +0800</pubDate><description><![CDATA[類神經網路訓練不起來怎麼辦 (五)： 批次標準化 (Batch Normalization) 簡介
做Feature Normalization:
为了更好的train， error surface更平滑， train更快（可能也更好）
实做中，Feature Normalization在Activation Function(e.g. Sigmoid)前或后，效果差异不大

网络中经过normalization之后，可能会再乘回 γ和β \gamma 和 \betaγ和β ,让特征回到原来的大小比例
St]]></description><category></category></item><item><title><![CDATA[【機器學習2021】4 Classification Loss]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833688</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833688</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:06:26 +0800</pubDate><description><![CDATA[類神經網路訓練不起來怎麼辦 (四)：
損失函數 (Loss) 也可能有影響
Mean Square Error &amp; Cross entropy

分类任务时：softmax和cross entropy常绑定使用

minimize cross entropy == maimum likelihood

]]></description><category></category></item><item><title><![CDATA[【機器學習2021】3.5 Framework of ML]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833631</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833631</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:04:41 +0800</pubDate><description><![CDATA[機器學習任務攻略
Framework of Machine Learning

Step1: Function with unknown
Step2: define loss function from training data
Step3: optimization
General Guide

Loss很高的话：
1 模型简单了 加输入的特征数、加层数…

2 optimizer不够好 比如gradient decent优化效果不行
判断是1还是2：Model Bias v.s. Optimizati]]></description><category></category></item><item><title><![CDATA[【機器學習2021】1-3 Local Minima]]></title><link>https://blog.youkuaiyun.com/yyyang_/article/details/123833484</link><guid>https://blog.youkuaiyun.com/yyyang_/article/details/123833484</guid><author>yyyang_</author><pubDate>Tue, 29 Mar 2022 22:01:46 +0800</pubDate><description><![CDATA[類神經網路訓練不起來怎麼辦 (一)： 局部最小值 (local minima) 與鞍點 (saddle point)]]></description><category></category></item></channel></rss>