博观约取，厚积薄发

图1 2个MSA。对应下面depth中的2。图2 SW-MSA 流程1. cyclic shift +reverse cyclic shift图3 cyclic shift， shift_size = 7 // 2 = 3depth = [2, 2, 6, 2] # MSA的个数[SwinTransformerBlock(...,shift_size=0 if (i % 2 == 0) else window_size // 2,...)for i in rang......

图3。(a) Swin Transformer (Swin-T)的架构;(b)两个连续的Swin Transformer Blocks(符号见公式(3))。W-MSA和SW-MSA分别是具有规则（regular）窗口配置和移动（shifted）窗口配置的multi-head self attention modules。源码：GitHub - microsoft/Swin-Transformer: This is an official implementation for "Swin ...