feitianlzk的博客

源自Sebastian Ruder的博文The secret ingredients of word2vecQuestionsQ1. Are embeddings superior to distributional methods?With the right hyperparameters, no approach has a consistent advantage over

momentum: μμ\mu,上一次更新的权重(保证更新方向不会和上次偏离过多) learning rate: αα\alpha,当前梯度的权重 weight decay: λλ\lambda 相当于L2 惩罚 refs: caffe,pytorch,stackoverflow Vt+1=μVt−α∇L(Wt)Vt+1=μVt−α∇L(Wt)V_{t+1} = \mu V_t - \a...

Tensor & VariableWhat is the difference between Tensors and Variables in Pytorch?torch tensors are actually the data.variables wrap tensors, and construct a chain of operations between the t...

A replacement for NumPy to use the power of GPUsa deep learning research platform that provides maximum flexibility and speed1. Basic1.1 tensorsx = torch.empty(5, 3) # unitializedx = torch...

Notes: 谷歌deeplabv3+的代码现在已经开源，详见deeplab(Github)，还有一个使用的demo样例。0.spatial pyramid poolingprobing the incoming features with filters or pooling operations at multiple rates and multiple effecti...

Transductive Unbiased Embedding for Zero-Shot LearningSummaryPROubias term: 在Loss添加一个针对未知类的loss, 部分抑制了zero shot天生倾向于带label数据的问题巧妙的数据利用，虽然target dataset没有用label(图片文字对应关系)，但是用了label的文字embedd...

Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach阅读笔记x. QuestionsAE步骤, 怎么判断收敛不好。万一有的图片需要erase 3次，有的erase2两次会怎么样? 实验直接选了3次 PSL，为什么conv7的结...

弱监督语义分割what: segment image only use image-level labelchallenge: how to build relation between image labels and pixels 参考1. Features针对输入图片的salience map gradient based针对最后一层的class activa...

Visualizing and Understandingwhat’s going on in CNNFirst layer:weights, Filter, visual layers because when input similar to weights, result will be maximizedhigher layer filters: meaning less...

Segmentation, Localization, DetectionSemantic Segmentationlabel each pixel in the image with a category labelknow classesidea: sliding window inefficient, Not reusing shared features between...

AlexNetCONV - MAXPOOL - NORM(not common) CONV - MAXPOOL - NORM CONV - CONV - CONV - MAXPOOL FC - FC - FC local response normalizationVGGdeeper (16-19layers)small filter (3x3 CONV stride...

CNN 笔记Convolution LayerPooling LayerFully Connected Layer(FC layer)Useful NotesPreprocessingWeigth InitializationRegularizationLossclassificationAttribute classificationregression...

原文A 2017 Guide to Semantic Segmentation with Deep Learning0. Intro1. Problem1.1 before deep1.2 current1.3 postprocessing2. Models0. Intromainly use natural/real world im...

Generative ModelsPixelCNN/RNNVAEGAN PixelRNN/CNN生成速度慢pro Can explicitly compute likelihood p(x)Explicit likelihood of training data gives good evaluation metricGood samplescon Sequ...

优先Few-shot Autoregressive Density Estimation: Towards Learning to Learn DistributionsMAMLSummary MAML: meta-learn an initial conditionLSTM optimization: meta-learn a good initial cond...

字体数据集 imagenet几个应用few shot classification (image) + RL + 推荐(cold start)

Conclusionparametric vs metric karpathy很好的paper notesAdvantages使用加权平均(metric)方法去做few-shot,优点: 速度快，需要训练量小将对S的训练加入模型 pθ(y|x,S)pθ(y|x,S)p_\theta(y|x,S)，具体做法是:分成task(或称为训练集)去训练, 每次选择一些类别L(label s...

softmax P(y=k)=exp(wkT⋅x)∑kexp(wkT⋅x)" role="presentation">P(y=k)=exp(wTk⋅x)∑kexp(wTk⋅x)P(y=k)=exp(wkT⋅x)∑kexp(wkT⋅x)P(y=k) = \frac{exp(\mathbf w_k^T \cdot \mathbf x)}{\sum_{k}e

FCN frameworkGood explanationglobal receptive fields: conv(no linearities) + downsamplespatial resolution lossencoder: dilated conv pro: expand receptive fieldcon: isolate pixels from cont...