PKU_ZZY的博客

2020.1.13 noteAdderNet: Do We Really Need Multiplications in Deep Learning?Compared with cheap addition operation, multiplication operation is of much higher computation complexity. The widely-used ...

2019.12.17 noteFilter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural NetworksBatch Normalization (BN) is a highly successful and widely used batch dependen...

2019.12.13 noteFast Transformer Decoding: One Write-Head is All You NeedThey have proposed an alternative to multi-head attention - multi-query attention, with much lower memory bandwidth requir...

2019.11.15 note (2)Intriguing properties of neural networksDeepFool: a simple and accurate method to fool deep neural networksTowards Evaluating the Robustness of Neural NetworksDISTRIBUTIO...

2019.11.15 note (1)EXPLAINING AND HARNESSING ADVERSARIAL EXAMPLESFGSM (Fast Gradient Sign Method)Adversarial examples in the physical worldAnother version:Towards Deep Learning Models Resi...

2019.11.5 noteDeepGCNs: Making GCNs Go as Deep as CNNsGraph Convolutional Networks (GCNs) offer an alternative that allows for non-Eucledian data as input to a neural network similar to CNNs. While ...

paper/2019.11.5 noteExploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerThey propose a framework to cast every task they consider as a unified task: feeding the model t...

2019.10.18 noteQuaternion Knowledge Graph EmbeddingsIn this work, authors move beyond the traditional complex-valued representations, introducing more expressive hypercomplex representations to mode...

2019.9.6 noteMeta-Learning with Implicit GradientsSimilar to paper darts, this work formulates Meta-learning as a two-level (inner/outer) optimization problem. To make the meta-learning process mode...

2019.10.12 noteExploiting Explicit Paths for Multi-hop Reading ComprehensionThe task and proposed method for 2 hops:(a) Find a passage p1p_1p1​ that contains a head entity heh_ehe​ from the questi...

2019.10.8 noteCommonsense Knowledge Base CompletionThis work proposes a task of complete commonsense knowledge base (generating new reasonable triplets in the knowledge base). This work also propose...

2019.9.5 noteA Structural Probe for Finding Syntax in Word RepresentationsThe probe identifies a linear transformation under which squared L2 distance encodes the distance between words in the pars...

paper/2019.9.4 notepaper/2019.9.4 noteA Simple Theoretical Model of Importance for SummarizationLogicENN: A Neural Based Knowledge Graphs Embedding Model with Logical RulesNorm-PreservationSqueeze-and...