Jay_Tang的博客

Logistic Regression, L1, L2 regularization, Gradient/Coordinate descent 详细MLE v.s. MAP ~ L1, L2 Math Derivation 详细XGBoost math Derivation 通俗易懂的详细推导Introduction to Convex Optimization Basic Concept...

文章目录往期文章链接目录Information Extraction v.s. Relation ExtractionExisting Works of REPattern-based MethodsStatistical Relation Extraction ModelsNeural Relation Extraction MethodsFuture DirectionsUtilizing More DataMethods to Denoise DS DataOpen Problem for Utili

文章目录往期文章目录链接Relation Extraction (RE)document-level REIntuitionContributionOverview of Proposed ModelProposed ModelSentence Encoding LayerGraph construction LayerNode ConstructionEdge ConstructionInference LayerFirst StepSecond StepClassification LayerResul

往期文章链接目录文章目录往期文章链接目录Cross-lingual learningCross-lingual resourcesMultilingual distributional representationsEvaluation of multilingual distributional representationsParallel corpusWord AlignmentsMachine TranslationUniversal features (out of fashion)Biling

往期文章链接目录文章目录往期文章链接目录BERT RecapOverviewBERT SpecificsThere are two steps to the BERT framework: pre-training and fine-tuningInput Output RepresentationsTasksresultsAblation studiesEffect of Pre-training TasksEffect of Model SizesReplication study of BERT p

文章目录往期文章链接目录Before we startSurface-Level Patterns in AttentionProbing Individual Attention HeadsProbing Attention Head CombinationsClustering Attention Heads往期文章链接目录往期文章链接目录Before we startIn this post, I mainly focus on the conclusions the authors reach

文章目录往期文章链接目录Ways of tokenizationWord-based tokenizationCharacter-based tokenizationSubword tokenizationExisting approaches for cross-lingual NLPOut-of-vocabulary (OOV) problem in mono/multi-lingual settingsM-BERT (Multi-lingual BERT)WHY MULTILINGUAL BERT W

文章目录往期文章链接目录Sequence DataWhy not use a standard neural network for sequence tasksRNNDifferent types of RNNsLoss function of RNNBackpropagation through timeVanishing gradients with RNNsAdvantages and Drawbacks of RNNLSTMTypes of gatesformulas and illustrati

文章目录往期文章链接目录Log-Linear modelConditional Random Fields (CRF)Formal definition of CRFLog-linear model to linear-CRFInference problem for CRFLearning problem for CRFLearning problem for general Log-Linear modelLearning problem for CRFCompute Z(xˉ,w)Z(\bar x,

往期文章链接目录Before reading this post, you should be familiar with the EM Algorithm and decent among of knowledge of convex optimization. If not, check out my previous postEM Algorithmconvex optimiz...

往期文章链接目录Probabilistic Graphical Model (PGM)Definition: A probabilistic graphical model is a probabilistic model for which a graph expresses the conditional dependence structure between random variables.In general, PGM obeys following rules:Sum Rul

往期文章链接目录Comparison between CBOW and Skip-gramThe major difference is that skip-gram is better for infrequent words than CBOW in word2vec. For simplicity, suppose there is a sentence “w1w2w3w4w_1w_2...

往期文章链接汇总Overview of various word representation and Embedding methodsLocal Representation v.s. Distributed RepresentationOne-hot encoding is local representation and is good for local generalizati...

NLP = NLU + NLGNLU: Natural Language UnderstandingNLG: Natural Language GenerationNLG may be viewed as the opposite of NLU: whereas in NLU, the system needs to disambiguate the input sentence to ...