Jay_Tang的博客

Logistic Regression, L1, L2 regularization, Gradient/Coordinate descent 详细MLE v.s. MAP ~ L1, L2 Math Derivation 详细XGBoost math Derivation 通俗易懂的详细推导Introduction to Convex Optimization Basic Concept...

文章目录IntroductionRecent DirectionsSampling StrategiesMethods of Learning ClassifiersClassifier Re-training (cRT)Nearest Class Mean classifier (NCM)τ\tauτ-normalized classifier (τ(\tau(τ-normalized)ExperimentsDatasetsEvaluation ProtocolResultsSampling matter

文章目录往期文章链接目录Shortcoming of normal neural networksGeneralization of InformationKnowledge DistillationA few DefinitionsGeneral idea of knowledge distillationTeacher and StudentTemperature & EntropyTraining the Distil Model往期文章链接目录往期文章链接目录Currently, esp

文章目录Deep Neural Network往期文章链接目录Forward PropagationLoss functions of neural networkBack-propagationcompute ∂ℓ∂f(x)\frac{\partial \ell}{\partial f(x)}∂f(x)∂ℓ​compute ∂ℓ∂a(L+1)(x)\frac{\partial \ell}{\partial a^{(L+1)}(x)}∂a(L+1)(x)∂ℓ​compute ∂ℓ∂h(k)(x)\frac

文章目录往期文章链接目录Log-Linear modelConditional Random Fields (CRF)Formal definition of CRFLog-linear model to linear-CRFInference problem for CRFLearning problem for CRFLearning problem for general Log-Linear modelLearning problem for CRFCompute Z(xˉ,w)Z(\bar x,

往期文章链接目录Before reading this post, you should be familiar with the EM Algorithm and decent among of knowledge of convex optimization. If not, check out my previous postEM Algorithmconvex optimiz...

往期文章链接目录Probabilistic Graphical Model (PGM)Definition: A probabilistic graphical model is a probabilistic model for which a graph expresses the conditional dependence structure between random variables.In general, PGM obeys following rules:Sum Rul

往期文章链接目录文章目录往期文章链接目录Gaussian mixture model (GMM)Interpretation from geometryInterpretation from mixture modelGMM Derivationset upSolve by MLESolve by EM AlgorithmK-means往期文章链接目录Gaussian mixture model (GMM)A Gaussian mixture model is a probabilistic mode

往期文章链接目录Jensen’s inequalityTheorem: Let fff be a convex function, and let XXX be a random variable. Then:E[f(X)]≥f(E[X])E[f(X)] \geq f(E[X])E[f(X)]≥f(E[X])\quad Moreover, if fff is strictly con...

Linear SVMIdea:We want to find a hyper-plane w⊤x+b=0w^\top x + b = 0w⊤x+b=0 that maximizes the margin.Set up:We first show that the vector www is orthogonal to this hyper-plane. Let x1x_1x1​, x2x...

Consider an optimization problem in the standard form (we call this a primal problem):We denote the optimal value of this as p⋆p^\starp⋆. We don’t assume the problem is convex.The Lagrange dual fun...

Optimization problemAll optimization problems can be written as:Optimization Categoriesconvex v.s. non-convexDeep Neural Network is non-convexcontinuous v.s.discreteMost are continuous vari...

Bagging v.s. Boosting:Bagging:Leverages unstable base learners that are weak because of overfitting.Boosting:Leverages stable base learners that are weak because of underfitting.XGBoostLearning ...

MLE v.s. MAPMLE: learn parameters from data.MAP: add a prior (experience) into the model; more reliable if data is limited. As we have more and more data, the prior becomes less useful.As data inc...

Generative model v.s. Discriminative model：Examples:Generative model: Naive Bayes, HMM, VAE, GAN.Discriminative model：Logistic Regression, CRF.Obejective function:Generative model: max  p (x,y...