Discrimitive model and Generative Model

以前对这两个概念非常模糊，今天从网上找到了关于这两个概念的两种风格的解释，比较一下！

第一种风格：

For a data sample: x and it class lable:,y,

 要检测图像中的淋巴结，有很多positive，很多的negetive。

 Discrimitive model: p(y|x), 给定x,算y。

 Generative model:和 discrimitive model 的区别：

Generative model focus 在自己的inclass 本身，不care 到底 decision boundary 在哪。 Generative model 实际上带的information 要比discrimitive model rich, 因为假设有generative model， 两类的，就完全得到了p(x|y)，而discrimitive model 只care decision boundary。这里说的generative model 和 discrimitive model，在行业里，这个说法是通用的。 由Generative model 可以得到 discrimitive model, 但由discrimitive model 得不到 generative model。因为需要用到P(x), 如果只是label 的话，p(y)很简单，但为什么不能直接用Generative model 呢？ 优缺 Discrimitive model: 相当于在图像上scan 一下，detection, 用一个path, 在不同的scale 上search, 每一词看probability, 在SVM上是positive 还是negetive。

 Discrimitive 比较easy to learn， 给出正负例，给出lable, focus on discrimitive model marginal distribution。 某种意义上，比generativemodel 要简单，但power 是 limited, 可以告诉你的时1还是2，但没有办法把整个场景描述出来。 P(x|y) P(y) 当一个分类，没有negetive，研究single class，比discrimitive model flex 多，learning 和 computing 要比 discrimitive model 复杂得多

 第二种解释 Discriminative Model是判别模型，又可以称为条件模型，或条件概率模型。 Generative Model是生成模型，又叫产生式模型。

二者的本质区别是 discriminative model 估计的是条件概率分布(conditional distribution)p(class|context) generative model 估计的是联合概率分布（joint probability distribution）p() 常见的Generative Model主要有：

 – Gaussians, Naive Bayes, Mixtures of multinomials – Mixtures of Gaussians, Mixtures of experts, HMMs – Sigmoidal belief networks, Bayesian networks – Markov random fields 常见的Discriminative Model主要有：

– logistic regression

– SVMs

– traditional neural networks

– Nearest neighbor Successes of Generative Methods： NLP

 – Traditional rule-based or Boolean logic systems Dialog and Lexis-Nexis) are giving way to statistical approaches (Markov models and stochastic context grammars) Medical Diagnosis

 – QMR knowledge base, initially a heuristic expert systems for reasoning about diseases and symptoms been augmented with decision theoretic formulation Genomics and Bioinformatics

– Sequences represented as generative HMMs

主要应用Discriminative Model： Image and document classification Biosequence analysis Time series prediction Discriminative Model缺点： Lack elegance of generative

– Priors, structure, uncertainty Alternative notions of penalty functions, regularization, kernel functions Feel like black-boxes – Relationships between variables are not explicit and visualizable Bridging Generative and Discriminative： Can performance of SVMs be combined elegantly with flexible Bayesian statistics? Maximum Entropy Discrimination marries both methods – Solve over a distribution of parameters (a distribution over solutions)