20180713DLday1课程笔记

Introduction

---

0

• Jian Tang
• tangjianpku@gmail.com

1

• History 1950-1970 logic rules; 1980-1990 knowledge acquisition; 2010

-. machine learning

• $Deep Learning \subset Machine Learning \subset Artificial Intelligence$
• machine learning
  
  • use statistical techniques, “learn” with data
  • extract features automatically, instead of by domain experts
  • learn automatically, instead of explicit programming
• Big Data-Big Computation-Big Model : Why deep learning now
• usage
  
  • …

2Probability

• Bayes’ Theorem

  • $p(Y|X) = \frac{p(X|Y)p(Y)}{p(X)}, p(X) = \sum \limits_Y p(X|Y)p(Y)$
  • posterior $\propto$ likelihood * prior

• variables

  • E[f] := the average value of f(X) under the distribution p(x)
  • $E[f] = \sum \limits_x p(x)f(x)$
  • V[f], cov[x, y]

• distributions

  • binomial distribution
  • $Bin(m|N, \mu) = \binom{N}{m} \mu^m(1-\mu)^{N-m}$
  • $E[m] = N\mu, var[m] = N\mu(1-\mu)$

• multinomial variables

  • x可以取k种值，$x = (0, 0, 1, 0, 0, 0)^T$表示x取了六种中的第三种

  $\mu = (\mu_1, \mu_2, ..., \mu_k)^T$，对应x向量每个位置上为1的概率

  从而某个特定的x出现的概率 $p(x|\mu) = \prod \limits_{k = 1}^K \mu_k^{x_k}$ (也就是$\mu_k$)

  • $E[{x}|mu] = \sum \limits_x p(x|\mu)x = (\mu_1, \mu_2, ..., \mu_k)^T = \mu$

  • maximum likelihood estimation

  $\mu_k = \frac{m_k}{N}, m_k = \sum \limits_N x_{nk}观察值的矩阵的每列和$

• gaussian univariate distribution正态分布

  • multivariate gaussian distribution
  • maximum likelihood estimation
  • mixture of gaussians-可以模拟其他各种分布

• gradient descent梯度下降

  • a way to minimize an object function $J(\theta)$
  • $\eta$: learning rate, which determines the size of the steps we take to reach a local minimum
  • update equation: $\theta = \theta - \eta * \nabla_\theta J(\theta)$