Machine Learning Week 1

Mathine Learning Week1

---

Classify

Supervised learning:

“right answers”given

i) Regression: Predict continuous valued outputs

ii) Classification: Discrete valued output(0 or 1 or even more)

Unsupervised learning

Clustering(Algorithm)

Liner Regression

Hypothesis Function

The Hypothesis Function:

$$h_\theta(x)=\theta_0+\theta_1x$$

in details:

the way of choosing parameters:

Cost Function

Cost Function:

$$J(\theta_0,\theta_1)=\frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)}-y^{(i)})^2$$

3D cost function figures:

using contour figures to represent 3D plots:

Gradient Descent

Gradient Descent:

$$\theta_j:=\theta_j-\alpha\frac{\partial}{\partial\theta_j}J(\theta_0,\theta_1)$$ for j=0 and j=1

Especially Gradient Descent for Linear Regression:
repeat until convergence:{

$$\theta_0:=\theta_0-\alpha\frac1m\sum^{m}_{i=1}(h_\theta(x^{(i)})-y^{(i)})$$
$$\theta_1:=\theta_1-\alpha\frac1m\sum^m_{i=1}((h_\theta(x^{(i)})-y^{(i)})x^{(i)})$$ }

Simultaneous update the parameters:

Gradient descent with one variable:

Why learning rate shouldn’t be too big or small:

But you can keep the learning rate fixed with the steps automatically getting small:

with 2 parameters:

Liner regression’s cost function is always bowl shape:
So there are no local optima but only one global optimum

---