Regression, Classification and clustering

Assignment 2: Regression, Classification and clustering

Given the following data points:
1.What is the cost function for linear regression?
2.If we use the gradient descent algorithm to minimize the cost function for linear regression, what are the θ values and cost values in the first three iterations? Suppose the initial θ values are [1, 0.5]Tand the learning rate α is 0.1.

3. If we use one-vs-all strategy to a three class classification problem with three classes: -1, 0, 1, how many classifiers shall we train? What are they?

4.Describe the difference between linear regression and logistic regression. Please list at least three.

5. Support Vector Machines
(a)Suppose we are using a linear SVM (i.e., no kernel) and are given the following data set. Draw the decision boundary of linear SVM. Give a brief explanation.

(b)In the following image, circle the points such that by removing that example from the training set and retraining SVM, we would get a different decision boundary than training on the full sample. You do not need to provide a formal proof, but give a one or two sentence explanation.

6: K-means
(a)Consider the unlabeled two-dimensional data represented in the following figure. Using the two points marqued as squares as initial centroids, draw (on that same figure) the clusters obtained after one iteration of the k-means algorithm (k = 2).

(b)Does your solution change after another iteration of the k-means algorithm? Why?