2022吴恩达机器学习Deeplearning.ai课程编程作业C3_W1: KMeans_Assignment

K-means Clustering

In this this exercise, you will implement the K-means algorithm and use it for image compression.

• You will start with a sample dataset that will help you gain an intuition of how the K-means algorithm works.
• After that, you wil use the K-means algorithm for image compression by reducing the number of colors that occur in an image to only those that are most common in that image.

import numpy as np
import matplotlib.pyplot as plt
from utils import *

%matplotlib inline

1 - Implementing K-means

The K-means algorithm is a method to automatically cluster similar
data points together.

• Concretely, you are given a training set { x ( 1 ) , . . . , x ( m ) } \{x^{(1)}, ..., x^{(m)}\} { x(1),...,x(m)}, and you want
  to group the data into a few cohesive “clusters”.

• K-means is an iterative procedure that

  • Starts by guessing the initial centroids, and then
  • Refines this guess by
    • Repeatedly assigning examples to their closest centroids, and then
    • Recomputing the centroids based on the assignments.

• In pseudocode, the K-means algorithm is as follows:

  # Initialize centroids
  # K is the number of clusters
  centroids = kMeans_init_centroids(X, K)
  
  for iter in range(iterations):
      # Cluster assignment step: 
      # Assign each data point to the closest centroid. 
      # idx[i] corresponds to the index of the centroid 
      # assigned to example i
      idx = find_closest_centroids(X, centroids)
  
      # Move centroid step: 
      # Compute means based on centroid assignments
      centroids = compute_means(X, idx, K)
  

• The inner-loop of the algorithm repeatedly carries out two steps:

  • (i) Assigning each training example $x^{(i)}$ to its closest centroid, and
  • (ii) Recomputing the mean of each centroid using the points assigned to it.

• The $K$-means algorithm will always converge to some final set of means for the centroids.

• However, that the converged solution may not always be ideal and depends on the initial setting of the centroids.

  • Therefore, in practice the K-means algorithm is usually run a few times with different random initializations.
  • One way to choose between these different solutions from different random initializations is to choose the one with the lowest cost function value (distortion).

You will implement the two phases of the K-means algorithm separately
in the next sections.

• You will start by completing find_closest_centroid and then proceed to complete compute_centroids.

1.1 Finding closest centroids

In the “cluster assignment” phase of the K-means algorithm, the
algorithm assigns every training example $x^{(i)}$ to its closest
centroid, given the current positions of centroids.

Exercise 1

Your task is to complete the code in find_closest_centroids.

• This function takes the data matrix X and the locations of all
  centroids inside centroids
• It should output a one-dimensional array idx (which has the same number of elements as X) that holds the index of the closest centroid (a value in { 1 , . . . , K } \{1,...,K\} { 1,...,K}, where $K$ is total number of centroids) to every training example .
• Specifically, for every example $x^{(i)}$ we set
  $$c^{(i)}:=j\mathrm{that}\mathrm{minimizes}||x^{(i)}-{\mu }_j|{|}^2$$