CS231n_assignment #1 Q1：k-Nearest Neighbor classifier

作业内容：

1. Q1: k-Nearest Neighbor classifier (20 points)
2. Q2: Training a Support Vector Machine (25 points)
3. Q3: Implement a Softmax classifier (20 points)
4. Q4: Two-Layer Neural Network (25 points)
5. Q5: Higher Level Representations: Image Features (10 points)

Q1：k-Nearest Neighbor classifier

完成一个KNN classifier 总共需要做两件事情：
第一件事情是将所有的training data读入
第二件事情是给定testing image 然后让其与所有的training data对比，然后将这幅图像的标签定位k个最近的image的标签。
所以，KNN可以说成是不需要训练时间的，但测试时间往往开销很大。

下面我将一步步解释cs231n assignment #1 中knn的代码

Step 1

# Run some setup code for this notebook.

import random
import numpy as np
from cs231n.data_utils import load_CIFAR10
import matplotlib.pyplot as plt

# This is a bit of magic to make matplotlib figures appear inline in the notebook
# rather than in a new window.
%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# Some more magic so that the notebook will reload external python modules;
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

在这里为load_CIFAR10模块的作用是加载数据集，此函数的返回值是return Xtr, Ytr, Xte, Yte
autoreload 2：自动重载%aimport排除的模块之外的所有模块，因为后面要求你修改.py文件里面的内容有了这个autoreeload之后，修改之后就会重新加载 ，在这里我们不深究是什么意思。

Step 2

# Load the raw CIFAR-10 data.
cifar10_dir = 'cs231n/datasets/cifar-10-batches-py'

# Cleaning up variables to prevent loading data multiple times (which may cause memory issue)
try:
   del X_train, y_train
   del X_test, y_test
   print('Clear previously loaded data.')
except:
   pass
# 这里的try except是为了防止多次loading
X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)

# As a sanity check, we print out the size of the training and test data.
print('Training data shape: ', X_train.shape)
print('Training labels shape: ', y_train.shape)
print('Test data shape: ', X_test.shape)
print('Test labels shape: ', y_test.shape)

数据集已经载入成功了：

Training data shape:  (50000, 32, 32, 3)
Training labels shape:  (50000,)
Test data shape:  (10000, 32, 32, 3)
Test labels shape:  (10000,)

Step 3

# Visualize some examples from the dataset.
# We show a few examples of training images from each class.
classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
num_classes = len(classes)
samples_per_class = 7
for y, cls in enumerate(classes):
    idxs = np.flatnonzero(y_train == y)
    # np.flatnonzero()函数输入一个矩阵，返回扁平化后矩阵中非零元素的位置（index）
    idxs = np.random.choice(idxs, samples_per_class, replace=False)
    for i, idx in enumerate(idxs):
        plt_idx = i * num_classes + y + 1
        plt.subplot(samples_per_class, num_classes, plt_idx)
        plt.imshow(X_train[idx].astype('uint8'))
        plt.axis('off')
        if i == 0:
            plt.title(cls)
plt.show()

Step 4

# Subsample the data for more efficient code execution in this exercise
num_training = 5000
mask = list(range(num_training))
X_train = X_train[mask]
y_train = y_train[mask]

num_test = 500
mask = list(range(num_test))
X_test = X_test[mask]
y_test = y_test[mask]

# Reshape the image data into rows
X_train = np.reshape(X_train, (X_train.shape[0], -1))
X_test = np.reshape(X_test, (X_test.shape[0], -1))
print(X_train.shape, X_test.shape)

这里的Subsample并不是下采样的意思，只是因为原来的数据集的图片数量很多，在我们这个小小的demo中只需要取少量来演示就行。所以就在原来50000张training中选取5000张，10000张testing testing data中选取500张作为我们demo的data set

(5000, 3072) (500, 3072)

Step 5

from cs231n.classifiers import KNearestNeighbor

# Create a kNN classifier instance. 
# Remember that training a kNN classifier is a noop(空操作): 
# the Classifier simply remembers the data and does no further processing 
classifier = KNearestNeighbor()
classifier.train(X_train, y_train)

这是原文，如果你不想看英文，下面有翻译：
We would now like to classify the test data with the kNN classifier. Recall that we can break down this process into two steps:

1. First we must compute the distances between all test examples and all train examples.
2. Given these distances, for each test example we find the k nearest examples and have them vote for the label

Lets begin with computing the distance matrix between all training and test examples. For example, if there are Ntr training examples and Nte test examples, this stage should result in a Nte x Ntr matrix where each element (i,j) is the distance between the i-th test and j-th train example.

Note: For the three distance computations that we require you to implement in this notebook, you may not use the np.linalg.norm() function that numpy provides.

First, open cs231n/classifiers/k_nearest_neighbor.py and implement the function compute_distances_two_loops that uses a (very inefficient) double loop over all pairs of (test, train) examples and computes the distance matrix one element at a time.

翻译如下：
先来对test data 进行分类，回想以下我们之前所说的，要实现分类我们应该：

1. 计算test data中每一张图片于training data中所有图片的distances
2. 更具K个最近的图片(distances 最小) 投票选出这张图片的lable

让我们从计算distance matrix 开始，假如你有N个training examples 和 M个testing examples那么我们需要做的就是计算出M x N 的矩阵，每一个矩阵中的元素element (i,j)都是一张testing image 到一张training image 的距离。

Note:每一个计算距离的算法都要求动手实现，并且还不能用np.linalg.norm()这些numpy
已经提供的函数，所以得自己写函数（斯坦福果然要求严格）

首先，打开cs231n/classifiers/k_nearest_neighbor.py并实现compute_distances_two_loops函数，该函数在所有（测试、训练）示例上使用非常低效的双循环，并一次计算一个矩阵中的元素。

打开文件之后 写下计算L2距离的公式：dists[i, j] = np.sqrt(np.sum(np.square(X[i] - self.X_train[j])))虽然这个公式的计算效率很低但是作为初学者，我们可以不太考虑效率的问题。

  def compute_distances_two_loops(self, X):
        """
        Compute the distance between each test point in X and each training point
        in self.X_train using a nested loop（嵌套循环） over both the training data and the
        test data.

        Inputs:
        - X: A numpy array of shape (num_test, D) containing test data.

        Returns:
        - dists: A numpy array of shape (num_test, num_train) where dists[i, j]
          is the Euclidean distance between the ith test point and the jth training
          point.
        """
        num_test = X.shape[0]
        num_train = self.X_train.shape[0]
        dists = np.zeros((num_test, num_train))
        for i in range(num_test):
            for j in range(num_train):
                #####################################################################
                # TODO:                                                             #
                # Compute the l2 distance between the ith test point and the jth    #
                # training point, and store the result in dists[i, j]. You should   #
                # not use a loop over dimension, nor use np.linalg.norm().          #
                #####################################################################
                # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
                dists[i, j] = np.sqrt(np.sum((X[i]-self.X_train[j])**2))
                pass

                # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
        return dists

Step 6

# Open cs231n/classifiers/k_nearest_neighbor.py and implement
# compute_distances_two_loops.

# Test your implementation:
dists = classifier.compute_distances_two_loops(X_test)
print(dists.shape)

(500, 5000)

# We can visualize the distance matrix: each row is a single test example and
# its distances to training examples
plt.imshow(dists, interpolation='none')
plt.show()

如果你显示的是全黑色就说明compute_distances_two_loops函数没有写或者是写错了