自编码实现浅层神经网络

简介

不使用python神经网络相关库，自行编码实现浅层神经网络。其中隐藏层设置了4个unit。相关可参考Andrew NG 的深度学习课程。

1 - Packages¶

# Packages import
import numpy as np
import matplotlib.pyplot as plt
import sklearn.datasets

%matplotlib inline

2 - Dataset

def load_extra_datasets():  
    N = 200
    noisy_circles = sklearn.datasets.make_circles(n_samples=N, factor=.5, noise=.3)
    noisy_moons = sklearn.datasets.make_moons(n_samples=N, noise=.2)
    blobs = sklearn.datasets.make_blobs(n_samples=N, random_state=5, n_features=2, centers=6)
    gaussian_quantiles = sklearn.datasets.make_gaussian_quantiles(mean=None, cov=0.5, n_samples=N, n_features=2, n_classes=2, shuffle=True, random_state=None)
    no_structure = np.random.rand(N, 2), np.random.rand(N, 2)
    
    return noisy_circles, noisy_moons, blobs, gaussian_quantiles, no_structure

# Datasets
noisy_circles, noisy_moons, blobs, gaussian_quantiles, no_structure = load_extra_datasets()

datasets = {
   
   "noisy_circles": noisy_circles, 
            "noisy_moons": noisy_moons,
            "blobs": blobs, 
            "gaussian_quantiles": gaussian_quantiles}
dataset = "gaussian_quantiles"
X, Y = datasets[dataset]
X = X.T;  Y = Y.reshape(1, Y.size);

# make blobs binary
if dataset == blobs:
    Y = Y%2
    
# Visualize the data
plt.scatter(X[0, :],X[1, :], c = Y.reshape(-1,), s = 40, cmap = plt.cm.Spectral)

3 - Neural Network model(背景知识)

Logistic regression did not work well on the “flower dataset”. You are going to train a Neural Network with a single hidden layer.

Here is our model:

Mathematically:

For one example $x^{(i)}$:
$$z^{[1](i)}=W^{[1]}{x}^{(i)}+b^{[1](i)}\text{\hspace{1em}(1)}$$
$$a^{[1](i)}=\tanh (z^{[1](i)})\text{\hspace{1em}(2)}$$
$$z^{[2](i)}=W^{[2]}{a}^{[1](i)}+b^{[2]}$$