机器学习-Adative Linear Neuron(AdaLine)

Section I: Brief Introduction on AdaLine

The key difference between the AdaLine rule and Rosenblatt’s perceptron is that the weights are updated based on a linear activation function rather than a unit step function like in the perceptron. More specifically, the AdaLine algorithm compares the true class labels with the linear activation function’s continuous valued output to compute the model error and update the weights. In contrast, the perceptron compares the true class labels to the predicted class labels.

Section II: Code Snipet

方式一：基于批处理梯度下降模式的AdaLine实现

import numpy as np

class AdalineGD(object):
    def __init__(self,eta=0.01,n_iter=50,random_state=1):
        self.eta=eta
        self.n_iter=n_iter
        self.random_state=random_state

    def fit(self,X,y):
        rgen=np.random.RandomState(self.random_state)
        self.w_=rgen.normal(loc=0.0,scale=0.01,size=1+X.shape[1])
        self.cost_=[]

        for i in range(self.n_iter):
            net_input=self.net_input(X)
            output=self.activation(net_input)
            errors=(y-output)
            self.w_[1:]+=self.eta*X.T.dot(errors)
            self.w_[0]+=self.eta*errors.sum()
            cost=(errors**2).sum()/2.0
            self.cost_.append(cost)
        return self

    def net_input(self,X):
        return np.dot(X,self.w_[1:])+self.w_[0]

    def activation(self,X):
        return X

    def predict(self,X):
        return np.where(self.activation(self.net_input(X))>=0.0,1,-1)

小结：
批处理的关键在于权重的一次更新，依赖于与完整的数据集，即每完整训练数据集一次，才更新权重。因此，在寻找最优的动态过程将是梯度最大方向。

方式二：基于随机梯度下降模式的AdaLine实现

import numpy as np

class AdalineSGD(object