2017年2月13日 Ridge Regression-优快云博客

本文介绍了如何使用Python从零开始实现岭回归，并将其结果与Scikit-learn库中的岭回归模型进行对比。通过随机生成的数据集，展示了岭回归求解过程及误差计算。

2019独角兽企业重金招聘Python工程师标准>>>

Suppose that for a known matrix A and vector b, we wish fo find a vector X such that

The ridge regression approach seeks to minimize the sum of squared residuals with a regularization term

An explicit solution is given by

import numpy as np
import ml_metrics as mtr

#prepare data
n_samples, n_features = 10, 5
np.random.seed(0)
X = np.random.randn(n_samples, n_features)
y = np.random.randn(n_samples)
y = (y-np.mean(y))/np.std(y)

#ridge regression implementation
def ridge_regression(X, y, alpha):
    tik_mat = alpha * np.identity(X.shape[1])
    coef = np.dot(np.transpose(X), X) + np.dot(np.transpose(tik_mat), tik_mat)
    coef = np.linalg.inv(coef)
    coef = np.dot(coef, np.transpose(X))
    coef = np.dot(coef, y)
    return coef

#train
coef = ridge_regression(X, y, 0)
print mtr.mse(np.dot(X, coef), y)
print coef

#0.677751350808
#[-0.30898281  0.02387927 -0.04666003 -0.2501281   0.16215742]

#scikit-learn implementation
from sklearn.linear_model import Ridge
r = Ridge(alpha=0, fit_intercept=False)
r.fit(X,y)
print mtr.mse(r.predict(X), y)
print r.coef_

#0.677751350808
#[-0.30898281  0.02387927 -0.04666003 -0.2501281   0.16215742]

转载于:https://my.oschina.net/airxiechao/blog/862746