回归预测3：机器学习处理悉尼-墨尔数据集

1 介绍

我们处理悉尼-墨尔本的房价预测问题，数据集有些变量是字符串形式，有些和时间相关，在本实验中，我们主要使用时间编码、异常点检测zscore、特征编码：LabelEncoder、MEstimateEncoder，最后通过实验对比，给出结论。
数据集下载：https://download.youkuaiyun.com/download/ww596520206/87504283

2 模型1：baseline模型

2.1 导入包和数据

import numpy as np
import pandas as pd
from datetime import datetime
from sklearn.ensemble import RandomForestRegressor,GradientBoostingRegressor

from sklearn.metrics import mean_squared_error,r2_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.preprocessing import LabelEncoder

np.random.seed(123)
data = pd.read_csv("data.csv")

2.2 处理时间维度，将date特征进行拆分，只取month、day

data["month"] = data.date.apply(lambda x: x.split()[0].split("-")[1]).astype("int")
data["day"] = data.date.apply(lambda x: x.split()[0].split("-")[2]).astype("int")

2.3 剔除异常点

我们使用z-score标准化先将数据转换为均值为0，方差为1的标准正态分布，然后只保留u-3σ到u+3σ之间的数据，为了方便，我们取绝对值，只保留小于u+3σ的数据。

from scipy import stats
z = np.abs(stats.zscore(data[['sqft_living','sqft_above','bathrooms','yr_built','sqft_lot','bedrooms']]))
data = data[(z<3).all(axis=1)]
print(len(data))

4435

2.3 删除无用的特征

data.drop(['street','statezip','city',"country","date"],axis=1, inplace = True)

2.4 划分数据集，数据归一化

x = data.drop(["price"],axis=1)
y = data["price"]
X_train, X_test, y_train, y_test = train_test_split(x, y,train_size=0.7,shuffle = False)  
scaler = MinMaxScaler() #归一化
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)

2.5 模型训练和验证

from sklearn.linear_model import LinearRegression
rf = LinearRegression()
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)
print("mse:{}".format(mean_squared_error(y_test, y_pred)))
print("r2-score:{}".format(r2_score(y_test, y_pred)))

mse:707591647854.4553
r2-score:0.04118363082296694

3 模型2：使用LabelEncoder编码

3.1 增加LabelEncoder编码

我们对’street’,‘statezip’,'city’这三个维度进行编码

data["street"] = LabelEncoder().fit_transform(data["street"])
data["city"] = LabelEncoder().fit_transform(data["city"])
data["statezip"] = LabelEncoder().fit_transform(data["statezip"])

3.2 整体代码

import numpy as np
import pandas as pd
from datetime import datetime
from sklearn.ensemble import RandomForestRegressor,GradientBoostingRegressor

from sklearn.metrics import mean_squared_error,r2_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.preprocessing import LabelEncoder

np.random.seed(123)
data = pd.read_csv("data.csv")

data["month"] = data.date.apply(lambda x: x.split()[0].split("-")[1]).astype("int")
data["day"] = data.date.apply(lambda x: x.split()[0].split("-")[2]).astype("int")

from scipy import stats
z = np.abs(stats.zscore(data[['sqft_living','sqft_above','bathrooms','yr_built','sqft_lot','bedrooms']]))
data = data[(z<3).all(axis=1)]

data["street"] = LabelEncoder().fit_transform(data["street"])
data["city"] = LabelEncoder().fit_transform(data["city"])
data["statezip"] = LabelEncoder().fit_transform(data["statezip"])

data.drop(["country","date"],axis=1, inplace = True)
x = data.drop(["price"],axis=1)
y = data["price"]
X_train, X_test, y_train, y_test = train_test_split(x, y,train_size=0.7,shuffle = False)  
scaler = MinMaxScaler()
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)

from sklearn.linear_model import LinearRegression
rf = LinearRegression()
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)
print("mse:{}".format(mean_squared_error(y_test, y_pred)))
print("r2-score:{}".format(r2_score(y_test, y_pred)))

mse:706767197718.183
r2-score:0.04230079534663678

4 模型3：使用MEstimateEncoder编码

4.1 MEstimateEncoder编码

使用category_encoders的MEstimateEncoder编码

from category_encoders import MEstimateEncoder
encoder = MEstimateEncoder(cols=['street','statezip','city'],m=0.5)
encoder.fit(x,y)
x= encoder.transform(x)

4.2 整体代码

import numpy as np
import pandas as pd
from datetime import datetime
from sklearn.ensemble import RandomForestRegressor,GradientBoostingRegressor

from sklearn.metrics import mean_squared_error,r2_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.preprocessing import LabelEncoder

np.random.seed(123)
data = pd.read_csv("data.csv")

data["month"] = data.date.apply(lambda x: x.split()[0].split("-")[1]).astype("int")
data["day"] = data.date.apply(lambda x: x.split()[0].split("-")[2]).astype("int")

from scipy import stats
z = np.abs(stats.zscore(data[['sqft_living','sqft_above','bathrooms','yr_built','sqft_lot','bedrooms']]))
data = data[(z<3).all(axis=1)]

data.drop(["country","date"],axis=1, inplace = True)
x = data.drop(["price"],axis=1)
y = data["price"]

from category_encoders import MEstimateEncoder
encoder = MEstimateEncoder(cols=['street','statezip','city'],m=0.5)
encoder.fit(x,y)
x= encoder.transform(x)

X_train, X_test, y_train, y_test = train_test_split(x, y,train_size=0.7,shuffle = False)  
scaler = MinMaxScaler()
scaler.fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)

from sklearn.linear_model import LinearRegression
rf = LinearRegression()
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)
print("mse:{}".format(mean_squared_error(y_test, y_pred)))
print("r2-score:{}".format(r2_score(y_test, y_pred)))

mse:612704480.2477968
r2-score:0.9991697597238308

4.3 对比

通过对比，我们发现不同的特征编码能大幅度提升模型的性能。