task4 掌握回归模型的评估及超参数调优

!pip install numpy
Requirement already satisfied: numpy in c:\users\lenovo\anaconda3\lib\site-packages (1.19.2)
!pip install pandas
Requirement already satisfied: pandas in c:\users\lenovo\anaconda3\lib\site-packages (1.1.3)
Requirement already satisfied: python-dateutil>=2.7.3 in c:\users\lenovo\anaconda3\lib\site-packages (from pandas) (2.8.1)
Requirement already satisfied: numpy>=1.15.4 in c:\users\lenovo\anaconda3\lib\site-packages (from pandas) (1.19.2)
Requirement already satisfied: pytz>=2017.2 in c:\users\lenovo\anaconda3\lib\site-packages (from pandas) (2020.1)
Requirement already satisfied: six>=1.5 in c:\users\lenovo\anaconda3\lib\site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
!pip install matplotlib
Requirement already satisfied: matplotlib in c:\users\lenovo\anaconda3\lib\site-packages (3.3.2)
Requirement already satisfied: cycler>=0.10 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (2.4.7)
Requirement already satisfied: pillow>=6.2.0 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (8.0.1)
Requirement already satisfied: certifi>=2020.06.20 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (2020.6.20)
Requirement already satisfied: numpy>=1.15 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (1.19.2)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (1.3.0)
Requirement already satisfied: python-dateutil>=2.1 in c:\users\lenovo\anaconda3\lib\site-packages (from matplotlib) (2.8.1)
Requirement already satisfied: six in c:\users\lenovo\anaconda3\lib\site-packages (from cycler>=0.10->matplotlib) (1.15.0)
# 引入相关科学计算包
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use("ggplot") 
import seaborn as sns
from sklearn import datasets
boston = datasets.load_boston() # 返回一个类似于字典的类
X = boston.data
y = boston.target
features = boston.feature_names
boston_data = pd.DataFrame(X,columns=features)
boston_data["Price"] = y
boston_data.head()
CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX PTRATIO B LSTAT Price
0 0.00632 18.0 2.31 0.0 0.538 6.575 65.2 4.0900 1.0 296.0 15.3 396.90 4.98 24.0
1 0.02731 0.0 7.07 0.0 0.469 6.421 78.9 4.9671 2.0 242.0 17.8 396.90 9.14 21.6
2 0.02729 0.0 7.07 0.0 0.469 7.185 61.1 4.9671 2.0 242.0 17.8 392.83 4.03 34.7
3 0.03237 0.0 2.18 0.0 0.458 6.998 45.8 6.0622 3.0 222.0 18.7 394.63 2.94 33.4
4 0.06905 0.0 2.18 0.0 0.458 7.147 54.2 6.0622 3.0 222.0 18.7 396.90 5.33 36.2
sns.scatterplot(boston_data['NOX'],boston_data['Price'],color="r",alpha=0.6)
plt.title("Price~NOX")
plt.show()
C:\Users\lenovo\anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-K8BUe6GK-1616577548323)(output_5_1.png)]

from sklearn import datasets
iris = datasets.load_iris()
X = iris.data
y = iris.target
features = iris.feature_names
iris_data = pd.DataFrame(X,columns=features)
iris_data['target'] = y
iris_data.head()

sepal length (cm) sepal width (cm) petal length (cm) petal width (cm) target
0 5.1 3.5 1.4 0.2 0
1 4.9 3.0 1.4 0.2 0
2 4.7 3.2 1.3 0.2 0
3 4.6 3.1 1.5 0.2 0
4 5.0 3.6 1.4 0.2 0
# 可视化特征
marker = ['s','x','o']
for index,c in enumerate(np.unique(y)):
    plt.scatter(x=iris_data.loc[y==c,"sepal length (cm)"],y=iris_data.loc[y==c,"sepal width (cm)"],alpha=0.9,label=c,marker=marker[c])
plt.xlabel("sepal length (cm)")
plt.ylabel("sepal width (cm)")
plt.legend()
plt.show()

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-NSwUJolW-1616577548326)(output_7_0.png)]

# 生成月牙型非凸集
from sklearn import datasets
x, y = datasets.make_moons(n_samples=4000, shuffle=True,
                  noise=0.05, random_state=None)
for index,c in enumerate(np.unique(y)):
    plt.scatter(x[y==c,0],x[y==c,1],s=7)
plt.show()

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-y1F53mnY-1616577548328)(output_8_0.png)]

from sklearn import datasets
x, y = datasets.make_blobs(n_samples=5000, n_features=2, centers=3)
for index,c in enumerate(np.unique(y)):
    plt.scatter(x[y==c,0],x[y==c,1],s=7)
plt.show()

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Tq4trmmQ-1616577548331)(output_9_0.png)]

# 生成符合正态分布的聚类数据
from sklearn import datasets
x, y = datasets.make_blobs(n_samples=1000, n_features=2, centers=4)
for index,c in enumerate(np.unique(y)):
    plt.scatter(x[y==c, 1], x[y==c, 0],s=7)
plt.show(
<think>好的,用户现在想了解XGBoost的超参数。之前他们问过关于R语言多线程和GPU加速的问题,可能是在进行机器学习模型训练时遇到了性能问题,现在想要化模型效果。我需要先回顾XGBoost的基本概念和常用参数,然后分步骤解释方法。 首先,用户可能对XGBoost已经有基本了解,但需要指导如何整参数来提升模型性能。我应该从参数分类开始,比如通用参数、Booster参数和学习目标参数,这样结构更清晰。接下来,可能需要介绍的方法,比如网格搜索、随机搜索和贝叶斯化,特别是R中的具体实现,比如mlr3或tidymodels包。 用户可能更关注实际应用,所以示例代码很重要。需要确保代码正确,并且用R语言展示,比如使用mlr3和paradox包进行网格搜索。同时,用户可能对自动化工具感兴趣,比如AutoML,但也要提醒他们注意过拟合和计算资源的问题。 还要考虑用户可能的误区,比如盲目参,应该先确保数据预处理正确,特征工程有效。另外,参的顺序也很重要,比如先整学习率,再树的结构参数,最后正则化参数。需要强交叉验证的必要性,以及早停法的作用,防止过拟合。 最后,检查是否需要补充注意事项,比如计算资源、早停法使用,以及特征工程的重要性。可能用户需要实际案例,但如果没有具体场景,保持一般性的建议即可。确保回答结构清晰,步骤明确,帮助用户逐步进行。</think>好的!XGBoost(eXtreme Gradient Boosting)的超参数是提升模型性能的关键步骤。以下是系统化的方法及关键参数解析: --- ### **一、XGBoost核心超参数分类** #### **1. 通用参数** - `nthread`:并行线程数(需与硬件匹配) - `booster`:基模型类型(`gbtree`/`gblinear`/`dart`,默认`gbtree`) #### **2. Booster参数(树模型)** - **树结构**: - `max_depth`:单棵树最大深度(控制过拟合,常用3-8) - `min_child_weight`:子节点最小样本权重和(防止过拟合) - `gamma`(`min_split_loss`):分裂所需最小损失减少量 - **随机性**: - `subsample`:样本采样比例(0.6-1.0) - `colsample_bytree`:特征采样比例(0.6-1.0) - **正则化**: - `lambda`:L2正则化系数(默认1) - `alpha`:L1正则化系数(默认0) #### **3. 学习目标参数** - `eta`(`learning_rate`):学习率(常用0.01-0.3) - `n_estimators`(`num_round`):树的数量(与`eta`反向节) - `objective`:损失函数(如`binary:logistic`、`reg:squarederror`) - `eval_metric`:评估指标(如`rmse`、`logloss`、`auc`) --- ### **二、方法** #### **1. 网格搜索(Grid Search)** - **适用场景**:参数组合少时精确搜索 - **R实现示例**(使用`mlr3`和`paradox`): ```r library(mlr3) library(mlr3learners) library(paradox) # 定义参数空间 params <- ps( max_depth = p_int(3, 8), eta = p_dbl(0.01, 0.3), subsample = p_dbl(0.6, 1) ) # 创建任务和学习器 task <- as_task_regr(iris, target = "Sepal.Length") learner <- lrn("regr.xgboost") # 网格搜索 instance <- tune( method = tnr("grid_search", resolution = 10), task = task, learner = learner, resampling = rsmp("cv", folds = 5), measure = msr("regr.rmse"), param_set = params ) # 获取最参数 instance$result ``` #### **2. 随机搜索(Random Search)** - **势**:高维参数空间更高效 - **R实现**(使用`tidymodels`): ```r library(tidymodels) library(xgboost) # 定义模型 xgb_spec <- boost_tree( tree_depth = tune(), learn_rate = tune(), loss_reduction = tune() ) %>% set_engine("xgboost") %>% set_mode("regression") # 参数范围 params <- parameters( tree_depth(range = c(3, 8)), learn_rate(range = c(-2, -1)), # 10^[-2, -1] loss_reduction(range = c(0, 0.3)) ) # 随机搜索 xgb_rs <- tune_grid( xgb_spec, preprocessor = recipe(Sepal.Length ~ ., iris), resamples = vfold_cv(iris, v = 5), grid = 20, # 随机生成20组参数 metrics = metric_set(rmse) ) show_best(xgb_rs, "rmse") ``` #### **3. 贝叶斯化(Bayesian Optimization)** - **工具**:`mlr3mbo`或`ParBayesianOptimization` - **势**:智能探索参数空间,减少计算量 --- ### **三、策略** #### **1. 先级排序** ```mermaid graph LR A[学习率 eta] --> B[树数量 n_estimators] B --> C[树深度 max_depth] C --> D[样本/特征采样 subsample/colsample] D --> E[正则化 lambda/alpha] ``` #### **2. 经验法则** - **学习率与树数量**: $$ \text{低eta} \rightarrow \text{高n_estimators} \quad (\text{如} \ eta=0.01, \ n\_estimators=5000) $$ 使用`early_stopping_rounds`自动控制迭代次数 - **树深度**: 从`max_depth=3`开始逐步增加,观察验证集性能 - **正则化**: 若模型过拟合,先增大`lambda`和`alpha` #### **3. 早停法(Early Stopping)** ```r # 在xgb.train中设置 watchlist <- list(train = dtrain, test = dtest) model <- xgb.train( params = list(eta = 0.1), data = dtrain, nrounds = 1000, watchlist = watchlist, early_stopping_rounds = 20 # 连续20轮无提升则停止 ) ``` --- ### **四、自动化工具** #### **1. AutoML框架** - `h2o.automl()`:自动完成超参数化 - `autoxgb()`:专用于XGBoost的自动化参 #### **2. 可视化工具** - **参数重要性分析**: ```r xgb.importance(model = model) %>% xgb.plot.importance() ``` - **学习曲线诊断**: ```r model$evaluation_log %>% ggplot(aes(iter, train_rmse)) + geom_line(color="blue") + geom_line(aes(y = test_rmse), color="red") ``` --- ### **五、注意事项** 1. **数据预处理**: 确保类别变量已编码,缺失值处理正确 2. **计算资源**: 参时启用并行(如`nthread=8`) 3. **过拟合验证**: 始终使用交叉验证,避免依赖单一训练集结果 4. **参数交互**: `subsample`与`eta`可能存在协同效应,需组合整 如果需要针对具体数据集(如分类/回归问题)的示例,可以补充说明任务类型!
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值