第10节-VC维和模型选取

最新推荐文章于 2022-10-07 22:29:20 发布

北邮张博

最新推荐文章于 2022-10-07 22:29:20 发布

阅读量821

点赞数

CC 4.0 BY-SA版权

分类专栏：机器学习机器学习要点文章标签：机器学习 VC维模型选取文本分类

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接：https://blog.youkuaiyun.com/Irving_zhang/article/details/62216348

机器学习同时被 2 个专栏收录

19 篇文章

订阅专栏

机器学习要点

15 篇文章

订阅专栏

本篇内容基于NG的第10个视频讲解VC维和模型选取的重要性，并通过文本分类的实际例子进行说明。作者分享了自己在实践中偏好的CHI公式及详细思路和代码。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

　　NG的第10个视频讲VC维和模型选取。NG举了文本分类的例子，说明模型选取在机器学习中的作用，我个人在实际应用中更习惯使用CHI公式，详细的文本分类的思路和代码可以参看我之前的博客（http://blog.youkuaiyun.com/irving_zhang/article/details/52214130）

博客等级

码龄11年

121
原创

279
点赞

789
收藏

846
粉丝

关注

私信

热门文章

分类专栏

展开全部收起

上一篇：: 第9节-偏差与方差、联合界定理和一致收敛定理

下一篇：: 第11节-贝叶斯正则化与ML应用建议

最新评论

Kaggle入门实例-预测房价
闻窗雪: 可视化缺失值部分。后面的就可以正常运行了。import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt train = pd.read_csv("C:/Users/lenovo/Desktop/train_engineered.csv") test = pd.read_csv("C:/Users/lenovo/Desktop/test_engineered.csv") train_ID = train['Id'] test_ID = test['Id'] train.drop('Id', axis=1, inplace=True) test.drop('Id', axis=1, inplace=True) # 如果训练集中有 'SalePrice'，则保存并删除 if 'SalePrice' in train.columns: y_train = train['SalePrice'] train.drop('SalePrice', axis=1, inplace=True) all_data = pd.concat([train, test], axis=0, ignore_index=True) all_data_na = (all_data.isnull().mean() * 100).sort_values(ascending=False) all_data_na = all_data_na[all_data_na > 0] f, ax = plt.subplots(figsize=(15, 12)) plt.xticks(rotation=90) sns.barplot(x=all_data_na.index, y=all_data_na.values) plt.xlabel('Features', fontsize=15) plt.ylabel('Percent of missing values', fontsize=15) plt.title('Percent missing data by feature', fontsize=15) plt.show()
Kaggle入门实例-预测房价
闻窗雪: 另外，我解决了博主没有粘贴成功合并测试集和训练集的问题：# 加载数据 train = pd.read_csv("C:/Users/lenovo/Desktop/train_engineered.csv") test = pd.read_csv("C:/Users/lenovo/Desktop/test_engineered.csv") # 保存原始ID，之后可能需要使用 train_ID = train['Id'] test_ID = test['Id'] # 如果存在目标变量，先将其从训练集中分离出来 if 'SalePrice' in train.columns: y_train = train['SalePrice'].values train.drop(['SalePrice'], axis=1, inplace=True) # 删除两个数据集中的 Id 列，因为它们对于模型来说通常是无关紧要的 train.drop(['Id'], axis=1, inplace=True) test.drop(['Id'], axis=1, inplace=True) # 合并训练集和测试集 all_data = pd.concat([train, test]).reset_index(drop=True)
Kaggle入门实例-预测房价
闻窗雪: # MSSubClass: NA most likely means No building class. # We can replace missing values with None all_data['MSSubClass'] = all_data['MSSubClass'].fillna("None") # Check if there are any remaining missing values all_data_na = (all_data.isnull().sum() / len(all_data)) * 100 all_data_na = all_data_na.drop(all_data_na[all_data_na == 0].index).sort_values(ascending=False) missing_data = pd.DataFrame({'Missing Ratio': all_data_na}) print(missing_data.head())
Kaggle入门实例-预测房价
闻窗雪: # Functional: data description says NA means typical all_data["Functional"] = all_data["Functional"].fillna("Typ") # Electrical: It has one NA value. Since this feature has mostly 'SBrkr', we can set that for the missing value. all_data['Electrical'] = all_data['Electrical'].fillna(all_data['Electrical'].mode()[0]) # KitchenQual: Only one NA value, and same as Electrical, we set 'TA' (which is the most frequent) # for the missing value in KitchenQual. all_data['KitchenQual'] = all_data['KitchenQual'].fillna(all_data['KitchenQual'].mode()[0]) # Exterior1st and Exterior2nd: Both have only one missing value. # We will just substitute in the most common string. all_data['Exterior1st'] = all_data['Exterior1st'].fillna(all_data['Exterior1st'].mode()[0]) all_data['Exterior2nd'] = all_data['Exterior2nd'].fillna(all_data['Exterior2nd'].mode()[0]) # SaleType: Fill in again with most frequent which is "WD" all_data['SaleType'] = all_data['SaleType'].fillna(all_data['SaleType'].mode()[0])
Kaggle入门实例-预测房价
闻窗雪: # other houses in its neighborhood, we can fill in missing values by the median LotFrontage of the neighborhood. all_data["LotFrontage"] = all_data.groupby("Neighborhood")["LotFrontage"].transform( lambda x: x.fillna(x.median())) # GarageType, GarageFinish, GarageQual and GarageCond: Replacing missing data with None for col in ('GarageType', 'GarageFinish', 'GarageQual', 'GarageCond'): all_data[col] = all_data[col].fillna("None") # GarageYrBlt, GarageArea and GarageCars: Replacing missing data with 0 # (Since No garage = no cars in such garage.) for col in ('GarageYrBlt', 'GarageArea', 'GarageCars'): all_data[col] = all_data[col].fillna(0) # BsmtFinSF1, BsmtFinSF2, BsmtUnfSF, TotalBsmtSF, BsmtFullBath and BsmtHalfBath: # missing values are likely zero for having no basement for col in ('BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF', 'BsmtFullBath', 'BsmtHalfBath'): all_data[col] = all_data[col].fillna(0) # BsmtQual, BsmtCond, BsmtExposure, BsmtFinType1 and

大家在看

最新文章

目录

展开全部

收起

评论

被折叠的条评论为什么被折叠?

到【灌水乐园】发言

查看更多评论

添加红包

成就一亿技术人!

hope_wisdom

发出的红包

实付元

使用余额支付

点击重新获取

扫码支付

钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。