组队学习第三天：心跳信号分类之特征工程

原题目地址：零基础入门数据挖掘-心跳信号分类预测
本文是鸣鹤星空小组组队学习的集体成果。
id:李由，一只小呆
源地址：组队学习

1 学习目标

• 学习时间序列数据的特征预处理方法
• 学习时间序列特征处理工具 Tsfresh（TimeSeries Fresh）的使用

2 内容介绍

• 数据预处理
  • 时间序列数据格式处理
  • 加入时间步特征time
• 特征工程
  • 时间序列特征构造
  • 特征筛选
  • 使用 tsfresh 进行时间序列特征处理

3 学习代码

# 包导入
import pandas as pd
import numpy as np
import tsfresh as tsf
from tsfresh import extract_features, select_features
from tsfresh.utilities.dataframe_functions import impute

E:\myanaconda\lib\site-packages\statsmodels\tools\_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm

data_train = pd.read_csv("train.csv")
data_test_A = pd.read_csv("testA.csv")

print(data_train.shape)
print(data_test_A.shape)

(100000, 3)
(20000, 2)

data_train.head()

	id	heartbeat_signals	label
0	0	0.9912297987616655,0.9435330436439665,0.764677...	0.0
1	1	0.9714822034884503,0.9289687459588268,0.572932...	0.0
2	2	1.0,0.9591487564065292,0.7013782792997189,0.23...	2.0
3	3	0.9757952826275774,0.9340884687738161,0.659636...	0.0
4	4	0.0,0.055816398940721094,0.26129357194994196,0...	2.0

data_test_A.head()

	id	heartbeat_signals
0	100000	0.9915713654170097,1.0,0.6318163407681274,0.13...
1	100001	0.6075533139615096,0.5417083883163654,0.340694...
2	100002	0.9752726292239277,0.6710965234906665,0.686758...
3	100003	0.9956348033996116,0.9170249621481004,0.521096...
4	100004	1.0,0.8879490481178918,0.745564725322326,0.531...

# 将以逗号隔开的心跳信号拆分，整合为新的dataframe
train_heartbeat_df = data_train["heartbeat_signals"].str.split(",", expand=True).stack()

train_heartbeat_df = train_heartbeat_df.reset_index()

train_heartbeat_df.set_index("level_0")

	level_1	0
level_0
0	0	0.9912297987616655
0	1	0.9435330436439665
0	2	0.7646772997256593
0	3	0.6185708990212999
0	4	0.3796321642826237
...	...	...
99999	200	0.0
99999	201	0.0
99999	202	0.0
99999	203	0.0
99999	204	0.0

20500000 rows × 2 columns

# 对心电特征进行行转列处理，同时为每个心电信号加入时间步特征time
# 将以逗号隔开的心跳信号拆分，整合为新的dataframe，使用stack函数整合为一维的dataframe
train_heartbeat_df = data_train["heartbeat_signals"].str.split(",", expand=True).stack()
# 重新命名列名
train_heartbeat_df = train_heartbeat_df.reset_index()
# 设置新的dataframe的索引为level_0列
train_heartbeat_df = train_heartbeat_df.set_index("level_0")
# 新的索引无命名
train_heartbeat_df.index.name = None
# 重命名列名
train_heartbeat_df.rename(columns={"level_1":"time", 0:"heartbeat_signals"}, inplace=True)
# 设置值的类型
train_heartbeat_df["heartbeat_signals"] = train_heartbeat_df["heartbeat_signals"].astype(float)

train_heartbeat_df

	time	heartbeat_signals
0	0	0.991230
0	1	0.943533
0	2	0.764677
0	3	0.618571
0	4	0.379632
...	...	...
99999	200	0.000000
99999	201	0.000000
99999	202	0.000000
99999	203	0.000000
99999	204	0.000000

20500000 rows × 2 columns

# 将处理后的心电特征加入到训练数据中，同时将训练数据label列单独存储
# 即拼接为带有时间序列的dataframe
data_train_label = data_train["label"]
data_train = data_train.drop("label", axis=1)
data_train = data_train.drop("heartbeat_signals", axis=1)
data_train = data_train.join(train_heartbeat_df)

data_train

	id	time	heartbeat_signals
0	0	0	0.991230
0	0	1	0.943533
0	0	2	0.764677
0	0	3	0.618571
0	0	4	0.379632
...	...	...	...
99999	99999	200	0.000000
99999	99999	201	0.000000
99999	99999	202	0.000000
99999	99999	203	0.000000
99999	99999	204	0.000000

20500000 rows × 3 columns

# 查看id为1的时间序列的集合
data_train[data_train["id"]==1]

	id	time	heartbeat_signals
1	1	0	0.971482
1	1	1	0.928969
1	1	2	0.572933
1	1	3	0.178457
1	1	4	0.122962
...	...	...	...
1	1	200	0.000000
1	1	201	0.000000
1	1	202	0.000000
1	1	203	0.000000
1	1	204	0.000000

205 rows × 3 columns

'''
# 通过调用库函数extract_features提取特征
from tsfresh import extract_features

# 特征提取
train_features = extract_features(data_train, column_id='id', column_sort='time')
train_features
'''

"\n# 通过调用库函数extract_features提取特征\nfrom tsfresh import extract_features\n\n# 特征提取\ntrain_features = extract_features(data_train, column_id='id', column_sort='time')\ntrain_features\n"

'''
from tsfresh.utilities.dataframe_functions import impute

# 去除抽取特征中的NaN值
impute(train_features)
'''

'\nfrom tsfresh.utilities.dataframe_functions import impute\n\n# 去除抽取特征中的NaN值\nimpute(train_features)\n'

'''
from tsfresh import select_features

# 按照特征和数据label之间的相关性进行特征选择
train_features_filtered = select_features(train_features, data_train_label)

train_features_filtered
'''

'\nfrom tsfresh import select_features\n\n# 按照特征和数据label之间的相关性进行特征选择\ntrain_features_filtered = select_features(train_features, data_train_label)\n\ntrain_features_filtered\n'

因为机器性能限制，我们未能运行出分类结果。
具体可以参考Task3 特征工程.md