np.mean(y_test == y_predict)的简要说明

最新推荐文章于 2024-01-11 23:50:44 发布

原创最新推荐文章于 2024-01-11 23:50:44 发布 · 5k 阅读

11 ·

CC 4.0 BY-SA版权

文章标签：

#numpy

小细节专栏收录该内容

1 篇文章

订阅专栏

本文探讨了Python中数组的比较操作，展示了如何使用numpy库进行数组元素级别的比较，并计算比较结果的平均值，以此衡量两个数组的相似度。

部署运行你感兴趣的模型镜像

# 例 1
x = [1, 0, 1, 1, 1, 1]
y = [0, 0, 0, 0, 0, 1]
print(x == y)
结果：
False
# 例 2
x = np.array([1, 0, 1, 1, 1, 1])
y = np.array([0, 0, 0, 0, 0, 1])
print(x == y)
结果：
[False True False False False True]
# 例 3
x = np.array([1, 0, 1, 1, 1, 1])
y = np.array([0, 0, 0, 0, 0, 1])
print("{:.2f}".format(np.mean(x == y)))
结果：
0.33

说明：
1、 x == y表示两个数组中的值相同时，输出True；否则输出False
2、例3对例2中结果取平均值，其中True=1,False=0;

您可能感兴趣的与本文相关的镜像

Python3.10

Conda

Python

Python 是一种高级、解释型、通用的编程语言，以其简洁易读的语法而闻名，适用于广泛的应用，包括Web开发、数据分析、人工智能和自动化脚本

确定要放弃本次机会？

福利倒计时

: :

立减 ¥

普通VIP年卡可用

立即使用

Dear_D

关注关注

11
点赞
踩
11

收藏

觉得还不错? 一键收藏
1
评论
分享

复制链接

分享到 QQ

分享到新浪微博

扫一扫
举报

举报

专栏目录

【机器学习之集成学习算法-01】| 基于重抽样自举法的集成学习方法：Bagging袋装法和Random Forests随机森林算法原理及PM2.5浓度回归预测实例分析

一伦明悦的博客

05-18

2303

集成学习是一种机器学习技术，其核心思想是通过结合多个基本模型的预测结果来提高整体性能。常见的集成学习方法包括bagging、boosting和stacking。

python学习：使用KNN最邻近分类算法进行iris数据集分析

weixin_49539269的博客

11-14

2860

概述：数据集的分析可以简单的分为两步： 1、根据已有的特征数据建立模型 2、在未来收集到新的特征数据时，可以根据建立的模型来判断新的数据属于哪一类而iris数据集包括150朵、三类（setosa、versicolor、virginica）鸢尾花的花萼和花瓣长度宽度的数据。那么对应上面的两步，我们要做的就是： 1、根据已有的鸢尾花花萼和花瓣的长度与宽度数据建立模型 2、在未来收集到新的鸢尾花数据时，可以根据模型来判断所收集的是哪种类型的鸢尾花(setosa，versicolor，virgi

1 条评论您还未登录，请先登录后发表或查看评论

1 条评论

徐yida 2021.05.27
不错。点赞

【python机器学习基础教程】(一)

Algernon98的博客

04-27

1779

边看书边复现代码，已更新第一部分——监督学习算法

Python机器学习基础教程鸢尾花分类

qq_41890177的博客

09-26

9919

一、数据集分析鸢尾花数据集保存在sklearn.datasets模块中，我们可以用load_iris函数加载数据，这个函数返回的iris对象是一个Bunch对象，与字典相似，包括键和值此处打印出iris数据集中的键值 from sklearn.datasets import load_iris iris_dataset=load_iris() print("keys of iris_...

机器学习——Logistic回归

diosamaehjs的博客

12-04

1183

线性模型线性模型是一种假设输入变量（自变量）和输出变量（因变量）之间存在线性关系的模型。在最简单的形式中，线性模型可以表示为一个等式，其中输出变量( y )是输入变量( x1, x2, ..., xn )的加权和，再加上一个常数项（也称为截距项）(数学上，这可以表达为：在回归分析中，线性模型用于预测连续的输出变量。例如，在房价预测问题中，我们可能会使用房屋的面积、位置、房龄等作为输入变量来预测房屋的价格。线性回归模型会尝试找到最佳的参数值（()系数），使得模型预测的价格与实际价格之间的差异最小。

np.mean

weixin_48225415的博客

12-06

460

不过我在学习机器学习的时候，发现定义np.mean(y.pred==y.test)可以求pred等于test的个数占所有个数的百分比。mean主要是求平均值。

numpy.mean()的几种用法

诺坎普奇迹的博客

04-22

2万+

import numpy as np x = np.array([1,2,3,4,5]) y = np.array([0,2,3,4,6]) z = np.array([[1,2],[3,4]])#二维数组 np.mean(x==y)#返回条件成立的占比 Out[5]: 0.59999999999999998 np.mean(x)#均值 Out[6]: 3.0np.mean(z) Out[10]:

193样本正负比2比三特征有53个，标签是乳腺癌患者her2是否表达大部分特征是心率变异的值，比如S1_Mean RR (ms) S1_SDNN (ms) S1_Mean HR (bpm) S1_SD HR (bpm) S1_Min HR (bpm) S1_Max HR (bpm)S1_RP_Lmean (beats) S1_RP_Lmax (beats) S1_RP_REC (%) S1_RP_DET (%) S1_RP_ShanEn S1_MSE_1 S1_MSE_2 S1_MSE_3 S1_MSE_4 S1_MSE_5 等,还有一些生理指标如年龄和bmi下面是我的数据操作和模型代码。写一个论文形式的模型搭建内容（包括使用了什么，为什么这么使用对比其他这个方法好在哪里，以文本形式输出你的回答）data = pd.read_excel('C:/lydata/test4.xlsx') X = data.drop('HER2_G', axis=1) y = data['HER2_G'] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=60) kf = KFold(n_splits=5, shuffle=True, random_state=91) accuracy_scores = [] precision_scores = [] recall_scores = [] f1_scores = [] auc_scores = [] total_confusion_matrix = np.zeros((len(np.unique(y_train)), len(np.unique(y_train))), dtype=int) pca = PCA(n_components=14) pipeline = Pipeline([ ('scaler', StandardScaler()), ('smote', SMOTE(k_neighbors=4, sampling_strategy=0.94, random_state=42)), ('pca', pca), ('gb', GradientBoostingClassifier( learning_rate=0.02, n_estimators=90, subsample=0.75, min_samples_split=5, min_samples_leaf=1, max_depth=6, random_state=42, warm_start=True, tol=0.0001, ccp_alpha=0, max_features=12, )) ]) for train_index, val_index in kf.split(X_train): X_train_fold, X_val = X_train.iloc[train_index], X_train.iloc[val_index] y_train_fold, y_val = y_train.iloc[train_index], y_train.iloc[val_index] pipeline.fit(X_train_fold, y_train_fold) y_pred = pipeline.predict(X_val) y_proba = pipeline.predict_proba(X_val)[:, 1] accuracy_scores.append(accuracy_score(y_val, y_pred)) precision_scores.append(precision_score(y_val, y_pred)) recall_scores.append(recall_score(y_val, y_pred)) f1_scores.append(f1_score(y_val, y_pred)) auc_scores.append(roc_auc_score(y_val, y_proba)) cm = confusion_matrix(y_val, y_pred) total_confusion_matrix += cm accuracy = np.mean(accuracy_scores) precision = np.mean(precision_scores) recall = np.mean(recall_scores) f1 = np.mean(f1_scores) auc = np.mean(auc_scores) print("Gradient Boosting 参数：") print(pipeline.named_steps['gb'].get_params()) print(f"Gradient Boosting 平均 accuracy: {accuracy:.2f}") print(f"Gradient Boosting 平均 precision: {precision:.2f}") print(f"Gradient Boosting 平均 recall: {recall:.2f}") print(f"Gradient Boosting 平均 F1 score: {f1:.2f}") print(f"Gradient Boosting 平均 AUC score: {auc:.2f}") print("综合混淆矩阵：") print(total_confusion_matrix) pipeline.fit(X_train, y_train) y_test_pred = pipeline.predict(X_test) y_test_proba = pipeline.predict_proba(X_test)[:, 1] accuracy_test = accuracy_score(y_test, y_test_pred) precision_test = precision_score(y_test, y_test_pred) recall_test = recall_score(y_test, y_test_pred) f1_test = f1_score(y_test, y_test_pred) auc_test = roc_auc_score(y_test, y_test_proba) print(f"测试集 accuracy: {accuracy_test:.2f}") print(f"测试集 precision: {precision_test:.2f}") print(f"测试集 recall: {recall_test:.2f}") print(f"测试集 F1 score: {f1_test:.2f}") print(f"测试集 AUC score: {auc_test:.2f}")

03-27

用户可能没有提到的点包括为什么选择K折交叉验证而不是留出法，以及为什么测试集比例是0.2，stratify参数的作用，这些可能需要简要解释。另外，PCA选择14个成分的原因，是否通过方差分析确定，可能需要假设或说明。 ...

请基于下面的框架，写一段代码：数据准备阶段首先需要收集与剪切力相关的特征变量以及对应的标签（即实际测量到的剪切力）。这些数据可以来源于实验记录或者仿真模拟结果。 Python import pandas as pd from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler # 假设我们有一个CSV文件包含了输入参数和输出剪切力的数据集 data = pd.read_csv('shear_force_data.csv') X = data.drop(columns=['ShearForce']) # 特征列 y = data['ShearForce'] # 目标值 # 将数据划分为训练集和测试集 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) scaler = StandardScaler() X_train_scaled = scaler.fit_transform(X_train) X_test_scaled = scaler.transform(X_test) 构建机器学习模型这里选用随机森林作为基础模型之一，因为它具有较强的泛化能力和鲁棒性，在处理复杂关系方面表现良好2。 Python from sklearn.ensemble import RandomForestRegressor from sklearn.metrics import mean_squared_error model = RandomForestRegressor(n_estimators=100, random_state=42) model.fit(X_train_scaled, y_train) predictions = model.predict(X_test_scaled) mse = mean_squared_error(y_test, predictions) print(f'Mean Squared Error: {mse}') 上述代码展示了完整的流程：加载数据、预处理、划分数据集、标准化数值范围、定义模型架构及其超参数设置最后评估性能指标均方误差(MSE)3。

03-14

例如，y_pred = model.predict(X_test_scaled)，然后mse = mean_squared_error(y_test, y_pred)。现在需要考虑如何将这些步骤整合成一个连贯的示例代码。可能需要先导入必要的库，比如pandas、sklearn的相关模块。...

算法及准确率： Min Distance（最小距离法）：0.8000 KNN（k=1，最近邻算法）：0.9280 Simple Neural Network（简单神经网络）：0.850分析我的代码import numpy as np import matplotlib.pyplot as plt from tensorflow.keras.datasets import mnist from sklearn.model_selection import train_test_split from sklearn.neighbors import KNeighborsClassifier from sklearn.svm import SVC from sklearn.metrics import accuracy_score, confusion_matrix from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D from tensorflow.keras.utils import to_categorical import seaborn as sns from docx import Document from docx.shared import Inches # 1. 数据准备 def load_and_sample_data(train_samples=6000, test_samples=1000): (train_images, train_labels), (test_images, test_labels) = mnist.load_data() # 合并数据集后随机抽取 all_images = np.concatenate((train_images, test_images)) all_labels = np.concatenate((train_labels, test_labels)) # 随机抽取样本 X_train, X_test, y_train, y_test = train_test_split( all_images, all_labels, train_size=train_samples, test_size=test_samples, random_state=42, stratify=all_labels ) # 归一化处理 X_train = X_train.astype('float32') / 255 X_test = X_test.astype('float32') / 255 # 为CNN添加通道维度 X_train_cnn = X_train.reshape(-1, 28, 28, 1) X_test_cnn = X_test.reshape(-1, 28, 28, 1) return X_train, X_test, y_train, y_test, X_train_cnn, X_test_cnn # 2. 最小距离分类器 def min_distance_classifier(X_train, y_train, X_test): class_centers = [] for digit in range(10): class_samples = X_train[y_train == digit] class_center = np.mean(class_samples, axis=0) class_centers.append(class_center) predictions = [] for sample in X_test: distances = [np.linalg.norm(sample - center) for center in class_centers] predictions.append(np.argmin(distances)) return np.array(predictions) # 3. KNN分类器 def knn_classifier(X_train, y_train, X_test, k_values=[1, 3, 5]): results = {} for k in k_values: knn = KNeighborsClassifier(n_neighbors=k) knn.fit(X_train.reshape(len(X_train), -1), y_train) pred = knn.predict(X_test.reshape(len(X_test), -1)) results[f'KNN (k={k})'] = pred return results # 4. 神经网络分类器 def neural_network_classifier(X_train, y_train, X_test): # 简单全连接网络 model = Sequential([ Flatten(input_shape=(28, 28)), Dense(128, activation='relu'), Dense(64, activation='relu'), Dense(10, activation='softmax') ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) # 减少训练轮数从10到3 model.fit(X_train, y_train, epochs=3, batch_size=32, verbose=0) nn_pred = np.argmax(model.predict(X_test), axis=1) return nn_pred, model # 5. CNN分类器 def cnn_classifier(X_train, y_train, X_test): # 转换为分类需要的格式 y_train_cat = to_categorical(y_train, 10) model = Sequential([ Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)), MaxPooling2D((2, 2)), Conv2D(64, (3, 3), activation='relu'), MaxPooling2D((2, 2)), Flatten(), Dense(128, activation='relu'), Dense(10, activation='softmax') ]) model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) # 减少训练轮数从10到3 model.fit(X_train, y_train_cat, epochs=3, batch_size=32, verbose=0) cnn_pred = np.argmax(model.predict(X_test), axis=1) return cnn_pred, model # 6. SVM分类器 def svm_classifier(X_train, y_train, X_test): svm = SVC(kernel='rbf', C=10, gamma=0.001) svm.fit(X_train.reshape(len(X_train), -1), y_train) return svm.predict(X_test.reshape(len(X_test), -1)) # 7. 结果可视化 def plot_results(y_true, predictions, model_names): plt.figure(figsize=(15, 10)) for i, (name, pred) in enumerate(predictions.items(), 1): acc = accuracy_score(y_true, pred) cm = confusion_matrix(y_true, pred) plt.subplot(2, 3, i) sns.heatmap(cm, annot=True, fmt='d', cmap='Blues') plt.title(f'{name}\nAccuracy: {acc:.4f}') plt.xlabel('Predicted') plt.ylabel('True') plt.tight_layout() plt.savefig('confusion_matrices.png') return 'confusion_matrices.png' # 8. 生成Word文档 def generate_report(results, img_path): doc = Document() # 标题 doc.add_heading('MNIST数据集分类算法对比报告', 0) # 算法介绍 doc.add_heading('一、算法介绍', 1) doc.add_paragraph('1. 最小距离法：计算每个类别的中心点（平均值），测试样本被分类到距离最近的类中心。') doc.add_paragraph('2. K近邻法(KNN)：根据最近的K个邻居的多数投票决定分类。测试了K=1,3,5三种参数。') doc.add_paragraph('3. 神经网络：使用包含两个隐藏层的全连接网络(128->64神经元)。') doc.add_paragraph('4. 卷积神经网络(CNN)：使用两个卷积层(32和64滤波器)和两个全连接层。') doc.add_paragraph('5. 支持向量机(SVM)：使用RBF核函数，C=10, gamma=0.001。') # 结果分析 doc.add_heading('二、结果分析', 1) doc.add_paragraph('各算法在1000个测试样本上的准确率：') # 创建表格 table = doc.add_table(rows=1, cols=2) table.style = 'LightShading' hdr_cells = table.rows[0].cells hdr_cells[0].text = '算法' hdr_cells[1].text = '准确率' for name, acc in results.items(): row_cells = table.add_row().cells row_cells[0].text = name row_cells[1].text = f'{acc:.4f}' doc.add_paragraph('\n关键观察：') doc.add_paragraph('- CNN和SVM表现最佳，展示了处理图像数据的强大能力') doc.add_paragraph('- KNN性能随K值增加略有下降，可能是噪声影响') doc.add_paragraph('- 最小距离法作为基线方法，性能明显低于其他方法') # 混淆矩阵 doc.add_heading('三、分类结果可视化', 1) doc.add_picture(img_path, width=Inches(6)) doc.add_paragraph('图1: 各算法混淆矩阵对比') # 保存文档 doc.save('MNIST_Classification_Report.docx') # 简化版主函数，确保能快速完成所有步骤 def main(): try: print("开始运行MNIST分类比较程序(精简版)...") # 加载数据 print("1. 加载和采样数据...") # 进一步减少样本数量以加快速度 X_train, X_test, y_train, y_test, X_train_cnn, X_test_cnn = load_and_sample_data(train_samples=3000, test_samples=500) print(f" 训练样本: {len(X_train)}, 测试样本: {len(X_test)}") # 只运行部分分类器以节省时间 print("2. 运行最小距离分类器...") min_dist_pred = min_distance_classifier(X_train, y_train, X_test) print(" 最小距离分类器完成") print("3. 运行KNN分类器(k=1)...") knn = KNeighborsClassifier(n_neighbors=1) knn.fit(X_train.reshape(len(X_train), -1), y_train) knn_pred = knn.predict(X_test.reshape(len(X_test), -1)) print(" KNN分类器完成") print("4. 运行简单神经网络分类器...") # 使用更简单的神经网络 model = Sequential([ Flatten(input_shape=(28, 28)), Dense(64, activation='relu'), Dense(10, activation='softmax') ]) model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) # 只训练1轮 model.fit(X_train, y_train, epochs=1, batch_size=32, verbose=0) nn_pred = np.argmax(model.predict(X_test), axis=1) print(" 神经网络分类器完成") # 整合结果 print("5. 整合结果...") all_predictions = { 'Min Distance': min_dist_pred, 'KNN (k=1)': knn_pred, 'Simple Neural Network': nn_pred } # 计算准确率 print("6. 计算准确率...") accuracy_results = {} for name, pred in all_predictions.items(): accuracy_results[name] = accuracy_score(y_test, pred) print(f" {name}: {accuracy_results[name]:.4f}") # 可视化 print("7. 生成混淆矩阵图...") img_path = plot_results(y_test, all_predictions, accuracy_results) print(f" 混淆矩阵图已保存到: {img_path}") # 生成报告 print("8. 生成Word报告...") generate_report(accuracy_results, img_path) print("报告已生成：MNIST_Classification_Report.docx") except Exception as e: print(f"程序运行出错: {e}") import traceback traceback.print_exc() if __name__ == "__main__": main()

12-08

# 假设train_X, train_y, test_X, test_y已加载 class MinDistanceClassifier: def fit(self, X, y): self.classes = np.unique(y) self.centroids = [] for c in self.classes: # 计算每个类的中心 self....

12-12

rmse = np.sqrt(mean_squared_error(y_true, y_pred)) rpd = np.std(y_true) / rmse return r2, mae, rpd, rmse def perform_plsr_analysis(file_path): # 读取文件 data = pd.read_csv(file_path) # 根据...

仅阅读、讲解以下代码 """ The Codes in this file are used to classify Human Activity using Channel State Information. The deep learning architecture used here is Bidirectional LSTM stacked with One Attention Layer. Author: https://github.com/ludlows 2019-12 """ import numpy as np import tensorflow as tf import glob import os import csv def merge_csi_label(csifile, labelfile, win_len=1000, thrshd=0.6, step=200): """ Merge CSV files into a Numpy Array X, csi amplitude feature Returns Numpy Array X, Shape(Num, Win_Len, 90) Args: csifile : str, csv file containing CSI data labelfile: str, csv fiel with activity label win_len : integer, window length thrshd : float, determine if an activity is strong enough inside a window step : integer, sliding window by step """ activity = [] with open(labelfile, 'r') as labelf: reader = csv.reader(labelf) for line in reader: label = line[0] if label == 'NoActivity': activity.append(0) else: activity.append(1) activity = np.array(activity) csi = [] with open(csifile, 'r') as csif: reader = csv.reader(csif) for line in reader: line_array = np.array([float(v) for v in line]) # extract the amplitude only line_array = line_array[1:91] csi.append(line_array[np.newaxis,...]) csi = np.concatenate(csi, axis=0) assert(csi.shape[0] == activity.shape[0]) # screen the data with a window index = 0 feature = [] while index + win_len <= csi.shape[0]: cur_activity = activity[index:index+win_len] if np.sum(cur_activity) < thrshd * win_len: index += step continue cur_feature = np.zeros((1, win_len, 90)) cur_feature[0] = csi[index:index+win_len, :] feature.append(cur_feature) index += step return np.concatenate(feature, axis=0) def extract_csi_by_label(raw_folder, label, labels, save=False, win_len=1000, thrshd=0.6, step=200): """ Returns all the samples (X,y) of "label" in the entire dataset Args: raw_foler: The path of Dataset folder label : str, could be one of labels labels : list of str, ['bed', 'fall', 'pickup', 'run', 'sitdown', 'standup', 'walk'] save : boolean, choose whether save the numpy array win_len : integer, window length thrshd : float, determine if an activity is strong enough inside a window step : integer, sliding window by step """ print('Starting Extract CSI for Label {}'.format(label)) label = label.lower() if not label in labels: raise ValueError("The label {} should be among 'bed','fall','pickup','run','sitdown','standup','walk'".format(labels)) data_path_pattern = os.path.join(raw_folder, 'input_*' + label + '*.csv') input_csv_files = sorted(glob.glob(data_path_pattern)) annot_csv_files = [os.path.basename(fname).replace('input_', 'annotation_') for fname in input_csv_files] annot_csv_files = [os.path.join(raw_folder, fname) for fname in annot_csv_files] feature = [] index = 0 for csi_file, label_file in zip(input_csv_files, annot_csv_files): index += 1 if not os.path.exists(label_file): print('Warning! Label File {} doesn\'t exist.'.format(label_file)) continue feature.append(merge_csi_label(csi_file, label_file, win_len=win_len, thrshd=thrshd, step=step)) print('Finished {:.2f}% for Label {}'.format(index / len(input_csv_files) * 100,label)) feat_arr = np.concatenate(feature, axis=0) if save: np.savez_compressed("X_{}_win_{}_thrshd_{}percent_step_{}.npz".format( label, win_len, int(thrshd*100), step), feat_arr) # one hot feat_label = np.zeros((feat_arr.shape[0], len(labels))) feat_label[:, labels.index(label)] = 1 return feat_arr, feat_label def train_valid_split(numpy_tuple, train_portion=0.9, seed=379): """ Returns Train and Valid Datset with the format of (x_train, y_train, x_valid, y_valid), where x_train and y_train are shuffled randomly. Args: numpy_tuple : tuple of numpy array: (x_bed, x_fall, x_pickup, x_run, x_sitdown, x_standup, x_walk) train_portion: float, range (0,1) seed : random seed """ np.random.seed(seed=seed) x_train = [] x_valid = [] y_valid = [] y_train = [] for i, x_arr in enumerate(numpy_tuple): index = np.random.permutation([i for i in range(x_arr.shape[0])]) split_len = int(train_portion * x_arr.shape[0]) x_train.append(x_arr[index[:split_len], ...]) tmpy = np.zeros((split_len,7)) tmpy[:, i] = 1 y_train.append(tmpy) x_valid.append(x_arr[index[split_len:],...]) tmpy = np.zeros((x_arr.shape[0]-split_len,7)) tmpy[:, i] = 1 y_valid.append(tmpy) x_train = np.concatenate(x_train, axis=0) y_train = np.concatenate(y_train, axis=0) x_valid = np.concatenate(x_valid, axis=0) y_valid = np.concatenate(y_valid, axis=0) index = np.random.permutation([i for i in range(x_train.shape[0])]) x_train = x_train[index, ...] y_train = y_train[index, ...] return x_train, y_train, x_valid, y_valid def extract_csi(raw_folder, labels, save=False, win_len=1000, thrshd=0.6, step=200): """ Return List of Array in the format of [X_label1, y_label1, X_label2, y_label2, .... X_Label7, y_label7] Args: raw_folder: the folder path of raw CSI csv files, input_* annotation_* labels : all the labels existing in the folder save : boolean, choose whether save the numpy array win_len : integer, window length thrshd : float, determine if an activity is strong enough inside a window step : integer, sliding window by step """ ans = [] for label in labels: feature_arr, label_arr = extract_csi_by_label(raw_folder, label, labels, save, win_len, thrshd, step) ans.append(feature_arr) ans.append(label_arr) return tuple(ans) class AttenLayer(tf.keras.layers.Layer): """ Attention Layers used to Compute Weighted Features along Time axis Args: num_state : number of hidden Attention state 2019-12, https://github.com/ludlows """ def __init__(self, num_state, **kw): super(AttenLayer, self).__init__(**kw) self.num_state = num_state def build(self, input_shape): self.kernel = self.add_weight('kernel', shape=[input_shape[-1], self.num_state]) self.bias = self.add_weight('bias', shape=[self.num_state]) self.prob_kernel = self.add_weight('prob_kernel', shape=[self.num_state]) def call(self, input_tensor): atten_state = tf.tanh(tf.tensordot(input_tensor, self.kernel, axes=1) + self.bias) logits = tf.tensordot(atten_state, self.prob_kernel, axes=1) prob = tf.nn.softmax(logits) weighted_feature = tf.reduce_sum(tf.multiply(input_tensor, tf.expand_dims(prob, -1)), axis=1) return weighted_feature # for saving the model def get_config(self): config = super().get_config().copy() config.update({ 'num_state': self.num_state,}) return config class CSIModelConfig: """ class for Human Activity Recognition ("bed", "fall", "pickup", "run", "sitdown", "standup", "walk") Using CSI (Channel State Information) Specifically, the author here wants to classify Human Activity using Channel State Information. The deep learning architecture used here is Bidirectional LSTM stacked with One Attention Layer. 2019-12, https://github.com/ludlows Args: win_len : integer (1000 default) window length for batching sequence step : integer (200 default) sliding window by this step thrshd : float (0.6 default) used to check if the activity is intensive inside a window downsample: integer >=1 (2 default) downsample along the time axis """ def __init__(self, win_len=1000, step=200, thrshd=0.6, downsample=2): self._win_len = win_len self._step = step self._thrshd = thrshd self._labels = ("bed", "fall", "pickup", "run", "sitdown", "standup", "walk") self._downsample = downsample def preprocessing(self, raw_folder, save=False): """ Returns the Numpy Array for training within the format of (X_lable1, y_label1, ...., X_label7, y_label7) Args: raw_folder: the folder containing raw CSI save : choose if save the numpy array """ numpy_tuple = extract_csi(raw_folder, self._labels, save, self._win_len, self._thrshd, self._step) if self._downsample > 1: return tuple([v[:, ::self._downsample,...] if i%2 ==0 else v for i, v in enumerate(numpy_tuple)]) return numpy_tuple def load_csi_data_from_files(self, np_files): """ Returns the Numpy Array for training within the format of (X_lable1, y_label1, ...., X_label7, y_label7) Args: np_files: ('x_bed.npz', 'x_fall.npz', 'x_pickup.npz', 'x_run.npz', 'x_sitdown.npz', 'x_standup.npz', 'x_walk.npz') """ if len(np_files) != 7: raise ValueError('There should be 7 numpy files for bed, fall, pickup, run, sitdown, standup, walk.') x = [np.load(f)['arr_0'] for f in np_files] if self._downsample > 1: x = [arr[:,::self._downsample, :] for arr in x] y = [np.zeros((arr.shape[0], len(self._labels))) for arr in x] numpy_list = [] for i in range(len(self._labels)): y[i][:,i] = 1 numpy_list.append(x[i]) numpy_list.append(y[i]) return tuple(numpy_list) def build_model(self, n_unit_lstm=200, n_unit_atten=400): """ Returns the Tensorflow Model which uses AttenLayer """ if self._downsample > 1: length = len(np.ones((self._win_len,))[::self._downsample]) x_in = tf.keras.Input(shape=(length, 90)) else: x_in = tf.keras.Input(shape=(self._win_len, 90)) x_tensor = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units=n_unit_lstm, return_sequences=True))(x_in) x_tensor = AttenLayer(n_unit_atten)(x_tensor) pred = tf.keras.layers.Dense(len(self._labels), activation='softmax')(x_tensor) model = tf.keras.Model(inputs=x_in, outputs=pred) return model @staticmethod def load_model(hdf5path): """ Returns the Tensorflow Model for AttenLayer Args: hdf5path: str, the model file path """ model = tf.keras.models.load_model(hdf5path, custom_objects={'AttenLayer':AttenLayer}) return model if __name__ == "__main__": import sys if len(sys.argv) != 2: print("Error! Correct Command: python3 csimodel.py Dataset_folder_path") raw_data_foler = sys.argv[1] # preprocessing cfg = CSIModelConfig(win_len=1000, step=200, thrshd=0.6, downsample=2) numpy_tuple = cfg.preprocessing('Dataset/Data/', save=True) # load previous saved numpy files, ignore this if you haven't saved numpy array to files before # numpy_tuple = cfg.load_csi_data_from_files(('x_bed.npz', 'x_fall.npz', 'x_pickup.npz', 'x_run.npz', 'x_sitdown.npz', 'x_standup.npz', 'x_walk.npz')) x_bed, y_bed, x_fall, y_fall, x_pickup, y_pickup, x_run, y_run, x_sitdown, y_sitdown, x_standup, y_standup, x_walk, y_walk = numpy_tuple x_train, y_train, x_valid, y_valid = train_valid_split( (x_bed, x_fall, x_pickup, x_run, x_sitdown, x_standup, x_walk), train_portion=0.9, seed=379) # parameters for Deep Learning Model model = cfg.build_model(n_unit_lstm=200, n_unit_atten=400) # train model.compile( optimizer=tf.keras.optimizers.Adam(learning_rate=1e-4), loss='categorical_crossentropy', metrics=['accuracy']) model.summary() model.fit( x_train, y_train, batch_size=128, epochs=60, validation_data=(x_valid, y_valid), callbacks=[ tf.keras.callbacks.ModelCheckpoint('best_atten.hdf5', monitor='val_accuracy', save_best_only=True, save_weights_only=False) ]) # load the best model model = cfg.load_model('best_atten.hdf5') y_pred = model.predict(x_valid) from sklearn.metrics import confusion_matrix print(confusion_matrix(np.argmax(y_valid, axis=1), np.argmax(y_pred, axis=1)))