Machine Learning -- algorithms performance measure (heart disease data)

最新推荐文章于 2024-05-26 09:59:09 发布
原创 最新推荐文章于 2024-05-26 09:59:09 发布 · 331 阅读 · 1 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文详细介绍了使用四种机器学习算法（SVM-RBF、SVM-POLY、Bagging、AdaBoost）对心脏病数据集进行预测分析的过程。通过构建混淆矩阵、计算F1分数、准确率、召回率、精确率和AUC等指标，对比了不同算法的性能，并绘制了ROC曲线以直观展示算法的分类效果。

OverView:

In this chapter, I’m gonna do performance stuffs for machine learning algorithms. From previous chapter, I have already built four model:
SVM-RBF, SVM-PLOY, Bagging, AdaBoost.

Step 1: Confusion Matrix

SVM-RBF (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

SVM-POLY (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

Bagging (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

Adaboost (without normalization and with normalization)

在这里插入图片描述

在这里插入图片描述

Step 2: Metrics

Based on the confusion matrix, I will calculate:

  1. F-measure score
  2. accuracy
  3. recall ( sensitivity, true positive rate )
  4. precision ( positive predictive value )
  5. AUC ( Area under the ROC curve )

–> This is a screen shot for all the metrics of each algorithm.

在这里插入图片描述

–> Put them into excel

在这里插入图片描述

–> Rank them to have an intuitive unstanding

在这里插入图片描述

Step 3: Plot ROC Curve

在这里插入图片描述



import pandas as pd
import numpy as np
from sklearn.preprocessing import OneHotEncoder
from sklearn import tree
from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn import svm
from sklearn.model_selection import StratifiedKFold, GridSearchCV
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.model_selection import cross_val_score
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix, f1_score, accuracy_score, recall_score, precision_score, roc_auc_score, \
    roc_curve





## load data
from plot_confusion_matrix import plot_confusion_matrix

trainSet = pd.read_csv("clevelandtrain.csv")
testSet = pd.read_csv("clevelandtest.csv")

xtrain = (trainSet.drop(["heartdisease::category|0|1"], axis=1)).iloc[:,:].values  # (152, 13)
ytrain = trainSet["heartdisease::category|0|1"].iloc[:].values                     # (152,)

xtest = (testSet.drop(["heartdisease::category|0|1"], axis=1)).iloc[:,:].values    # (145, 13)
ytest = testSet["heartdisease::category|0|1"].iloc[:].values                       # (145,)



## data preprocessing

 # without: one-hot-encoder:

# xTrain = xtrain
# yTrain = ytrain
# xTest = xtest
# yTest = ytest

  # one-hot-encoder: #9 (cp), #19 (restecg),  #41 (slope), #51 (thal)

xtrain_pre = trainSet.drop(["cp", "restecg", "slope", "thal", "heartdisease::category|0|1"], axis=1).iloc[:,:].values # (152, 9)
xtrain_cp = trainSet["cp"].iloc[:].values
xtrain_restecg = trainSet["restecg"].iloc[:].values
xtrain_slope = trainSet["slope"].iloc[:].values
xtrain_thal = trainSet["thal"].iloc[:].values

ohe1 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')
ohe2 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')
ohe3 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')
ohe4 = OneHotEncoder(sparse = False,categories='auto',handle_unknown='ignore')

xtrain_cp = ohe1.fit_transform(xtrain_cp.reshape(-1,1))                    # (152, 4)
xtrain_restecg = ohe2.fit_transform(xtrain_restecg.reshape(-1,1))          # (152, 3)
xtrain_slope = ohe3.fit_transform(xtrain_slope.reshape(-1,1))              # (152, 3)
xtrain_thal = ohe4.fit_transform(xtrain_thal.reshape(-1,1))                # (152, 3)


xTrain = np.hstack((xtrain_pre, xtrain_cp, xtrain_restecg, xtrain_slope, xtrain_thal))   # (152, 22)
yTrain = ytrain                                                                          # (152,)



xtest_pre = testSet.drop(["cp", "restecg", "slope", "thal", "heartdisease::category|0|1"], axis=1).iloc[:,:].values   # (145, 9)
xtest_cp = testSet["cp"].iloc[:].values
xtest_restecg = testSet["restecg"].iloc[:].values
xtest_slope = testSet["slope"].iloc[:].values
xtest_thal = testSet["thal"].iloc[:].values

xtest_cp = ohe1.transform(xtest_cp.reshape(-1,1))                 # (145, 4)
xtest_restecg = ohe2.transform(xtest_restecg.reshape(-1,1))       # (145, 3)
xtest_slope = ohe3.transform(xtest_slope.reshape(-1,1))           # (145, 3)
xtest_thal = ohe4.transform(xtest_thal.reshape(-1,1))             # (145, 3)

xTest = np.hstack((xtest_pre, xtest_cp, xtest_restecg, xtest_slope, xtest_thal))   # (145, 22)
yTest = ytest                                                                      # (145,)


print("-----------------------------------------------------------------")


class_names = np.array([1,0])


svcRBF = SVC(C=300.0,gamma=0.0001,kernel='rbf',probability=True)
svcRBF.fit(xTrain,yTrain)
svcRBFScore = svcRBF.score(xTest, yTest)
prediction_svcRBF = svcRBF.predict(xTest)



# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcRBF, classes=class_names, title='Confusion matrix svcRBF, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcRBF, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix svcRBF')
#
# plt.show()
#
#
# f1 = f1_score(yTest, prediction_svcRBF)
# acc1 = accuracy_score(yTest, prediction_svcRBF)
# rec1 = recall_score(yTest, prediction_svcRBF)
# pre1 = precision_score(yTest, prediction_svcRBF)
# auc1 = roc_auc_score(yTest, prediction_svcRBF)
# print(f1)
# print(acc1)
# print(rec1)
# print(pre1)
# print(auc1)
# print("-----------------------------------------------------------------")




svcPoly = SVC(C=1.0,degree = 8.666666,coef0=1.0,gamma = 'scale',max_iter=-1,kernel='poly',probability=True)
svcPoly.fit(xTrain,yTrain)
svcPolyScore = svcPoly.score(xTest, yTest)
prediction_svcPLOY = svcPoly.predict(xTest)

# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcPLOY, classes=class_names, title='Confusion matrix svcPLOY, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_svcPLOY, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix svcPLOY')
#
# plt.show()
#
# f2 = f1_score(yTest, prediction_svcPLOY)
# acc2 = accuracy_score(yTest, prediction_svcPLOY)
# rec2 = recall_score(yTest, prediction_svcPLOY)
# pre2 = precision_score(yTest, prediction_svcPLOY)
# auc2 = roc_auc_score(yTest, prediction_svcPLOY)
# print(f2)
# print(acc2)
# print(rec2)
# print(pre2)
# print(auc2)
# print("-----------------------------------------------------------------")



decisonTree = tree.DecisionTreeClassifier()
decisonTreeBagging = BaggingClassifier(decisonTree,max_samples=0.7, max_features=1.0)
decisonTreeBagging.fit(xTrain,yTrain)
Bagging_score = decisonTreeBagging.score(xTest,yTest)

prediction_Bagging = decisonTreeBagging.predict(xTest)



# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_Bagging, classes=class_names, title='Confusion matrix decisonTreeBagging, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_Bagging, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix decisonTreeBagging')
#
# plt.show()
#
# f3 = f1_score(yTest, prediction_Bagging)
# acc3 = accuracy_score(yTest, prediction_Bagging)
# rec3 = recall_score(yTest, prediction_Bagging)
# pre3 = precision_score(yTest, prediction_Bagging)
# auc3 = roc_auc_score(yTest, prediction_Bagging)
# print(f3)
# print(acc3)
# print(rec3)
# print(pre3)
# print(auc3)
# print("-----------------------------------------------------------------")



decisonTreeAda = AdaBoostClassifier(decisonTree,n_estimators=10,random_state=np.random.RandomState(1))
decisonTreeAda.fit(xTrain,yTrain)
AdaBoost_score = decisonTreeAda.score(xTest,yTest)

prediction_AdaBoost = decisonTreeAda.predict(xTest)


# # Plot non-normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_AdaBoost, classes=class_names, title='Confusion matrix AdaBoost, without normalization')
#
# # Plot normalized confusion matrix
# plot_confusion_matrix(yTest, prediction_AdaBoost, classes=class_names, normalize=True,
#                       title='Normalized confusion matrix AdaBoost')
#
# plt.show()
#
#
# f4 = f1_score(yTest, prediction_AdaBoost)
# acc4 = accuracy_score(yTest, prediction_AdaBoost)
# rec4 = recall_score(yTest, prediction_AdaBoost)
# pre4 = precision_score(yTest, prediction_AdaBoost)
# auc4 = roc_auc_score(yTest, prediction_AdaBoost)
# print(f4)
# print(acc4)
# print(rec4)
# print(pre4)
# print(auc4)
# print("-----------------------------------------------------------------")





## plot roc curve

prediction_prob_svmRBF = svcRBF.predict_proba(xTest)[:, 1]
prediction_prob_svmPOLY = svcPoly.predict_proba(xTest)[:, 1]
prediction_prob_Bagging = decisonTreeBagging.predict_proba(xTest)[:, 1]
prediction_prob_Adaboost = decisonTreeAda.predict_proba(xTest)[:, 1]

fpr_svmRBF, tpr_svmRBF, _ = roc_curve(yTest, prediction_prob_svmRBF)
fpr_svmPOLY, tpr_svmPOLY, _ = roc_curve(yTest, prediction_prob_svmPOLY)
fpr_Bagging, tpr_Bagging, _ = roc_curve(yTest, prediction_prob_Bagging)
fpr_Adaboost, tpr_Adaboost, _ = roc_curve(yTest, prediction_prob_Adaboost)

plt.figure(1)
plt.plot([0, 1], [0, 1], 'k--')

plt.plot(fpr_svmRBF, tpr_svmRBF, label='SVM-RBF')
plt.plot(fpr_svmPOLY, tpr_svmPOLY, label='SVM-POLY')
plt.plot(fpr_Bagging, tpr_Bagging, label='Bagging')
plt.plot(fpr_Adaboost, tpr_Adaboost, label='Adaboost')

plt.xlabel('False positive rate')
plt.ylabel('True positive rate')
plt.title('ROC curve')
plt.legend(loc='best')
plt.show()
Machine Learning Engineering Case Studies with Python notebook
AI天才研究院
08-06 1426
作者：禅与计算机程序设计艺术Machine learning engineering (MLE) is the process of developing machine learning systems that can perform tasks with high accuracy and efficiency at scale. MLE involves designing, building, testing, deploying, monitoring, and maintaining mach
使用深度学习神经网络从X射线图像检测肺炎
weixin_26749843的博客
10-07 2345
This blog is probably going to be in terms of coding the most technically driven that I have written so far, and it will be about a project using Deep Learning Neural Networks to classify a set of str...
周志华《Machine Learning》学习笔记（2）--性能度量
VAY-长跑
11-20 7783
本篇主要是对第二章剩余知识的理解，包括：性能度量、比较检验和偏差与方差，写到后面发现篇幅冗长读起来十分没有快感，故本篇主要知识点为性能度量。
Machine Learning Algorithms
08-16
Machine Learning Algorithms by Giuseppe Bonaccorso English | 24 July 2017 | ISBN: 1785889621 | ASIN: B072QBG11J | 360 Pages | AZW3 | 12.18 MB Build strong foundation for entering the world of Machine Learning and data science with the help of this comprehensive guide About This Book Get started in the field of Machine Learning with the help of this solid, concept-rich, yet highly practical guide. Your one-stop solution for everything that matters in mastering the whats and whys of Machine Learning algorithms and their implementation. Get a solid foundation for your entry into Machine Learning by strengthening your roots (algorithms) with this comprehensive guide. Who This Book Is For This book is for IT professionals who want to enter the field of data science and are very new to Machine Learning. Familiarity with languages such as R and Python will be invaluable here. What You Will Learn Acquaint yourself with important elements of Machine Learning Understand the feature selection and feature engineering process Assess performance and error trade-offs for Linear Regression Build a data model and understand how it works by using different types of algorithm Learn to tune the parameters of Support Vector machines Implement clusters to a dataset Explore the concept of Natural Processing Language and Recommendation Systems Create a ML architecture from scratch. In Detail As the amount of data continues to grow at an almost incomprehensible rate, being able to understand and process data is becoming a key differentiator for competitive organizations. Machine learning applications are everywhere, from self-driving cars, spam detection, document search, and trading strategies, to speech recognition. This makes machine learning well-suited to the present-day era of Big Data and Data Science. The main challenge is how to transform data into actionable knowledge. In this book you will learn all the important Machine Learning algorithms that are commonly used in the
Machine Learning：十大机器学习算法
茅坤宝骏氹的博客
04-17 6665
转载自 Machine Learning：十大机器学习算法摘要: - 机器学习算法分类：监督学习、无监督学习、强化学习 - 基本的机器学习算法：线性回归、支持向量机(SVM)、最近邻居(KNN)、逻辑回归、决策树、k平均、随机森林、朴素贝叶斯、降维、梯度增强机器学习算法大致可以分为三类：监督学习算法 (Supervised Algorithms）:在监督学习训练过程中，可以由训练数据集学到或建立一...
面试必备 | 机器学习这十大算法你确定会了吗？
zenRRan的博客
03-20 907
点击上方，选择星标或置顶，每天给你送干货！阅读大概需要11分钟跟随小博主，每天进步一丢丢选自Medium作者：garvitanand2机器之心编译参与：Geek AI、路本文介绍了 10...
Conceptual Challenges for Interpretable Machine Learning
weixin_42786150的博客
03-01 1694
Conceptual Challenges for Interpretable Machine Learning David S. Watson1 'Department of Statistical Science, University College London, London, UK Email for correspondence: david.watson@ucl.ac.uk §0 Abstract As machine learning has gradually entere..
ICCV2017论文摘要汇总
super_chicken的博客
04-18 7452
1. Globally-Optimal Inlier Set Maximisation for Simultaneous Camera Pose and Feature Correspondence Abstract: Estimating the 6-DoF pose of a camera from a single image relative to a pre-computed 3D p...
深度学习的时间序列分类
weixin_26713457的博客
09-03 3682
为什么要进行时间序列分类？(Why Time Series Classification?) First of all it’s important to underline why this problem is so important today, and therefore why it is very interesting to understand the role and the ...
机器学习：模型性能度量(performance measure)(待补充)
weixin_30302609的博客
07-19 450
机器学习：模型性能度量(performance measure)(待补充) 对学习器的泛化性能进行评估，不仅需要有效的实验估计方法，还需要有衡量模型泛化性能的评准指标，这就是性能度量。性能度量反应任务需求，对比不同模型能力时，使用不同性能度量能导致不同的评判结果。因此，模型的好坏是相对的，模型不仅取决于算法和数据，还绝对于任务需求。 1. 回...
周志华《Machine Learning》学习笔记 2 ——性能度量
zhuangV587的博客
06-11 558
       本篇主要是对第二章剩余知识的理解，包括：性能度量、比较检验 和 偏差与方差 。在上一篇中，我们解决了评估学习器泛化性能的方法，即用测试集的“测试误差”作为“泛化误差”的近似，当我们划分好 训练/测试集后，那如何计算“测试误差”呢？这就是性能度量，例如：均方差，错误率等，即“测试误差”的一个评价标准。有了评估方法和性能度量，就可以计算出学习器的“测试误差”，但由于“测试误差”受到很多因...
【Machine Learning系列】带你快速学习十大机器学习算法
热门推荐
zhangxia_的博客
04-10 2万+
机器学习算法的学习是一个长期的过程，需要不断地练习和实践才能够掌握。通过坚持不懈地学习和实践，你将能够快速掌握机器学习算法并应用于实际问题中。
Machine Learning Algorithms: Which One to Choose for Your Problem
xiangz_csdn的博客
10-28 944
Intuition of using different kinds of algorithms in different tasks
探索机器学习算法的宝藏：Machine Learning Algorithms
最新发布
gitblog_00032的博客
05-26 458
探索机器学习算法的宝藏：Machine Learning Algorithms???? 在这个信息爆炸的时代，数据成为新的石油，而机器学习则像炼金术一样从中提炼出宝贵的洞见。【Machine Learning Algorithms】仓库就是这样一个宝库，它整理并分类了几乎所有的机器学习和深度学习算法，旨在帮助开发者和学者更深入地理解这些神奇的算法。 项目介绍 这是一个由Sahith02维护的开源项目，以...
Machine Learning-算法汇总介绍
Pysamlam的博客
04-25 378
之前一个一个讲还是有些零散而且更新的速度也慢，所以我还是打算先整理一波全部的算法，再逐一展开介绍比较好，原文来自Jason Brownlee 的文章《A Tour of ...
吴恩达机器学习笔记一（Machine learning algorithms）
weixin_43294620的博客
12-29 925
Machine learning algorithms （纯粹促进自己学习整理） 1. Machine learning algorithms: Main two types: Supervised learning 监督学习 In supervised learning, the idea is that we’re going to teach the computer how to do something. Unsupervised learning 无监督学习 In unsupervised
机器学习模型的性能指标
weixin_26752765的博客
09-04 3185
There are various metrics that we can use to evaluate the performance of ML algorithms, classification as well as regression algorithms. We must carefully choose the metrics for evaluating ML performa...
Pattern Recognition and Machine Learning-01-Preface
06-08
The preface of the book "Pattern Recognition and Machine Learning" by Christopher Bishop provides an overview of the field of pattern recognition and machine learning, and the motivation behind writing the book. In the preface, Bishop highlights the importance of pattern recognition and machine learning in a wide range of applications, including computer vision, speech recognition, natural language processing, and robotics. He also discusses the challenges and opportunities in this field, such as dealing with high-dimensional data and designing effective algorithms. Bishop explains that the book is intended to be a comprehensive introduction to the field of pattern recognition and machine learning, suitable for both students and researchers. The book covers a range of topics, including Bayesian decision theory, linear models for regression and classification, neural networks, kernel methods, clustering, and dimensionality reduction. Bishop emphasizes the importance of a probabilistic approach, and how it can be used to unify many of the different techniques used in pattern recognition and machine learning. The preface concludes by acknowledging the contributions of many researchers in the field, and expressing the hope that the book will help to inspire and educate the next generation of researchers in pattern recognition and machine learning.
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
表情包 代码片
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值