Supervised machine learning methods

目录

1. Overview

2. Process

2.1 Overall process

2.2 How the models learns

3. Models

3.1 Linear Regression

3.1.1 Linear regression

3.1.2 Logistic Regression

3.1.3 Lasso and Ridge

3.2 Ensemble（Boosting）

3.2.1 XGBoost

3.2.2 CatBoost

3.2.3 LightGBM

3.2.4 Adaboost

3.2.5 Differences between boosting models

3.3 Ensemble (Bagging)

3.3.1 RandomForest

3.4 Others

3.4.1 Decision Tree

3.4.2 SVM

3.4.3 Nerual Network

---

1. Overview

Supervised machine learning is a machine learning method whose core idea is to train a model through the relationship between known inputs and corresponding outputs. In supervised learning, the model receives a training data set consisting of inputs and corresponding target outputs, and is trained by learning the mapping relationship between inputs and outputs. This means that the model is able to gradually improve its prediction accuracy on the input data by adjusting its internal parameters during training.

2. Process

2.1 Overall process

Here are some basic steps when using Supervised machine learning methods.

Data collection: Collect a training dataset containing input features and corresponding labels. The quality and diversity of data are critical to model performance.

Data preprocessing: Clean and preprocess data to ensure data consistency and availability. This may include missing value handling, data normalization, feature engineering, etc.

Data splitting: Divide the entire data set into a training set and a test set. The training set is used to train the model, and the test set is used to evaluate the model performance.

Select model architecture: Select an appropriate model architecture, such as decision tree, support vector machine, neural network, etc., based on the nature of the task and the characteristics of the data.

Model training: Use the training set to train the model. This involves adjusting the parameters of the model to minimize the loss function. The training process usually involves multiple iterations until the model converges.

Model evaluation: Use the test set to evaluate the model's performance on unseen data. Common evaluation indicators include accuracy, precision, recall, F1 score, etc. The specific indicators depend on the nature of the task.

Tuning the model: Based on the results of the evaluation, it may be necessary to adjust the model's hyperparameters, feature engineering, or other aspects to improve model performance.

2.2 How the models learns

Firstly, we split our dataset into training set and test set. In the training set, models should learn the mapping between the features and the target. Then, we use our models to predict target according to features in test set. Finally, by comparing the predicted target with the actual target, we can get how well the models fit to the data.

3. Models

I' ll introduce 12 different models which use supervised learning methods. All these models can help figure out regression or classification problems. 

3.1 Linear Regression

Regression models are used to model and predict continuous target variables. These models attempt to find the relationship between input features and target variables and are used to solve regression problems.

3.1.1 Linear regression

Linear models are a basic class of machine l