96、Machine Learning Model Selection and Regularization Techniques

Machine Learning Model Selection and Regularization Techniques

1. Model Selection Methods

1.1 Best Subset Selection

Best subset selection is a method used to pick the best model with the help of a subset of predictors. It fits the OLS equation to every combination of (p) predictors. In total, there are (2^p) possible combinations.

The algorithm steps are as follows:
1. Start with a model having no predictors and denote it as (m_0).
2. Let (x = 1,2,3, \cdots, p) predictors:
- Fit all the models for the (k) predictors.
- Select the best model among those created by (k) predictors. We can use RSS (Residual Sum of Squares) or (R^2) to determine the best model.
3. Finally, choose the best model among the best - selected