集成剪枝分类算法的Bagging与Adaboost示例

Bagging (Bootstrap Aggregation)

Pruning Classification is one of the simplest classification algorithms. It works just like if-then. However, when aggregating a lot of prunnings we are able to create a powerful classifier.

The process of Bagging based on pruning is really simple but not trivial:

1. For $j=1,\dots ,b$,
   2. Pick up $m$ samples from a sample set with $n$ samples $\{(x_i,y_i){\}}_{i=1}^n$. Repeating is permitted. Then we get a new sample set.
   3. Train the pruning classifier ${\psi }_j$ with the new sample set.
2. For all of the pruning classifiers { ψj}j=1b\{\psi_{j}\}_{j=1}^{b}{ ψj​}j=1b​, calculate their average and get $f$: $$f(x)\leftarrow \frac{1}{b}\sum _{j=1}^b{\psi }_j(x)$$

n=50; x=randn(n,2); 
y=2*(x(:,1)>x(:,2))-1;
b=5000; a=50; Y=zeros(a,a);
X0=linspace(-3,3,a);
[X(:,:,1), X(:,:,