Bayes classifier

Bayes classifier

For classification, we use 0-1 loss function to choosing classifier, which can be denoted as $\mathrm{L}(y,\hat y)$

If k-class is considered, our epxect prediction error is

$$\begin{aligned} \mathrm{EPE} = \mathrm{E}[L(G,\hat G(X))] =\mathrm{E}_X\mathrm{E}_{Y|X}[L(G,\hat G(X))|X] \end{aligned}$$
by minimizing pointwise,
$$\begin{aligned} \hat G(x) &=\arg\min \sum_{g=1}^k L(g, \hat G(X))\mathrm{p}(g|X=x)\\ &=\arg\min_{g\in\mathscr{G}} [1 - \mathrm{p}(g|X=x)]\\ &=\arg\max_{g\in\mathscr{G}}\mathrm{p}(g|X=x) \end{aligned}$$

So Bayes classifier chooses the class with max posterior probability.