Lecture5-3Effective number of hypotheses

Dichotomies

$\mathcal{H} = \{\textrm{hypothesis } h: \mathcal{X} \rightarrow \{x,o\}\}$

Call:

$$h(\mathbf{x_1, x_2, ... , x_N}) = (h(\mathbf{x_1}), h(\mathbf{x_2}), ... , h(\mathbf{x_N})) \in \{ x, o\}^N$$

• A dichotomy: hypothesis limited to the eyes of $\mathbf{x_1, x_2, ..., x_N}$
• $\mathcal{H}(\mathbf{x_1,x_2,...,x_N})$: all dichotomies implemented by $\mathcal{H}$ on the $\mathbf{x_1,x_2,...,x_N}$

$|\mathcal{H}|$: Candidate for replacing $M$

Growth Function

• $\mathcal{H}(\mathbf{x_1,x_2,...,x_N})$ depends ont inputs $\mathbf{x_1,x_2,...,x_N}$
  • growth function: remove dependencies by taking the MAX of all $\mathcal{H}(\mathbf{x_1,x_2,...,x_N})$
  $$m_\mathcal{H} (N)= \max_{{\mathbf{x_1,x_2,...,x_N}}\in \mathcal{X}} \mathcal{H}(\mathbf{x_1,x_2,...,x_N})$$

  Lines in 2D

  $N$	$m_{\mathcal{H}}(N)$
  1	2
  2	4
  3	$\max(...,6,8) = 8$
  4	$14 < 2^N$

  • Finite, upper bounded by $2^N$

  Growth function for positive rays(1D perceptron)

  

  Growth function for positive intervals

  

  Growth function for convex sets