[阅读笔记]Introduction to Boosted Trees

概念(要学什么)

回归树(CART)

回归树emsemble

模型

$$y_i = \sum_{k=1}^{K}f_k(x_i) , f \in F$ $

参数

GB(如何学习)

additve training(boosting)

$$L^{(t)}=\sum_{i=1}^n l(y_i, \hat{y}_{i}^{(t-1)} + f_t(x_i) ) + \Omega(f_t)$$
$$L^{(t)} \approx \sum_{i=1}^n [l(y_i, \hat{y}_{i}^{(t-1)}) + g_i f(x_i) + \frac {1} {2} h_i^2 f^2(x_i) ] + \Omega(f_t)$$
很关键的一点
$$g_i = \partial_{\hat{y}_{i}^{(t-1)}}l(y_i, \hat{y}_{i}^{(t-1)})$$
$$h_i = \partial^2_{\hat{y}_{i}^{(t-1)}}l(y_i, \hat{y}_{i}^{(t-1)})$$

$$\tilde{L}^{(t)} = \sum_{i=1}^n [g_i f(x_i) + \frac {1} {2} h_i^2 f^2(x_i) ] + \gamma T + \frac 1 2 \lambda \sum_{j=1}^T w_j^2$$

---

Rabit文档
https://github.com/dmlc/rabit/blob/master/doc/guide.md
allreduce和ps的对比
http://hunch.net/?p=151364
陈天奇关于rabit的说明
http://weibo.com/p/1001603801281637563132?pids=Pl_Official_CardMixFeedv6__4&feed_filter=2