lgb xgb gbdt

XGBoost

https://github.com/dmlc/xgboost

XGBoost的原理

http://djjowfy.com/2017/08/01/XGBoost%E7%9A%84%E5%8E%9F%E7%90%86/

陈天奇讲解  XGBoost 与 Boosted Tree

http://www.52cs.org/?p=429

XGBoost：参数解释

https://blog.youkuaiyun.com/zc02051126/article/details/46711047

XGBoost的demo

https://github.com/dmlc/xgboost/tree/master/demo/guide-python

https://github.com/dmlc/xgboost/blob/master/doc/python/python_intro.md

调参指南

https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/

LightGBM

https://blog.youkuaiyun.com/niaolianjiulin/article/details/76584785

https://zhuanlan.zhihu.com/p/25308051

http://mp.weixin.qq.com/s?__biz=MzA3MzI4MjgzMw==&mid=2650719786&idx=3&sn=ab1c5a77237dc4b2ee5ae12c7a68ff87&chksm=871b0254b06c8b42d5a4fdf3327f7284c9ffbe72fe7911301d368b157024b32923d88401c2a8&scene=0&open_source=weibo_search

https://github.com/Microsoft/LightGBM

https://github.com/Microsoft/LightGBM/tree/master/python-package   官方的python封装

https://github.com/ArdalanM/pyLightGBM   非官方的python封装

这个东西被视为比xgboost更好的GBDT开源实现，同时微软出品。

对比细节可参考

http://blog.youkuaiyun.com/chary8088/article/details/54316708

还有人专门写了代码PK

https://github.com/tks0123456789/XGBoost_vs_LightGBM

特色：

https://github.com/Microsoft/LightGBM/wiki/Features

配置和调参

https://github.com/Microsoft/LightGBM/wiki/Configuration

python范例

https://github.com/Microsoft/LightGBM/blob/master/docs/Python-intro.md

相对xgboost来说，有以下特点:

更好的准备率

消耗更少的内存

支持并行学习

GBDT

GBDT是使用最多的模型之一，特意搜集了GBDT的一些博文

https://www.cnblogs.com/pinard/p/6140514.html

https://www.cnblogs.com/ModifyRong/p/7744987.html

特别注意GBDT容易过拟合，新的回归树不断拟合之前的残差，在训练集上的误差可能逐渐变为0，导致过拟合。GBDT也有相应避免过拟合的正则化方法。正则化的方式有好几种

http://blog.youkuaiyun.com/thriving_fcl/article/details/51255513

GBDT的调参和调用

https://www.cnblogs.com/DjangoBlog/p/6201663.html

http://scikit-learn.org/stable/auto_examples/ensemble/plot_gradient_boosting_regression.html

xgboost是对GBDT做了一些改进，所以xgboost需要和GBDT一起比较，详细看下一篇博文