爬虫资料总结

爬虫资料

---

爬虫实例

https://github.com/facert/awesome-spider

https://github.com/lining0806/PythonSpiderNotes#4-%E5%AF%B9%E4%BA%8E%E6%96%AD%E7%BA%BF%E9%87%8D%E8%BF%9E

http://www.cnblogs.com/xinyangsdut/default.html?page=2

防采集应用

https://github.com/xchaoinfo/fuck-login

https://cuiqingcai.com/3256.html

https://github.com/luyishisi/Anti-Anti-Spider

https://www.jianshu.com/p/ebf2e5b34aad

https://www.jianshu.com/p/e75ee27ac9a1

https://github.com/brandonxiang/example-requests

爬虫、反爬虫理解

http://wangxin123.com/2016/12/21/%E7%88%AC%E8%99%AB%E3%80%81%E5%8F%8D%E7%88%AC%E8%99%AB%E3%80%81%E5%8F%8D%E5%8F%8D%E7%88%AC%E8%99%AB/## 标题

反击爬虫，前端工程师的脑洞可以有多大？

https://imweb.io/topic/595b7161d6ca6b4f0ac71f05

XML/HTML/JSON——数据抓取过程中不得不知的几个概念

https://juejin.im/entry/5a0be0c4f265da43231a824c

Python 爬虫 —— 获取js渲染的内容

https://blog.youkuaiyun.com/and_w/article/details/73611325

beautifulsoup

https://www.crummy.com/software/BeautifulSoup/bs4/doc/index.zh.html#id10

requests

http://cn.python-requests.org/zh_CN/latest/user/quickstart.html#id2

ip服务器地址查询

http://ip.chinaz.com/

http测试

https://httpbin.org/

UserAgent（提供）

https://www.jianshu.com/p/da6a44d0791e

http代理（提供）

https://www.xicidaili.com/nn/1
http://www.site-digger.com/html/articles/20110516/proxieslist.html

python练习

https://github.com/aosabook/500lines/blob/master/README.md

https://github.com/Show-Me-the-Code/show-me-the-code