Derek_Zhang_的专栏

1. Static Resource HTTP Response Headercache-control:public, max-age=30758400cf-cache-status:HITcf-ray:1afc29518836124f-HKGcontent-encoding:gzipcontent-type:text/cssdate:Wed, 28 Jan 2015 09:28:

When I try to get document from a website using jsoup, I got the error after seconds of stucking.java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Met

1. Open chrome and its developer tool2. Conduct the login operation in chrome, record the network intercourse.3. Analyze the HTTP Request and HTTP ResponseRequest HeadersPOST /member/xlogin.ph

www.freeformatter.com is very handy for web scraping stuff, atm I'm using the xpath-tester for validating my xpath expression.http://www.freeformatter.com/xpath-tester.html

scrapy: application framework for web scraping and crawlingbeautifulsoup: library for parsing HTMLmechanizelxml