老司机的Blog

整理于北京理工大学-嵩天老师的爬虫mooc 包括了爬虫程序和对中文字符居中问题的解决 程序：  import requests from bs4 import BeautifulSoup import bs4 def getHTMLText(url): //获取HTML的内容 try: r = requests.get(url,timeout = 30) ...

整理自北京理工大学嵩天老师的mooc 1 京东商品页面爬取 import requests url = "https://item.jd.com/7652143.html" try: r = requests.get(url) #获取网页内容 r.raise_for_status() #如果返回状态不是200会出现一个异常 r.encoding = r.a...

#-0 创建虚拟环境 ``` mkdir -p python/scrapy virtualenv --no-site-packages -p python3 python/scrapy ```     #如果出现：Command /home/tomblack/python/scrapy/bin/python3 - setuptools pkg_resources pip wheel failed ...