u012329294的专栏

UnicodeDecodeError: 'utf-8' codec can't decode byte 报错。原因：爬取的网站编码和decode默认的utf8编码不同，例如：gbk解决办法：1、忽略掉utf8编码def parse(self, response): print(response.body.decode("utf8", "ignore")) return但这样做的...

class ZfSpider(RedisSpider): handle_httpstatus_list = [404]在自己的代码中，加入handle_httpstatus_list = [404]就可以了。

折腾了好久，终于解决csv存入中文是乱码的情况，直接见代码def writeData(self, movieList): with open('81.csv', 'w', encoding='utf_8_sig', newline='') as f: writer = csv.DictWriter(f, fieldnames=['content']) wri...

pop：以下实例展示了 pop()函数的使用方法：实例#!/usr/bin/python3#coding=utf-8 list1 = ['Google', 'Runoob', 'Taobao']list_pop=list1.pop(1)print "删除的项为 :", list_popprint "列表现在为 : ", list1以上实例输出结果如下：删除的项为 : Runoob列表现在为 : ...

now = datetime.datetime.now()addda = datetime.timedelta(days=1)newday = now+daysprint newday.strftime('%Y-%m-%d %H:%M:%S')

1、str -> array arr = strval.split('')2、array -> strstrval = ''.join(arr)

1、str -> dateimport datetimedetester = ‘2017-01-01'date = datetime.datetime.strptime(detester,’%Y-%m-%d')12342、date -> strimport datetimedate = datetime.now()detester = date.strftime(‘%Y-...

原理：使用urlparse，先解析整个url，然后使用split（'/'）方式构造为词典，下面就可以直接读取from urllib.parse import urlparseurl = 'http://www.abc.cn/jfjbmap/content/2016-01/03/node_2.htm'urldic = urlparse(url)print(urldic.path)p...

def parse(self, response): selector = Selector(response) data = selector.xpath("//div[@class='header']/div/p")[1] header = data.xpath("string(.)").extract() print(header) info = '...

class ZufangDownloaderMiddleware(object): # Not all methods need to be defined. If a method is not defined, # scrapy acts as if the downloader middleware does not modify the # passed objec...

解决方法：def parse(self, response): print(response.body.decode('unicode_escape'))核心就是解码使用unicode_escape

网络原文：https://blog.youkuaiyun.com/dassh/article/details/50961362http://localhost:8888/   安装证书，如果提示没有证书（8888是你在Tool->Fiddler Option->Connections设置的Fiddler监听端口）cd "d:\Program Files\Fiddler"makecert.exe -...