Mata.Gao的博客

在ubuntu 14.04上安装virtualenv，virtualenvwrapper时，出现："no module named upload""virtualenvwrapper ERROR:root:Error parsing"更新好setuptools，就可以解决：sudo pip install --upgrade setuptools安装 爬虫Scrap...

1.同步class MysqlPipeline(object): """ 采用同步的机制写入mysql """ def __init__(self): self.conn = pymysql.connect(host="127.0.0.1", user="root", password="ts123456", db="art_sc

import hashlibdef get_md5(url): if isinstance(url, str): # 判断是否是unicode编码 url = url.encode('utf-8') m = hashlib.md5() m.update(url) return m.hexdigest()if __name__ == "_...

1.用户代理池downloader middleware实现随机更换User-Agent.fake-useragent库:up to date simple useragent faker with real world database.github-搜索fake-useragenthttps://fake-useragent.herokuapp.com/browsers/0.1.5以上这个ur...

1.pymysql安装pip install pymysql(测试连接：import pymysql，如编译未出错，即表示pymysql安装成功)2.插入操作import pymysql db=pymysql.connect(host="127.0.0.1",user="root",password="ts123456",db="blog_spider",port=3307) # 使...

1.创建数据库create database db_name default charset utf8 collate utf8_unicode_ci;2.安装pymysql项目的__init__.py下：import pymysqlpymysql.install_as_MySQLdb()错误一：django.core.exceptions.ImproperlyCo...