爬虫

pycharm2018  IDLE

下载requests库 http://www.python-requests.org/en/master/

 

r = requests.get(url)

 

win平台 以管理员身份运行cmd，执行

pip install requests

pip install beautifulsoup4

IDLE  requests爬虫常用语句：

import requests
r = requests.get("http://203.91.44.4:32004/Login")
r.status_code
r.text
r.encoding
r.apparent_encoding
r.content

requests.request()
requests.get(url,params=None,**kwargs)
requests.head()
requests.post()
requests.put()
requests.patch()
requests.delete()

requests.ConnectionError
requests.HTTPError
requests.URLRquired
requests.TooManyRedirects
requests.Timeout
r.raise_for_status()

使用OCR识别验证码

pip install lxml

pip install tushare

pip install baidu-aip

pip install cssselect

 

Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> from aip import AipOcr
>>> APP_ID = '11760004 '
>>> API_KEY = 'AwmwVQxrDFE4eXDg07A0k6qc'
>>> SECRET_KEY = 'VdI1s4gjXNlPPm1Z1c5BCj871vn4cc7Z'
>>> client = AipOcr(APP_ID, API_KEY, SECRET_KEY)
>>> def get_file_content(filePath):
    with open(filePath, 'rb') as fp:
        return fp.read()

>>> def get_file_content(filePath):
    with open(filePath, 'rb') as fp:
        return fp.read()
image = get_file_content('F:\\GetValidateCode.jpg')
SyntaxError: invalid syntax
>>> def get_file_content(filePath):
    with open(filePath, 'rb') as fp:
        return fp.read()

>>> image = get_file_content('F:\\GetValidateCode.jpg')
>>> result = client.basicGeneral(image)
>>> options = {
        'detect_direction' : 'true',
        'language_type' : 'CHN_ENG',

}
>>> result = client.basicGeneral(image,options)
>>> print(result)
{'log_id': 291960611943065081, 'direction': 0, 'words_result_num': 1, 'words_result': [{'words': '25112'}]}

>>> for word in result['words_result']:
	print(word['words'])

	
25112
>>> 
for word in result['words_result']:
	valid = word['words']
	print(valid)

	
25112
>>>