python读网页解析内容写入文件

最新推荐文章于 2022-08-01 16:01:28 发布

whymm

最新推荐文章于 2022-08-01 16:01:28 发布

阅读量579

点赞数 1

分类专栏：学习笔记

本文链接：https://blog.youkuaiyun.com/whymm/article/details/50589054

版权

学习笔记专栏收录该内容

11 篇文章

订阅专栏

快速入门 http://developer.51cto.com/art/201207/347006.htm

注意编译指令有一处错误

python -O -m py_compile hello.py

工程发布

http://blog.youkuaiyun.com/u011465933/article/details/9567375

下面是相关代码

import sys
import urllib
import random
import time
def GetNowTime():
    return time.strftime("%Y-%m-%d %H:%M:%S",time.localtime(time.time()))
def GetDayTime():
    return time.strftime("%Y-%m-%d",time.localtime(time.time()))
def WriteFile(ip, dat):
	strCurTime = GetDayTime()
	output = open(strCurTime + "_" + ip, 'a+')
	output.writelines(dat)
	output.close()
def GetHtml(url):
	response=urllib.urlopen(url)
	html = response.read()
	return html
devIp = raw_input("ip: ") 
print devIp
baseUrl = 'http://' + devIp + '/goform/formIPQAM?type=3&cmd=1&language=1&gbeNo=0&ran='
while True:
	url = baseUrl + str(random.uniform(0, 1)) + '&rowIndex=0'
	html = GetHtml(url);
	arrTsk =  html.split('<*1*>')
	nIndx = len(arrTsk) - 1
	totalTskNum = 0
	if nIndx > 0:
		strTail = arrTsk[-1]
		strTail = strTail.split('<*2*>')
		strTail = strTail[-1]
		strTail = strTail.split('<')
		strTail = strTail[0]
		totalTskNum = int(strTail)
	WriteFile(devIp, "*****" + GetNowTime() + "****" + str(totalTskNum) + "********************\n")
	if nIndx > 0:
		WriteFile(devIp, html)
		while totalTskNum > nIndx:
			url = baseUrl + str(random.uniform(0, 1)) + '&rowIndex=' + str(nIndx)
			html = GetHtml(url)
			WriteFile(devIp, html)
			nIndx += 16
	print totalTskNum
	WriteFile(devIp, "*****end****************************************\n")
	time.sleep(3600)