下载博客首页的博客列表

最新推荐文章于 2023-10-14 15:15:58 发布

weixin_30716141

最新推荐文章于 2023-10-14 15:15:58 发布

阅读量53

点赞数

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/y15821933792/p/7797197.html

本文介绍了一段用于抓取新浪博客文章列表的Python代码。该代码利用urllib库从指定URL读取网页内容，并通过字符串查找的方法解析出每篇文章的链接地址，随后下载这些文章到本地文件夹中。

url_con = urllib.urlopen('http://blog.sina.com.cn/s/articlelist_1193111400_0_1.html').read()
print 'con' ,url_con

url = ['']*40
i = 0
title = url_con.find(r'<a title=')

print "title",title
href = url_con.find(r'href=',title)
print "href",href

html = url_con.find(r'.html',href)
print "html",html


while title != -1 and href != -1 and html != -1 and i < 40:
    url[i] = url_con[href+6:html+5]
    print url[i]
    title = url_con.find(r'<a title=',html)
    
    href = url_con.find(r'href=',title)
    
    html = url_con.find(r'.html',href)
    
    filename = url[-26:]

    i = i + 1
j = 0
while j < 40:
    content = urllib.urlopen(url[j]).read()
    filename = url[j][-26:]
    open(r'blog/'+ filename,'w').write(content)
    j = j+ 1
    time.sleep(5)