python抓取数据例子

最新推荐文章于 2022-05-24 14:03:08 发布

原创最新推荐文章于 2022-05-24 14:03:08 发布 · 904 阅读

0 ·

CC 4.0 BY-SA版权

python 专栏收录该内容

74 篇文章

订阅专栏

本示例展示了如何使用Python的urllib和re模块从指定网址抓取网页内容，并通过正则表达式提取所需的文本部分。该过程包括打开URL、读取HTML内容、解析并提取特定的div标签内的文本。

#coding=utf-8

#-*- coding: UTF-8 -*-  
import re
import urllib

def main():
    urls = ['http://www.forbeschina.com/review/201301/0022393.shtml',
    'http://www.forbeschina.com/review/201301/0022393_2.shtml',
    'http://www.forbeschina.com/review/201301/0022393_3.shtml',
    'http://www.forbeschina.com/review/201301/0022393_4.shtml',
    'http://www.forbeschina.com/review/201301/0022393_5.shtml',
    'http://www.forbeschina.com/review/201301/0022393_6.shtml']

    for url in urls:
        f = urllib.urlopen(url)
        html = f.read()
        content = re.compile(u'<div class="p_detail">.*?</div>', re.DOTALL) 
        style = content.search(html.decode('utf8'))
        if style:
            html = style.group(0)
            print 'Found it'
        else:
            print 'Not found.'
        para = re.sub('<[^>]*>', '', html);
        print para.encode('utf8') 


if __name__ == "__main__":
    main()