urllib2 爬虫 打印页面内容,部分无法显示

在尝试爬取起点小说网的内容时,使用urllib2进行网页抓取,发现打印的页面内容存在部分丢失的情况。代码执行后,一些数据未能正常显示。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

爬了一下起点小说网的内容,结果发现打印页面内容时丢失了部分内容

这个是我的代码

import re
import urllib2
user_agent ='Mozilla/5.0 (Windows NT 10.0; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0' 
headers = { 'User-Agent':user_agent}
url="https://www.qidian.com/finish"
reque=urllib2.Request(url,headers=headers)
respo=urllib2.urlopen(reque)
page=respo.read()
print page

爬下来无法显示的内容:

        </div>
    </div>
</div>
        <div class="main-content-wrap fl" data-l1="5">
            
<div class="tool-bar cf">
    <div class="view-mode" id="view-mode">
        <a id="img-text" class="iconfont act" href="//www.qidian.com/finish?action=hidden&orderId=&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B54"></a><em>|</em><a id="only-text" class="iconfont  " href="//www.qidian.com/finish?action=hidden&orderId=&page=1&style=2&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B55"></a>
    </div>
    <div class="select-wrap">
        
        <a data-id="" class=act href="//www.qidian.com/finish?action=hidden&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B62">人气排序<cite class="iconfont"></cite></a>
        <a data-id="6"  href="//www.qidian.com/finish?action=hidden&orderId=6&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B49">完本时间</a>
        <a data-id="11"  href="//www.qidian.com/finish?action=hidden&orderId=11&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B80">总收藏</a>
        <a data-id="3"  href="//www.qidian.com/finish?action=hidden&orderId=3&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B52">总字数</a>
        <div id="vipClicks" class="lbf-combobox vipClicks">
            
            <a href="javascript:" class="lbf-button lbf-combobox-label " hidefocus="true">
                <span class="lbf-combobox-caption">会员点击</span>
                <span class="lbf-combobox-icon"> </span>
            </a>
        </div>
        
        <div id="recomm" class="lbf-combobox">
            <a href="javascript:" class="lbf-button lbf-combobox-label " hidefocus="true">
                <span class="lbf-combobox-caption">推荐票</span>
                <span class="lbf-combobox-icon"> </span>
            </a>
        </div>
        <em>|</em>
                                <span class="lbf-checkbox get-new-book " data-eid="qd_B53">
                                    <input id="only" type="checkbox" />
                                </span>
        <label for="only" data-eid="qd_B53">只看3个月内新书</label>
        <div class="sort-count count-info" id="sort-count">
            <div class="count-text">共<span>44433</span>本相关作品
            </div>
            <div class="yiwen" id="yiwen">
                <div class="iconfont"></div>
            </div>
            <i>|</i>
        </div>
    </div>
</div>

            
<div class="all-book-list">
    
        <div class="book-img-text">
            <ul class="all-img-list cf">
                
                <li data-rid="1">
                <div class="book-img-box">
                    <a href="//book.qidian.com/info/1003438608" data-bid="1003438608" data-eid="qd_B57" target="_blank"><img src="//qidian.qpic.cn/qdbimg/349573/1003438608/150"></a>
                </div>
                <div class="book-mid-info">
                    <h4><a href="//book.qidian.com/info/1003438608" target="_blank" data-eid="qd_B58" data-bid="1003438608">废土崛起</a></h4>
                    <p class="author">
                        <img src="//qidian.gtimg.com/qd/images/ico/user.f22d3.png"><a class="name" href="//my.qidian.com/author/2962092" data-eid="qd_B59" target="_blank">通吃道人.QD</a><em>|</em><a href="//www.qidian.com/kehuan" target="_blank" data-eid="qd_B60">科幻</a><i>·</i><a class="go-sub-type" data-typeid="9" data-subtypeid="251" href="javascript:" data-eid="qd_B61">时空穿梭</a><em>|</em><span >完本</span>
                        
                    </p>
                    <p class="intro">
                        开局一条狗,装备全靠捡。带着新手福利杀进末世将临的世界。发起狠来,核弹都朝自己头上砸!这是废土之王的崛起之路。验证群:669035131。
                    </p>
                    <p class="update"><span >441.12万字</span>
                        
                    </p>
                </div>
                </li>
                
                <li data-rid="2">
                <div class="book-img-box">
                    <a href="//book.qidian.com/info/1003543215" data-bid="1003543215" data-eid="qd_B57" target="_blank"><img src="//qidian.qpic.cn/qdbimg/349573/1003543215/150"></a>
                </div>
                <div class="book-mid-info">
                    <h4><a href="//book.qidian.com/info/1003543215" target="_blank" data-eid="qd_B58" data-bid="1003543215">恐怖广播</a></h4>
                    <p class="author">
                        <img src="//qidian.gtimg.com/qd/images/ico/user.f22d3.png"><a class="name" href="//my.qidian.com/author/3780268" data-eid="qd_B59" target="_blank">纯洁滴小龙</a><em>|</em><a href="//www.qidian.com/qihuan" target="_blank" data-eid="qd_B60">奇幻</a><i>·</i><a class="go-sub-type" data-typeid="1" data-subtypeid="202" href="javascript:" data-eid="qd_B61">黑暗幻想</a><em>|</em><span >完本</span>
                        
                    </p>
                    <p class="intro">
                        “听众朋友们,欢迎收听【恐怖广播】,希望您能够好好享受我们节目陪伴你的时光,好了,下面,请您转过头,看一看您的身后,是不是有一张人脸正在微笑着看着你?”——————起点
                    </p>
                    <p class="update"><span >339.19万字</span>
                        
                    </p>
                </div>
                </li>
                
                <li data-rid="3">
                <div class="book-img-box">
                    <a href="//book.qidian.com/info/2597043" data-bid="2597043" data-eid="qd_B57" target="_blank"><img src="//qidian.qpic.cn/qdbimg/349573/2597043/150"></a>
                </div>
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值