爬了一下起点小说网的内容,结果发现打印页面内容时丢失了部分内容
这个是我的代码
import re
import urllib2
user_agent ='Mozilla/5.0 (Windows NT 10.0; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0'
headers = { 'User-Agent':user_agent}
url="https://www.qidian.com/finish"
reque=urllib2.Request(url,headers=headers)
respo=urllib2.urlopen(reque)
page=respo.read()
print page
爬下来无法显示的内容:
</div>
</div>
</div>
<div class="main-content-wrap fl" data-l1="5">
<div class="tool-bar cf">
<div class="view-mode" id="view-mode">
<a id="img-text" class="iconfont act" href="//www.qidian.com/finish?action=hidden&orderId=&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B54"></a><em>|</em><a id="only-text" class="iconfont " href="//www.qidian.com/finish?action=hidden&orderId=&page=1&style=2&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B55"></a>
</div>
<div class="select-wrap">
<a data-id="" class=act href="//www.qidian.com/finish?action=hidden&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B62">人气排序<cite class="iconfont"></cite></a>
<a data-id="6" href="//www.qidian.com/finish?action=hidden&orderId=6&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B49">完本时间</a>
<a data-id="11" href="//www.qidian.com/finish?action=hidden&orderId=11&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B80">总收藏</a>
<a data-id="3" href="//www.qidian.com/finish?action=hidden&orderId=3&page=1&style=1&pageSize=20&siteid=1&pubflag=0&hiddenField=2" data-eid="qd_B52">总字数</a>
<div id="vipClicks" class="lbf-combobox vipClicks">
<a href="javascript:" class="lbf-button lbf-combobox-label " hidefocus="true">
<span class="lbf-combobox-caption">会员点击</span>
<span class="lbf-combobox-icon"> </span>
</a>
</div>
<div id="recomm" class="lbf-combobox">
<a href="javascript:" class="lbf-button lbf-combobox-label " hidefocus="true">
<span class="lbf-combobox-caption">推荐票</span>
<span class="lbf-combobox-icon"> </span>
</a>
</div>
<em>|</em>
<span class="lbf-checkbox get-new-book " data-eid="qd_B53">
<input id="only" type="checkbox" />
</span>
<label for="only" data-eid="qd_B53">只看3个月内新书</label>
<div class="sort-count count-info" id="sort-count">
<div class="count-text">共<span>44433</span>本相关作品
</div>
<div class="yiwen" id="yiwen">
<div class="iconfont"></div>
</div>
<i>|</i>
</div>
</div>
</div>
<div class="all-book-list">
<div class="book-img-text">
<ul class="all-img-list cf">
<li data-rid="1">
<div class="book-img-box">
<a href="//book.qidian.com/info/1003438608" data-bid="1003438608" data-eid="qd_B57" target="_blank"><img src="//qidian.qpic.cn/qdbimg/349573/1003438608/150"></a>
</div>
<div class="book-mid-info">
<h4><a href="//book.qidian.com/info/1003438608" target="_blank" data-eid="qd_B58" data-bid="1003438608">废土崛起</a></h4>
<p class="author">
<img src="//qidian.gtimg.com/qd/images/ico/user.f22d3.png"><a class="name" href="//my.qidian.com/author/2962092" data-eid="qd_B59" target="_blank">通吃道人.QD</a><em>|</em><a href="//www.qidian.com/kehuan" target="_blank" data-eid="qd_B60">科幻</a><i>·</i><a class="go-sub-type" data-typeid="9" data-subtypeid="251" href="javascript:" data-eid="qd_B61">时空穿梭</a><em>|</em><span >完本</span>
</p>
<p class="intro">
开局一条狗,装备全靠捡。带着新手福利杀进末世将临的世界。发起狠来,核弹都朝自己头上砸!这是废土之王的崛起之路。验证群:669035131。
</p>
<p class="update"><span >441.12万字</span>
</p>
</div>
</li>
<li data-rid="2">
<div class="book-img-box">
<a href="//book.qidian.com/info/1003543215" data-bid="1003543215" data-eid="qd_B57" target="_blank"><img src="//qidian.qpic.cn/qdbimg/349573/1003543215/150"></a>
</div>
<div class="book-mid-info">
<h4><a href="//book.qidian.com/info/1003543215" target="_blank" data-eid="qd_B58" data-bid="1003543215">恐怖广播</a></h4>
<p class="author">
<img src="//qidian.gtimg.com/qd/images/ico/user.f22d3.png"><a class="name" href="//my.qidian.com/author/3780268" data-eid="qd_B59" target="_blank">纯洁滴小龙</a><em>|</em><a href="//www.qidian.com/qihuan" target="_blank" data-eid="qd_B60">奇幻</a><i>·</i><a class="go-sub-type" data-typeid="1" data-subtypeid="202" href="javascript:" data-eid="qd_B61">黑暗幻想</a><em>|</em><span >完本</span>
</p>
<p class="intro">
“听众朋友们,欢迎收听【恐怖广播】,希望您能够好好享受我们节目陪伴你的时光,好了,下面,请您转过头,看一看您的身后,是不是有一张人脸正在微笑着看着你?”——————起点
</p>
<p class="update"><span >339.19万字</span>
</p>
</div>
</li>
<li data-rid="3">
<div class="book-img-box">
<a href="//book.qidian.com/info/2597043" data-bid="2597043" data-eid="qd_B57" target="_blank"><img src="//qidian.qpic.cn/qdbimg/349573/2597043/150"></a>
</div>