一次失败的QThread多线程加载Webpage.load()页面的尝试

问题

前段时间在用PyQt4写网页加载部分时一开始用的从QWebView.load()(QWebFrame.load())，但由于这样会卡住GUI，于是换了QNetworkAccessManager模块来访问网络，返回内容用setcontent()传递给QWebView：

http://blog.youkuaiyun.com/chroming/article/details/51832244

当时还觉得解决了一个挺有意义的问题，毕竟搜索GUI卡住的问题确实有不少人因为load()导致GUI界面卡住。

由于QNetworkAccessManager是比较底层的访问网络模块，不少东西要自己处理。比如某些网址HTML中声明的编码与response header中声明的编码不同导致的网页显示乱码，某些网页无法显示等问题。对于乱码问题我还通过读取response header解决了一部分，但对于无法显示的问题却没找到原因。后来测试QWebFrame.load()，发现这些问题都不存在。于是考虑使用多线程加载网页，既能解决各种兼容性问题，又不会导致GUI卡住。

寻找解决办法

虽然Python自己有多线程模块，但因为程序里用了很多Qt模块，这里也考虑直接用Qt的自带多线程模块，QThread。搜了资料，写出了独立的多线程模块：

# -*- coding: utf-8 -*-
from PyQt4 import QtCore


# load加载页面线程
class loadUrl(QtCore.QThread):
    def __init__(self, qurl, qwebframe, parent=None):
        super(loadUrl, self).__init__(parent)
        self.qurl = qurl
        self.qwebframe = qwebframe

    def run(self):
        self.qwebframe.load(self.qurl)

在主代码中调用多线程模块：

class test():
    ...
    def testfun():
        ...
        from multiThread import loadUrl
        self.multiget = loadUrl(QUrl(input_url), self.webView.page().mainFrame())
        self.multiget.start()

结果出现了错误：

QObject: Cannot create children for a parent that is in a different thread

网上查到类似的情况：http://stackoverflow.com/questions/7143088/how-do-you-set-up-multiple-multithreaded-qwebviews-in-pyqt

回答如下：

There are multiple issues with your question and code:

• You are talking about QWebFrame, but are actually passing a QWebView to your worker(s). Since this is a QWidget, it belongs to the main (GUI) thread and should not be modified by other threads.
• One QWebView / QWebFrame can only load one URL at a time, so you cannot share it across multiple workers.
• QWebFrame.load() loads data asynchronously, i.e. a call to load() returns immediately and there will be no data to read yet. You will have to wait for the loadFinished() signal to be emitted before you can access the data.
• Since the actual loading is done by the networking layer of the operating system, and the *load()*method does not block, there is no need to run it in a separate thread in the first place. Why do you claim that this should be faster – it makes no sense.
• Since you want to load hundreds of URLs in parallel (or about 10, you are mentioning both in the same sentence), are you sure that you want to use QWebFrame, which is a presentation class? Do you actually want to render the HTML or are you just interested in the data retrieved?

其中第一个issue回答了为什么会出现这个报错：主线程中的QWebFrame不能用子线程修改。但相比这条我更在意第三第四条：QWebFrame.load()是异步加载页面的，并不会阻塞主线程，不需要用子线程去加载页面。这就奇怪了，之前我用的时候明明是因为加载页面导致GUI卡住的。于是回去看自己的代码，发现了一点：

self.webView.load(QUrl(input_url))
app.restoreOverrideCursor()

于是随手把app.restoreOverrideCursor() 改到load之前。结果发现不卡了！一直以来困扰我的竟然是由于这个！

总结

这个问题让我花了一些冤枉时间来处理网页兼容性，结果发现只是一个小小的失误导致的。虽然之前在搜索网页卡住的问题时也看到有人说load()是异步加载，但由于自己的程序确实卡住了这个事实，也没深究为什么异步还会卡住。今后在处理问题时需要更加仔细。

参考资料：

1. 《PyQt 分离UI主线程与工作线程》http://blog.youkuaiyun.com/Mr_Zing/article/details/46945011
2. 《How do you set up multiple multithreaded QWebViews in PyQt?》http://stackoverflow.com/questions/7143088/how-do-you-set-up-multiple-multithreaded-qwebviews-in-pyqt