爬虫超时报错socket.timeout: timed out/NameError: name ‘socket‘ is not defined

原创已于 2024-01-25 15:58:45 修改 · 9.2k 阅读

5 ·

CC 4.0 BY-SA版权

文章标签：

#python

于 2021-06-29 09:10:07 首次发布

Python学习私人笔记专栏收录该内容

64 篇文章

订阅专栏

本文介绍了解决Python中使用urllib库进行网络请求时遇到的超时问题及NameError异常，通过引入socket库并正确使用try/except语句来捕获异常，实现了对超时情况的有效处理。

问题一：socket.timeout: timed out

源代码：

import urllib.request#获取一个get请求

import urllib.parse   #获取一个pos请求

import urllib.error
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'}
data1 = urllib.request.Request("http://httpbin.org/post", headers=headers)

response = urllib.request.Request("http://httpbin.org/get", headers=headers)
response1 = urllib.request.urlopen(response, timeout=0.01)
# response1 = urllib.request.urlopen(response)
print(response1.read().decode("utf-8"))

异常运行结果：

Traceback (most recent call last):
  File "D:\PycharmProjects\pythonProject\douban\test1\testurllib.py", line 53, in <module>
    response1 = urllib.request.urlopen(response, timeout=0.01)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 517, in open
    response = self._open(req, data)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 494, in _call_chain
    result = func(*args)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1375, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1350, in do_open
    r = h.getresponse()
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1345, in getresponse
    response.begin()
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\socket.py", line 704, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

Process finished with exit code 1

问题二：NameError: name ‘socket’ is not defined

源代码：


import urllib.request#获取一个get请求

import urllib.parse   #获取一个pos请求
import urllib.error
try:
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'}
    # data1 = urllib.request.Request("http://httpbin.org/post", headers=headers)

    response = urllib.request.Request("http://httpbin.org/get", headers=headers)
    response1 = urllib.request.urlopen(response, timeout=0.01)
    print(response1.read().decode("utf-8"))

except socket.timeout as e:
# except Exceptio as e:
# except urllib.error.URLError as e:##报错socket.timeout: timed out
    print("time out！")

异常运行结果：

Traceback (most recent call last):
  File "D:\PycharmProjects\pythonProject\douban\test1\temp.py", line 19, in <module>
    response1 = urllib.request.urlopen(response, timeout=0.01)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 517, in open
    response = self._open(req, data)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 494, in _call_chain
    result = func(*args)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1375, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 1350, in do_open
    r = h.getresponse()
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1345, in getresponse
    response.begin()
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "C:\Users\lord\AppData\Local\Programs\Python\Python39\lib\socket.py", line 704, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\PycharmProjects\pythonProject\douban\test1\temp.py", line 22, in <module>
    except socket.timeout as e:
NameError: name 'socket' is not defined

解决方案：

参考网络资源，重新删除安装了Python，依旧报错。

通过引入socket库，解决了问题一和问题二。

import socket    ##引入socket库

正确代码：

import socket    ##引入socket库

import urllib.request  # 获取一个get请求

import urllib.parse  # 获取一个pos请求
import urllib.error

try:
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'}
    # data1 = urllib.request.Request("http://httpbin.org/post", headers=headers)

    response = urllib.request.Request("http://httpbin.org/get", headers=headers)
    response1 = urllib.request.urlopen(response, timeout=0.01)
    print(response1.read().decode("utf-8"))

except socket.timeout as e:
# except Exceptio as e:##正常运行
# except urllib.error.URLError as e:## 报错socket.timeout: timed out
    print("time out！")

正确运行结果：

time out！

Process finished with exit code 0

总结：

若未使用try/except语句，程序发生报错A。
添加了try/except语句，要注意except的异常名称要与A对应，这里报错“NameError: name ‘socket’ is not defined”源于没有导入sock库。