玩的Python爬虫的小伙伴们一般知道pycurl模块,但是在MACOS下安装这个模块,可真不是一般的折腾啊〜啊〜啊〜
操作系统版本:macOS Mojave 10.14.6
一、缓缓入坑
sudo pip install pycurl --user
安装结束,但是在运行的时候报错来啦:
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
这是由于新版的Mac系统中使用的SSL版本和之前的不一样了,针对的OpenSSL的漏洞苹果方面进行了升级,现在系统默认使用的是LibreSSL而不是OpenSSL的。按照网上各方坑友们的操作指导建议下先手动安装OpenSSL的
-
如果没有安装brew,那就先安装吧:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
2.安装OpenSSL:brew install openssl,安装完它会提示你这些:
openssl is keg-only, which means it was not symlinked into /usr/local,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.
If you need to have openssl first in your PATH run:
echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile
For compilers to find openssl you may need to set:
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
==> Summary
? /usr/local/Cellar/openssl/1.0.2s: 1,795 files, 12.0MB
3.安装完的OpenSSL后,再来试一波:
export LDFLAGS="-L/usr/local/opt/openssl/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include"
export PYCURL_SSL_LIBRARY=openssl
sudo pip install pycurl --with-openssl --compile --user
安装过程依然没报错,但是...运行依然:
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
总不能把curl也重新安装吧...要不试试?试试就试试:brew install curl
安装完再来一遍......还是报错!
二、尝试出坑
这次选择源码安装:从官网下载,或者GitHub的上下载
官网下载:
curl -O https://files.pythonhosted.org/packages/ac/b3/0f3979633b7890bab6098d84c84467030b807a1e2b31f5d30103af5a71ca/pycurl-7.43.0.3.tar.gz
解压:
tar zxvf pycurl-7.43.0.3.tar.gz
cd pycurl-7.43.0.3
慢着!这源码编译需要啥特殊的参数呢?直接python setup.py安装可行吗?为了避免再走无谓的弯路,我决定好好研究一下:
1.先看看源码安装的参数选项:
python setup.py --help
PycURL Unix options:
--curl-config=/path/to/curl-config use specified curl-config binary
--libcurl-dll=[/path/to/]libcurl.so obtain SSL library from libcurl.so
--openssl-dir=/path/to/openssl/dir path to OpenSSL headers and libraries
--with-openssl libcurl is linked against OpenSSL
--with-ssl legacy alias for --with-openssl
--with-gnutls libcurl is linked against GnuTLS
--with-nss libcurl is linked against NSS
--with-mbedtls libcurl is linked against mbedTLS
Common commands: (see '--help-commands' for more)
setup.py build will build the package underneath 'build/'
setup.py install will install the package
Global options:
--verbose (-v) run verbosely (default)
--quiet (-q) run quietly (turns verbosity off)
--dry-run (-n) don't actually do anything
--help (-h) show detailed help message
--no-user-cfg ignore pydistutils.cfg in your home directory
--command-packages list of packages that provide distutils commands
Information display options (just display information, ignore any commands)
--help-commands list all available commands
--name print package name
--version (-V) print package version
--fullname print <package name>-<version>
--author print the author's name
--author-email print the author's email address
--maintainer print the maintainer's name
--maintainer-email print the maintainer's email address
--contact print the maintainer's name if known, else the author's
--contact-email print the maintainer's email address if known, else the
author's
--url print the URL for this package
--license print the license of the package
--licence alias for --license
--description print the package description
--long-description print the long package description
--platforms print the list of platforms
--classifiers print the list of classifiers
--keywords print the list of keywords
--provides print the list of packages/modules provided
--requires print the list of packages/modules required
--obsoletes print the list of packages/modules made obsolete
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
2.找到官网的文档中一段很有用的文字
PycURL requires that the SSL library that it is built against is the same one libcurl, and therefore PycURL, uses at runtime. PycURL’s setup.py uses curl-config to attempt to figure out which SSL library libcurl was compiled against, however this does not always work.
意思就是说PycURL要求它构建的SSL库与libcurl的相同,一般使用curl-config
来试图找出哪些SSL库的libcurl中被一起编译,但是这并不总是有效的。
3.先试一波
sudo python setup.py --curl-config=/usr/bin/curl-config install --user
>>> import pycurl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
果然,不是“总是有效的”,官网要不那么靠谱啊。
4.看样子还要点其他的参数,这时候关于ssl的就剩下with-openssl了(--with-ssl是别名,效果一样),试一下?
sudo python setup.py --curl-config=/usr/bin/curl-config --openssl-dir=/usr/local/Cellar/openssl/1.0.2s --with-openssl install --user
>>> import pycurl
>>> pycurl.version
'PycURL/7.43.0.3 libcurl/7.54.0 LibreSSL/2.6.5 zlib/1.2.11 nghttp2/1.24.1'
欧耶!出坑啦!但是〜是不是看到了很奇怪的东西,是的,明明参数指定的是OpenSSL,但是编译成功后的版本信息中查到的却是“ LibreSSL ”,而不是“OpenSSL” ,是不是很奇怪?是啊,我也觉得奇怪,如果有人知道的话,请与我分享一下吧!
这次的坑还挺深的,从网上看遇到这类的朋友不少,也有成功的样例,但是解决方案没有适合我这次的安装场景,毕竟有时候官网提供的方案都不能“总是有效”的呢!比较幸运,终于搞定了,现在这个安装步骤也是一种解决方案啦,欢迎大家试用!请记住这组命令,希望能够帮到你:
sudo python setup.py --curl-config=/usr/bin/curl-config --openssl-dir=/usr/local/Cellar/openssl/1.0.2s --with-openssl install --user
官方安装参考文档
https://pypi.org/project/pycurl/7.43.0.3/
http://pycurl.io/docs/latest/install.html#pip-and-cached-pycurl-package