mac os用pip安装scrapy时报错#include "libxml/xmlversion.h" file not found

最新推荐文章于 2019-11-05 10:37:30 发布
原创 最新推荐文章于 2019-11-05 10:37:30 发布 · 7.1k 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#爬虫 #python

本文详细介绍了在安装Python爬虫框架Scrapy时遇到libxml/xmlversion.h文件未找到的错误，并提供了通过指定C_INCLUDE_PATH路径来解决问题的方法。包括在XCode MacOSX SDK中的正确配置步骤和使用sudo权限执行pip install Scrapy。

安装一个python爬虫框架scrapy，pip安装时总是报如下错误：

lxml/src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
    #include "libxml/xmlversion.h"
             ^
    1 error generated.
    error: command 'cc' failed with exit status 1

试了几次仍然不行，用作为'libxml/xmlversion.h' file not found关键词搜索

很快就找到来答案，记录一下。

C_INCLUDE_PATH要指定到XCode MacOSX SDK中libxml路径，一开始只看到：

C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/libxml2:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/libxml2/libxml:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include
后来发现还要打上sudo和pip install才行 

sudo C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/libxml2:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/libxml2/libxml:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include pip install scrapy
爬虫教程（ 2 ） --- 爬虫框架 Scrapy、Scrapy 实战
墨鱼菜鸡
07-11 2124
From：https://piaosanlang.gitbooks.io/spiders/content/ scrapy-cookbook ：https://scrapy-cookbook.readthedocs.io/zh_CN/latest/index.html 1.爬虫框架 Scrapy 爬虫框架中比较好用的是 Scrapy 和 PyS...
linux下scrapy环境搭建
github_26532395的博客
03-16 567
最近使用scrapy做数据挖掘，使用scrapy定时抓取数据并存入MongoDB，本文记录环境搭建过程以作备忘 OS:ubuntu 14.04 python:2.7.6 scrapy:1.0.5 DB:MongoDB 3 ps.ubuntu14.04内置python2.7，因此python及pip安装不再赘述。一.安装scrapy 　　pip install Scrapy
fatal error: 'libxml/xmlversion.h' file not found
weixin_30621711的博客
09-15 220
问题： MACOS安装scrapy时，安装lxml出现错误 In file included from src/lxml/lxml.etree.c:232: /tmp/easy_install-O2UfP7/lxml-3.4.0/src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not ...
libxml/tree.h file not found解决办法
MOKA
07-30 3433
1.  导入libxml2.dylib 包 2.设置Header Search Paths 为 /usr/include/libxml2 如图:   file not found解决办法" name="image_operate_39671351567458229" alt="libxml/tree.h file not found解决办法" src="http://s
致命错误：libxml/xmlversion.h：没有那个文件或目录 #include "libxml/xmlversion.h"
sonia_liss的博客
11-05 2837
python 安装错误记录： > cpp -o build/temp.linux-x86_64-3.6/dragnet/blocks.o cc1plus: > 警告：command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but > not for C++ [默认启用] dragnet/blocks.cpp:48...
The way to fix error: Lexical or Preprocessor Issue 'libxml/xmlversion.h' file not found
lwf97055143的专栏
08-02 3584
When using RXMLElement.h, I always get below error when compiling the code:  Lexical or Preprocessor Issue 'libxml/xmlversion.h' file not found. When checking, I can always see xmlversion.h existing
python安装软件出现错误 fatal error: 'libxml/xmlversion.h' file not found
鹰击长空
12-31 3223
解决办法： sudo C_INCLUDE_PATH=/Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/usr/include/libxml2:/Applications/Xcode.app/Contents/D
这些是啥？2025-07-27 17:56:44 [scrapy.utils.log] INFO: Scrapy 2.13.3 started (bot: scrapybot) 2025-07-27 17:56:44 [scrapy.utils.log] INFO: Versions: {'lxml': '5.4.0', 'libxml2': '2.11.9', 'cssselect': '1.3.0', 'parsel': '1.10.0', 'w3lib': '2.3.1', 'Twisted': '25.5.0', 'Python': '3.13.0 (tags/v3.13.0:60403a5, Oct 7 2024, 09:38:07) [MSC v.1941 ' '64 bit (AMD64)]', 'pyOpenSSL': '25.1.0 (OpenSSL 3.5.1 1 Jul 2025)', 'cryptography': '45.0.5', 'Platform': 'Windows-10-10.0.19045-SP0'} 2025-07-27 17:56:44 [scrapy.addons] INFO: Enabled addons: [] 2025-07-27 17:56:44 [asyncio] DEBUG: Using selector: SelectSelector 2025-07-27 17:56:44 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor 2025-07-27 17:56:44 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.windows_events._WindowsSelectorEventLoop 2025-07-27 17:56:44 [scrapy.extensions.telnet] INFO: Telnet Password: 70d0475b95b184a2 2025-07-27 17:56:44 [py.warnings] WARNING: C:\Users\12572\PyCharmMiscProject\.venv\Lib\site-packages\scrapy\extensions\feedexport.py:455: ScrapyDeprecationWarning: The `FEED_URI` and `FEED_FORMAT` settings have been deprecated in favor of the `FEEDS` setting. Please see the `FEEDS` setting docs for more details exporter = cls(crawler) 2025-07-27 17:56:44 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats'] 2025-07-27 17:56:44 [scrapy.crawler] INFO: Overridden settings: {'CONCURRENT_REQUESTS': 4, 'DOWNLOAD_DELAY': 1.5, 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' '(KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'} 2025-07-27 17:56:44 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2025-07-27 17:56:45 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.start.StartSpiderMiddleware', 'scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-07-27 17:56:45 [scrapy.middleware] INFO: Enabled item pipelines: [] 2025-07-27 17:56:45 [scrapy.core.engine] INFO: Spider opened 2025-07-27 17:56:45 [py.warnings] WARNING: C:\Users\12572\PyCharmMiscProject\.venv\Lib\site-packages\scrapy\core\spidermw.py:433: ScrapyDeprecationWarning: __main__.BilibiliSpider defines the deprecated start_requests() method. start_requests() has been deprecated in favor of a new method, start(), to support asynchronous code execution. start_requests() will stop being called in a future version of Scrapy. If you use Scrapy 2.13 or higher only, replace start_requests() with start(); note that start() is a coroutine (async def). If you need to maintain compatibility with lower Scrapy versions, when overriding start_requests() in a spider class, override start() as well; you can use super() to reuse the inherited start() implementation without copy-pasting. See the release notes of Scrapy 2.13 for details: https://docs.scrapy.org/en/2.13/news.html warn( 2025-07-27 17:56:45 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-07-27 17:56:45 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2025-07-27 17:56:45 [selenium.webdriver.common.selenium_manager] DEBUG: Selenium Manager binary found at: C:\Users\12572\PyCharmMiscProject\.venv\Lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe 2025-07-27 17:56:45 [selenium.webdriver.common.selenium_manager] DEBUG: Executing process: C:\Users\12572\PyCharmMiscProject\.venv\Lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe --browser chrome --language-binding python --output json
最新发布
07-28
| `lxml not found, using built-in selector` | 解析性能下降 | 安装lxml：`pip install lxml` | | `ValueError: Missing scheme in request url` | URL格式错误 | 检查URL是否包含`http://`或`https://` | | `...
Scrapy 全部文件 和 安装步骤
01-08
如果你的Python环境中还没有pip，可以使用以下命令安装： ``` python3 -m ensurepip --upgrade ``` 3. **安装Scrapy**：打开终端或命令提示符，输入以下命令安装Scrapy： ``` pip3 install scrapy ``` 4. **...
python3.7安装scrapy
05-29
根据引用[2]，安装顺序应该是先安装Python3.7，然后安装pip，接着安装pywin32、pyOpenSSL、lxml、zope.interface、Twisted，最后安装Scrapy。同时，引用[3]和[4]提到可能需要单独下载某些库的.whl文件，尤其是...
libxml/tree.h file not found
weixin_33671935的博客
07-10 120
stackoverflow用户对添加libxml2库表现出了极大的抱怨 ，原因在要把它好好地添加进去实在是太复杂了。 我就是因为出现了'libxml/tree.h file not found’错误，才发现的这篇贴子，照着做，错误就消除了，备注如下： 原始地址：http://stackoverflow.com/questions/1428847/libxml-tree-h-no-such-fi...
Scrapy安装过程中出现“'libxml/xmlversion.h' file not found“报错的处理方法
weixin_30457881的博客
03-23 218
在Mac上使用“easy_install scrapy”命令安装Scrapy时，出现xmlversion.h头文件不存在的报错： 查阅了一些资料，有可能是Mac升级到了Yosemite造成的，使用控制台安装command line tools就可以解决了。在控制台执行如下命令：xcode-select —install 。 安装完成后，再执行“easy_install scrap...
#import<libxml/tree.h> not found
Leemin_ios的博客
08-31 2279
1.targets->Build Phases->Link Binary With Libraries点击加号 2.输入libxml，添加libxml.tbd 3.找到Search Paths选项卡，其下有Header Search Paths条目，双击debug选项，添加/usr/include/libxml2就行了。
Mac安装 nokogiri 失败。缺少libxml2 解决方法
看飘雪的猪
04-28 630
  mac 10.12.4 ruby 2.4  安装 nokogiri 失败。缺少libxml2 解决方法 gem install nokogiri -- --use-system-libraries=true --with-xml2-include=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/...
MAC OSX下用pip安装lxml时遇到xmlversion.h not found的解决办法
Wave的专栏
07-12 6840
今天在Mac下用pip
python 安装提示fatal error: libxml/xmlversion.h: No such file or directory
u010856284的博客
12-05 1481
因为缺少相关的依赖包导致出现了错误提示： 输入以下命令即可：sudo apt-get install libxml2-dev libxslt1-dev
error: libxml/xmlversion.h: No such file or directory
shuishensanchi的专栏
10-27 3447
错误信息如下： /usr/include/libxml2/libxml/xmlmemory.h:16: error: libxml/xmlversion.h: No such file or directory #include ^ 找了半个上午错误，网上的解决办法大多都是安装libxml2，但是这个已经安装，查看了
python 2.7安装某些包出现错误：“libxml/xmlversion.h：没有那个文件或目录“
不见其长，日有所长
08-20 897
&#13; 解决办法：&#13; 1. ubuntu系统：&#13; 首先：&#13; &#13; 1 apt-get install libxml2-dev&#13; 2 sudo ln -s /usr/include/libxml2/libxml /usr/include/libxml&#13; &#13; 然后：&#13; &#13; 1 apt-get insta...
安装lxml遇到的问题
化茧成蝶007
12-08 989
在安装lxml时出现的错误: src/lxml/includes/etree_defs.h:14:31: fatal error: libxml/xmlversion.h: No s  1910  mv /var/lib/dpkg/info /var/lib/dpkg/info.bak  1911  mkdir /var/lib/dpkg/info  1912  sudo apt-
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值