Hadoop util.NativeCodeLoader: Unable to load native-hadoop library for your platform

最新推荐文章于 2021-11-20 01:00:00 发布

原创最新推荐文章于 2021-11-20 01:00:00 发布 · 873 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#Hadoop #HADOOP_OPTS #hadoop-env.sh #native-hadoop library

Hadoop 专栏收录该内容

3 篇文章

订阅专栏

本文详细介绍了在Hadoop环境中遇到无法加载本地库的问题，包括hadoop、zlib、snappy、lz4、bzip2和openssl等，并提供了解决方案。通过修改hadoop配置文件，追加java.library.path环境变量指向本地库路径，成功解决了问题。

参考资料:

http://blog.sina.com.cn/s/blog_3d9e90ad0102wqrp.html

报错呈现

[root@Master sbin]# $HADOOP_HOME/bin/hadoop checknative
19/09/19 15:27:31 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop:  false 
zlib:    false 
snappy:  false 
lz4:     false 
bzip2:   false 
openssl: false 
19/09/19 15:27:31 INFO util.ExitUtil: Exiting with status 1

解决方案:

修改hadoop配置文件

cd /opt/hadoop-2.8.5/etc/hadoop
vi hadoop-env.sh

后面追加

export HADOOP_OPTS="-Djava.library.path=${HADOOP_HOME}/lib/native/"

${HADOOP_HOME}是已经配置的系统环境变量,根据自己机器环境自行修改.

重启 hadoop

检查问题是否修复

[root@Master hadoop]# $HADOOP_HOME/bin/hadoop checknative checknative
19/09/19 15:49:06 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
19/09/19 15:49:06 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:
hadoop:  true /opt/hadoop-2.8.5/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
snappy:  true /lib64/libsnappy.so.1
lz4:     true revision:10301
bzip2:   true /lib64/libbz2.so.1
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!