hadoop 2.4.1 安装之后本地库问题

最新推荐文章于 2022-08-23 17:38:21 发布

pianzif

最新推荐文章于 2022-08-23 17:38:21 发布

阅读量3.6k

点赞数 1

分类专栏： Hadoop 文章标签： hadoop集群解决方案 64位本地库

本文链接：https://blog.youkuaiyun.com/fuwencaho/article/details/38025161

版权

Hadoop 专栏收录该内容

28 篇文章

订阅专栏

昨天为了安装，整到晚上12点左右，勉强安装好了，现在是晚上十点半，想到昨天安装的有点不完美，现在特地解决一点小瑕疵

启动hdfs

[root@hd0 bin]# ../sbin/start-dfs.sh 
14/07/22 04:47:24 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hd0]
hd0: starting namenode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-namenode-hd0.out
hd1: starting datanode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-datanode-hd1.out
hd2: starting datanode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-datanode-hd2.out
Starting secondary namenodes [hd0]
hd0: starting secondarynamenode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-secondarynamenode-hd0.out
14/07/22 04:47:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@hd0 bin]#

前后报的警告是怎么回事呢

看下各个主机的进程

hd0：

[root@hd0 bin]# ps -ef|grep name
root      1257     1 10 04:47 ?        00:00:25 /hd/hdinstall/jdk1.7/bin/java -Dproc_namenode -Xmx1000m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,console -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop-root-namenode-hd0.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.NameNode
root      1397     1  6 04:47 ?        00:00:14 /hd/hdinstall/jdk1.7/bin/java -Dproc_secondarynamenode -Xmx1000m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,console -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop-root-secondarynamenode-hd0.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode
root      1537  1150  0 04:51 pts/0    00:00:00 grep name
[root@hd0 bin]#

[root@hd0 logs]# more hadoop-root-namenode-hd0.out
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /hd/hdinstall/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The 
VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
ted
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7778
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@hd0 logs]#

hd1：

[root@hd1 ~]# ps -ef|grep data
root      1190     1 10 04:47 ?        00:00:26 /hd/hdinstall/jdk1.7/bin/java -Dproc_datanode -Xmx1000m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,console -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop-root-datanode-hd1.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -Dhadoop.security.logger=ERROR,RFAS -Dhadoop.security.logger=ERROR,RFAS -Dhadoop.security.logger=ERROR,RFAS -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.datanode.DataNode
root      1258  1156  0 04:51 pts/0    00:00:00 grep data
[root@hd1 ~]#

[root@hd1 logs]# more hadoop-root-datanode-hd1.out
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /hd/hdinstall/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The 
VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
ted
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7778
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@hd1 logs]#

hd2：

[root@hd2 ~]# ps -ef|grep data
root      1185     1 10 04:47 ?        00:00:25 /hd/hdinstall/jdk1.7/bin/java -Dproc_datanode -Xmx1000m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,console -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hd/hdinstall/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop-root-datanode-hd2.log -Dhadoop.home.dir=/hd/hdinstall/hadoop-2.4.1 -Dhadoop.id.str=root -Dhadoop.root.logger=INFO,RFA -Djava.library.path=/hd/hdinstall/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -Dhadoop.security.logger=ERROR,RFAS -Dhadoop.security.logger=ERROR,RFAS -Dhadoop.security.logger=ERROR,RFAS -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.datanode.DataNode
root      1253  1151  0 04:51 pts/0    00:00:00 grep data

[root@hd2 logs]# more hadoop-root-datanode-hd2.out
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /hd/hdinstall/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The 
VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
ted
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7778
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@hd2 logs]#

现在还不知道什么原因报错，我们增加调试参数重启

export HADOOP_ROOT_LOGGER=DEBUG,console

关闭：$HADOOP_HOME/sbin/stop-all.sh

[root@hd0 sbin]# ./stop-all.sh 
This script is Deprecated. Instead use stop-dfs.sh and stop-yarn.sh
14/07/21 22:53:34 DEBUG util.Shell: setsid exited with exit code 0
14/07/21 22:53:35 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:53:35 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:53:35 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:53:35 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/07/21 22:53:36 DEBUG security.Groups:  Creating new Groups object
14/07/21 22:53:36 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/07/21 22:53:36 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
14/07/21 22:53:36 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1//lib
14/07/21 22:53:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/21 22:53:36 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/07/21 22:53:36 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/07/21 22:53:36 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/07/21 22:53:36 DEBUG security.UserGroupInformation: hadoop login
14/07/21 22:53:36 DEBUG security.UserGroupInformation: hadoop login commit
14/07/21 22:53:36 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/07/21 22:53:36 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/07/21 22:53:36 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
Stopping namenodes on [hd0]
hd0: stopping namenode
hd1: stopping datanode
hd2: stopping datanode
Stopping secondary namenodes [hd0]
hd0: stopping secondarynamenode
14/07/21 22:54:07 DEBUG util.Shell: setsid exited with exit code 0
14/07/21 22:54:09 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:54:09 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:54:09 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:54:09 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/07/21 22:54:09 DEBUG security.Groups:  Creating new Groups object
14/07/21 22:54:10 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/07/21 22:54:10 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
14/07/21 22:54:10 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1//lib
14/07/21 22:54:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/21 22:54:10 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/07/21 22:54:10 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/07/21 22:54:10 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/07/21 22:54:10 DEBUG security.UserGroupInformation: hadoop login
14/07/21 22:54:10 DEBUG security.UserGroupInformation: hadoop login commit
14/07/21 22:54:10 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/07/21 22:54:10 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/07/21 22:54:10 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
stopping yarn daemons
no resourcemanager to stop
hd1: no nodemanager to stop
hd2: no nodemanager to stop
no proxyserver to stop

启动：

[root@hd0 sbin]# ./start-dfs.sh 
14/07/21 22:59:24 DEBUG util.Shell: setsid exited with exit code 0
14/07/21 22:59:25 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:59:25 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:59:25 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:59:25 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/07/21 22:59:26 DEBUG security.Groups:  Creating new Groups object
14/07/21 22:59:26 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/07/21 22:59:26 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
14/07/21 22:59:26 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1//lib
14/07/21 22:59:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/21 22:59:26 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/07/21 22:59:26 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/07/21 22:59:27 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/07/21 22:59:27 DEBUG security.UserGroupInformation: hadoop login
14/07/21 22:59:27 DEBUG security.UserGroupInformation: hadoop login commit
14/07/21 22:59:27 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/07/21 22:59:27 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/07/21 22:59:27 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
Starting namenodes on [hd0]
hd0: starting namenode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-namenode-hd0.out
hd1: starting datanode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-datanode-hd1.out
hd2: starting datanode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-datanode-hd2.out
Starting secondary namenodes [hd0]
hd0: starting secondarynamenode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-secondarynamenode-hd0.out
14/07/21 22:59:50 DEBUG util.Shell: setsid exited with exit code 0
14/07/21 22:59:51 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:59:51 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:59:51 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 22:59:51 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/07/21 22:59:52 DEBUG security.Groups:  Creating new Groups object
14/07/21 22:59:52 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/07/21 22:59:52 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
14/07/21 22:59:52 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1//lib
14/07/21 22:59:52 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/21 22:59:52 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/07/21 22:59:52 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/07/21 22:59:53 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/07/21 22:59:53 DEBUG security.UserGroupInformation: hadoop login
14/07/21 22:59:53 DEBUG security.UserGroupInformation: hadoop login commit
14/07/21 22:59:53 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/07/21 22:59:53 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/07/21 22:59:53 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
[root@hd0 sbin]#

可以看到关闭和重启都有一个信息便是

14/07/21 22:59:26 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path

接着看下一句

14/07/21 22:59:26 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1//lib

这个路径怎么是这样呀

[root@hd0 sbin]# echo $HADOOP_COMMON_LIB_NATIVE_DIR
/hd/hdinstall/hadoop-2.4.1//lib/native
[root@hd0 sbin]# echo $HADOOP_OPTS
-Djava.library.path=/hd/hdinstall/hadoop-2.4.1//lib
[root@hd0 sbin]#

修改/etc/profile,环境变量改成

export HADOOP_HOME=/hd/hdinstall/hadoop-2.4.1
export HADOOP_PREFIX=/hd/hdinstall/hadoop-2.4.1
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_PREFIX}/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"
export PATH=$PATH:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin

重新启动，这回路径是对了，但是还是报一样的错呀

[root@hd0 sbin]# ./start-dfs.sh 
14/07/21 23:17:45 DEBUG util.Shell: setsid exited with exit code 0
14/07/21 23:17:46 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 23:17:46 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 23:17:46 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 23:17:46 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/07/21 23:17:47 DEBUG security.Groups:  Creating new Groups object
14/07/21 23:17:47 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/07/21 23:17:47 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
14/07/21 23:17:47 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1/lib
14/07/21 23:17:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/21 23:17:47 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/07/21 23:17:47 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/07/21 23:17:47 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/07/21 23:17:47 DEBUG security.UserGroupInformation: hadoop login
14/07/21 23:17:47 DEBUG security.UserGroupInformation: hadoop login commit
14/07/21 23:17:47 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/07/21 23:17:47 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/07/21 23:17:47 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
Starting namenodes on [hd0]
hd0: starting namenode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-namenode-hd0.out
hd1: starting datanode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-datanode-hd1.out
hd2: starting datanode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-datanode-hd2.out
Starting secondary namenodes [hd0]
hd0: starting secondarynamenode, logging to /hd/hdinstall/hadoop-2.4.1/logs/hadoop-root-secondarynamenode-hd0.out
14/07/21 23:18:12 DEBUG util.Shell: setsid exited with exit code 0
14/07/21 23:18:14 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 23:18:14 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 23:18:14 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[GetGroups], about=, type=DEFAULT, always=false, sampleName=Ops)
14/07/21 23:18:14 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
14/07/21 23:18:15 DEBUG security.Groups:  Creating new Groups object
14/07/21 23:18:15 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
14/07/21 23:18:15 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
14/07/21 23:18:15 DEBUG util.NativeCodeLoader: java.library.path=/hd/hdinstall/hadoop-2.4.1/lib
14/07/21 23:18:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/21 23:18:15 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
14/07/21 23:18:15 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
14/07/21 23:18:15 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
14/07/21 23:18:15 DEBUG security.UserGroupInformation: hadoop login
14/07/21 23:18:15 DEBUG security.UserGroupInformation: hadoop login commit
14/07/21 23:18:15 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
14/07/21 23:18:15 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
14/07/21 23:18:15 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
[root@hd0 sbin]#

查看官方文档 Apache > Hadoop > Apache Hadoop Project Dist POM > Apache Hadoop 2.4.1，看到如下解释，预编译的是在32为平台，而现在我用的是预编译且系统是centos6.4 64位

Supported Platforms

The native hadoop library is supported on *nix platforms only. The library does not to work with Cygwin or the Mac OS X platform.

The native hadoop library is mainly used on the GNU/Linus platform and has been tested on these distributions:

RHEL4/Fedora
Ubuntu
Gentoo

On all the above distributions a 32/64 bit native hadoop library will work with a respective 32/64 bit jvm.

Download

The pre-built 32-bit i386-Linux native hadoop library is available as part of the hadoop distribution and is located in the lib/native directory. You can download the hadoop distribution from Hadoop Common Releases.

Be sure to install the zlib and/or gzip development packages - whichever compression codecs you want to use with your deployment.

难道是这个原因！但是先不管这个吧，现在只是一个警告，不知在后续的hadoop学习中这个问题是怎样暴露出来的，暂且不改，后续再看!唉，又十二点过了

主要参考：

http://www.oschina.net/question/154627_147919

http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html

https://gist.github.com/ruo91/7154697

--一种不重新编译hadoop的方法

http://www.cnblogs.com/gpcuster/archive/2011/02/17/1957042.html

--另一种解决方案，包括重新编译hadoop成64位版本

http://blog.youkuaiyun.com/bamuta/article/details/13506843

---相关问题

http://blog.sina.com.cn/s/blog_ac9fdc0b0101rwgm.html