hadoop（2.7.1）配置过程问题

1. formate 出错

ERROR namenode.NameNode: Failed to start namenode.
java.io.IOException: Cannot create directory /usr/hadoop/dfs/name/current
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDi rectory(Storage.java:337)

解决方法：/usr/hadoop/dfs 和 /usr/hadoop/tmp 都没有权限，更改如下

sudo chown -R hadoop:hadoop /usr/hadoop/dfs
sudo chown -R hadoop:hadoop /usr/hadoop/tmp

2. start-all.sh 启动登陆密码

设置完免密码后，master 登陆master 仍然需要登陆密码，

解决方法为：
将id_rsa.pub 公钥内容，拷贝到 authorized_keys 中。

3. hdfs上传文件出错

17/03/02 21:56:02 WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /data/input/REA DME.txt.COPYING could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this opera tion.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarg et4NewBlock(BlockManager.java:1550)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTarget s(FSNamesystem.java:3110)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBloc k(FSNamesystem.java:3034)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(Nam eNodeRpcServer.java:723)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTra nslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$Cl ientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.cal l(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma tion.java:1657) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

    at org.apache.hadoop.ipc.Client.call(Client.java:1476)
    at org.apache.hadoop.ipc.Client.call(Client.java:1407)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEng                                                                                                             ine.java:229)
    at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.                                                                                                             addBlock(ClientNamenodeProtocolTranslatorPB.java:418)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.                                                                                                             java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces                                                                                                             sorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryI                                                                                                             nvocationHandler.java:187)
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocat                                                                                                             ionHandler.java:102)
    at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
    at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBl                                                                                                             ock(DFSOutputStream.java:1430)
    at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputSt                                                                                                             ream(DFSOutputStream.java:1226)
    at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStre                                                                                                             am.java:449)

put: File /data/input/README.txt.COPYING could only be replicated to 0 nodes i nstead of minReplication (=1). There are 0 datanode(s) running and no node(s) a re excluded in this operation.

方法：

1、关闭防火墙： sudo ufw disable
2、 将/etc/hosts 文件下， #127.0.1.1 master 屏蔽掉
3、start-dfs.sh start-yarn.sh