java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
搭建好Hadoop集群后写java程序测试确出现如下异常:
2017-03-10 11:48:49,887 ERROR [main] util.Shell (Shell.java:getWinUtilsPath(397)) - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:379)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:394)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:387)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:610)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:273)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:261)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:791)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:761)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:634)
at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:72)
at org.apache.hadoop.mapreduce.Job.<init>(Job.java:142)
at org.apache.hadoop.mapreduce.Job.<init>(Job.java:129)
at com.sct.hadoop.mapreduce.Temperature.main(Temperature.java:106)
查看hadoop源码发现里有这么一段:
public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable;
File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}
return exeFile.getCanonicalPath();
}
private static String HADOOP_HOME_DIR = checkHadoopHome();
private static String checkHadoopHome() {
// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir");
// fall back to the system/user-global env variable
if (home == null) {
home = System.getenv("HADOOP_HOME");
}
...
}
很明显应该是HADOOP_HOME的问题。如果HADOOP_HOME为空,必然fullExeName为null\bin\winutils.exe。解决方法很简单,配置环境变量,不想重启电脑可以在程序里加上:
System.setProperty("hadoop.home.dir", "C:\\Users\\Administrator\\Desktop\\hadoop-2.7.3\\hadoop-2.7.3");
不过直接这样还是不行,因为Hadoop-2.7.3/bin下面还是没有winutils.exe这个文件
于是我告诉你,你可以去github下载一个,地球人都知道的地址发你一个。
地址:https://github.com/srccodes/hadoop-common-2.2.0-bin
不要顾虑它的版本,不用怕,因为我用的最新的hadoop-2.7.0都没问题!下载好后,把winutils.exe加入你的hadoop-x.x.x/bin下。
重新运行程序就OK了
本文解决Hadoop集群测试中出现的“Could not locate executable nullinwinutils.exe”错误。通过设置HADOOP_HOME环境变量及从GitHub下载winutils.exe文件,解决了Java程序无法找到该文件的问题。
880

被折叠的 条评论
为什么被折叠?



