Hadoop报错 Exception in thread “main“ java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.Na

问题描述

在 idea 中运行 MapReduce 项目出现报错

Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Ljava/lang/String;)Lorg/apache/hadoop/io/nativeio/NativeIO$POSIX$Stat;
	at org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Native Method)
	at org.apache.hadoop.io.nativeio.NativeIO$POSIX.getStat(NativeIO.java:455)
	at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfoByNativeIO(RawLocalFileSystem.java:796)
	at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:710)
	at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:678)
	at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:52)
	at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:2091)
	at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:2071)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:314)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:274)
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:396)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:310)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:327)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1570)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1567)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1567)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1588)
	at com.bnz.mapreduce.wordcount.WordCountDriver.main(WordCountDriver.java:41)

启动失败

分析原因

org.apache.hadoop.io.nativeio.NativeIO 包不匹配

解决方案

1. 在项目中新建一个与 org.apache.hadoop.io.nativeio.NativeIO 同路径的文件
   
2. 在 idea中 连按两下 shift 弹出搜索框，搜索 NativeIO
   
3. 选择java文件，将其内容复制，并粘贴到自己创建的 NativeIO 文件中，使其替代源文件
4. 在NativeIO中 ctrl+f 搜索 access，找到以下方法，将其返回值修改为 true

        public static boolean access(String path, AccessRight desiredAccess)
                throws IOException {
//            return access0(path, desiredAccess.accessRight());
            return true;
        }

5. 最后注意： hadoop\bin 文件夹中的 hadoop.dll 文件将其复制一份，放入 C:\Windows\System32
6. 测试运行成功