java.io.IOException: (null) entry in command string: null ls -F E:\tmp\hive解决办法

博客主要讲述本地执行Spark SQL生成数据时出现java.io.IOException错误的解决办法。包括下载并解压hadoop-2.9.2,将winutils.exe文件放入hadoop的bin目录,配置hadoop环境变量,对报错文件夹授权,查看权限更改情况,最后重启idea重新运行程序。

问题:

本地执行spark sql,生成数据时会出现java.io.IOException: (null) entry in command string: null ls -F E:\tmp\hive错误

 

解决办法:

1. 下载hadoop-2.9.2(下载地址:https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz),并解压到本地

2. 下载winutils.exe(下载地址:https://pan.baidu.com/s/1Rb5ROUQMSqp7SeQINlLZkA 提取码:n8t6),解压出来的winutils.exe文件放到hadoop中bin目录下

3. 配置hadoop的环境变量,在系统变量path中添加hadoop的bin和sbin路径变量

4. 在cmd中执行D:\hadoop\hadoop-2.9.2\bin\winutils.exe chmod 777 E:\tmp\hive 命令,对E:\tmp\hive文件夹进行授权,(此处E:\tmp\hive根据自身程序报错信息进行更改)

5. 然后再执行D:\hadoop\hadoop-2.9.2\bin\winutils.exe ls E:\tmp\hive命令,查看文件夹权限是否更改

6. 重启idea,重新运行程序即可。

log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Exception in thread "main" java.io.IOException: (null) entry in command string: null chmod 0700 D:\tmp\hadoop-曼荼罗\mapred\staging\ÂüݱÂÞ1381487167\.staging at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:772) at org.apache.hadoop.util.Shell.execCommand(Shell.java:869) at org.apache.hadoop.util.Shell.execCommand(Shell.java:852) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:733) at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:491) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:531) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:509) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:312) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:133) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:144) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) at mapreduce.EmailCount.main(EmailCount.java:99) 进程已结束,退出代码1
09-17
评论 4
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值