Hadoop:hadoop.job.ugi

最新推荐文章于 2023-05-25 08:58:09 发布
转载 最新推荐文章于 2023-05-25 08:58:09 发布 · 6.1k 阅读 · 1

本文介绍了如何通过设置Hadoop作业的用户标识属性(hadoop.job.ugi)来指定Hadoop任务运行时使用的用户名和组名。这对于使用不同于系统登录用户的Hadoop用户来说尤其有用。

转发自：http://my.oschina.net/ale0512/blog/55003



如果Hadoop用户标识不同于客户机上的用户账号，可以通过hadoop.job.ugi属性来显式设定Hadoop用户名和组名。

用户名和组名之间有半角逗号隔开例如：

1preston,diecotrs,inventors

其中preston为用户名，diecotrs和inventor为组名。
【HDFS】Hadoop分布式文件系统
starlywang的博客
03-14 692
【HDFS】Hadoop分布式文件系统一、总体介绍二、架构1. 名称节点(NameNode)2. 镜像和日志（Image and Journal）3. 数据节点（DataNodes）4. HDFS客户端5. Checkpoint节点6. 备份节点（BackupNode）7. 升级和文件系统快照三、文件IO操作和副本管理1. 文件读写2. Block块放置3. 复制管理4. 均衡器5. Block...
【Java】Java 安全认证和 Hadoop UGI 原理解析
HHoao的博客
09-24 1572
一般来说，Java 安全认证主要通过自定义 Subject、LoginContext、LoginModule、Configuration 在 Java 中进行安全认证。
Hadoop: hadoop.job.ugi
wjlwangluo的博客
08-04 6772
如果Hadoop用户标识不同于客户机上的用户账号，可以通过hadoop.job.ugi属性来显式设定Hadoop用户名和组名。 用户名和组名之间有半角逗号隔开例如： 1 preston,diecotrs,inventors 其中preston为用户名，diecotrs和inventor为组名。
Hadoop集群常见错误
欢迎朋友，我是一名十多年开发经验的新手，希望多多指教。
05-25 1645
Hadoop常见错误原因分析及解决方法
Clouder CDH3B3开始后hadoop.job.ugi不再生效
Royce Wang专栏
08-25 7637
Clouder CDH3B3开始后hadoop.job.ugi不再生效！ 困扰了我好几天的，终于找到了原因。以前公司用的原版hadoop-0.20.2,使用java设置 hadoop.job.ugi为正确的hadoop用户和组即可正常访问hdfs并可创建删除等。 更新
服务器执行job文件,Hadoop Job运行问题解决
weixin_32147929的博客
08-12 500
问题现象：job提交后总是在本地机器运行，不能提交到Hadoop job tracker，在http://192.168.1.130:50030/jobtracker.jsp，不能看到运行中的job。在控制台输出中出现：14/02/15 00:04:20 INFO mapred.LocalJobRunner: reduce > sort原因分析：第一次尝试：将原来使用的默认new Confi...
hadoop常见问题及解决办法（一）
wumingshixiaowu的专栏
03-18 1354
我们经常会遇到一些问题，而且可能会重复性遇到，这些方案可以收藏为以后备用。我们经常遇到如下问题： 1.两次以上格式化造成NameNode 和 DataNode namespaceID 不一致，有几种解决办法？ 2.如何动态添加DataNode 动态将某个节点加入到集群中 3.用window 提交eclipse 任务发现权限不通过： 4.eclipse 运行中发现 Name node
hadoop的mapreducer处理数据（Python）
jack.li的博客
03-20 735
1.hadoop客户端环境 1.直接找有hadoop服务的机器，这样你访问的就是本机的hadoop集群，也就不用在配置了 2.如果你要远程其他hadoop集群，那么你需要配置相关文件，配置方式如同配置hadoop集群一样 hadoop集群搭建详见：https://blog.csdn.net/xzpdxz/article/details/86692631 修改相应的配置 注意确保你的环境有j...
Hadoop 分布式存储系统介绍
m0_67393828的博客
08-16 2639
hadoop组成分布式存储(hdfs)hadoop1.x 存储系统hadoop 2.x 存储系统存储账本原理常用命令操作Java 代码实现配置文件获取对象创建文件夹上传文件下载文件递归列出文件夹中所有内容(包含子目录)参考文献。...
hadoop基本命令
qq_34124060的博客
02-24 375
用户命令 distcp 用于大规模集群内部和集群之间拷贝的工具， 使用Map/Reduce实现文件分发，错误处理和恢复，以及报告生成 把文件和目录的列表作为map任务的输入，每个任务会完成源列表中部分文件的拷贝 命令行中可以指定多个源目录： bash$ hadoop distcp hdfs://nn1:8020/foo/a \ hdfs://...
Hadoop开发基础知识记录
东杰书屋
06-19 4841
（1）当你把一个文件加入distribution cache的时候，要注意：如果你是以addCacheFile()的方式添加的，而你在mapper中取出来的时候，却是以archive的方式取出来——getLocalCacheArchives()，那么，你将得不到cache文件的路径，因为放进去和取出来的方式要一致。 （2）在mapper中获取当前正在处理的HDFS文件名/HDFS目录名 有时候
【Hadoop---05】HDFS常用命令 『Hadoop常用命令』
渐渐的忘记，赶不上明天（读研ing，2024年毕业）
09-26 1239
1. 基本语法 2. 查看帮助信息 3. 常用命令 3.1 与Linux命令一样的命令 3.2 上传 / 下载 命令 3.3 其他
Hadoop安全认证(2)
热门推荐
行人事，知天命
08-02 1万+
凭证过期处理策略 在最早的 Security features for Hadoop 设计中提出这样的假设: A Hadoop job will run no longer than 7 days (configurable) on a MapReduce cluster or accessing HDFS from the job will fail.
linux下创建用户
shupan001的专栏
03-01 6629
<br />[root@b etc]#useradd -d /home/jc -m jc<br /> [root@b etc]# passwd jc<br /> [root@b etc]#chown jc -R /home/jc<br /> [root@b etc]#su － jc 就ok拉<br /><br /><br /> linux下创建用户(一）<br /> linux下创建用户(一）<br /> 2008-04-02 18:06<br /> Linux 系统是一个多用户多任务的分
hadoop常用命令
蘑菇丁的专栏
11-10 1273
查看CPU：cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c 查看内存：free –m 查看磁盘：df  -m 处理器信息：dmidecode | grep -A48 'Processor Information$' 查看目录的大小 [hadoop@masternode1 ~]$ hdfs dfs -ls / Found 8
Hadoop授权令牌解释(原标题 Hadoop Delegation Tokens Explained)
hxiaowang的博客
06-09 5706
转载：https://blog.cloudera.com/hadoop-delegation-tokens-explained/ 很好的文章，但是要翻墙转载给国内的伙伴，有问题请联系删除 第一部分谷歌翻译版；下边有英语版 Apache Hadoop的安全性是在2009年左右设计和实施的，此后一直保持稳定。但是，由于缺少有关此领域的文档，因此出现问题时很难理解或调试。设计了委托令牌，并将其作为身份验证方法在Hadoop生态系统中广泛使用。这篇博客文章介绍了Hadoop分布式文件系统（HDF...
java ugi确认框_在对hadoop进行任何操作之前，我应该调用ugi.checkTGTAndReloginFrom
weixin_42571280的博客
02-25 588
Hadoop提交者在这里！这是一个很好的问题。不幸的是，如果不深入研究应用程序的特定使用模式，很难给出明确的答案。相反，我可以提供一般准则，并描述Hadoop何时为您自动处理票证更新或从keytab重新登录，以及何时不这样做。Hadoop生态系统中Kerberos身份验证的主要用例是Hadoop的RPC框架，该框架使用SASL进行身份验证。Hadoop生态系统中的大多数守护进程都通过UserGro...
输入代码 import java.io.IOException; import java.util.Iterator; import java.util.StringTokenizer; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.util.GenericOptionsParser; public class WordCount { public WordCount() { } public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); Job job = Job.getInstance(conf, "word count"); job.setJarByClass(WordCount.class); job.setMapperClass(WordCount.TokenizerMapper.class); job.setCombinerClass(WordCount.IntSumReducer.class); job.setReducerClass(WordCount.IntSumReducer.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); FileInputFormat.addInputPath(job, new Path("hdfs://10.23.175.100:9000/user/wordcount.txt")); FileOutputFormat.setOutputPath(job, new Path("hdfs://10.23.175.100:9000/user/result15")); System.exit(job.waitForCompletion(true)?0:1); } public static class TokenizerMapper extends Mapper<Object, Text, Text, IntWritable> { private static final IntWritable one = new IntWritable(1); private Text word = new Text(); public TokenizerMapper() { } public void map(Object key, Text value, Mapper<Object, Text, Text, IntWritable>.Context context) throws IOException, InterruptedException { StringTokenizer itr = new StringTokenizer(value.toString()); while(itr.hasMoreTokens()) { this.word.set(itr.nextToken()); context.write(this.word, one); } } } public static class IntSumReducer extends Reducer<Text, IntWritable, Text, IntWritable> { private IntWritable result = new IntWritable(); public IntSumReducer() { } public void reduce(Text key, Iterable<IntWritable> values, Reducer<Text, IntWritable, Text, IntWritable>.Context context) throws IOException, InterruptedException { int sum = 0; IntWritable val; for(Iterator i$ = values.iterator(); i$.hasNext(); sum += val.get()) { val = (IntWritable)i$.next(); } this.result.set(sum); context.write(key, this.result); } } } 后报错 D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\bin\java.exe "-javaagent:D:\2025.9\JAVA\IDEA 2023.1\IntelliJ IDEA 2023.1\lib\idea_rt.jar=4323:D:\2025.9\JAVA\IDEA 2023.1\IntelliJ IDEA 2023.1\bin" -Dfile.encoding=UTF-8 -classpath D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\charsets.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\deploy.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\access-bridge-64.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\cldrdata.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\dnsns.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\jaccess.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\jfxrt.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\localedata.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\nashorn.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\sunec.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\sunjce_provider.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\sunmscapi.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\sunpkcs11.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\ext\zipfs.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\javaws.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\jce.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\jfr.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\jfxswt.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\jsse.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\management-agent.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\plugin.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\resources.jar;D:\2025.9\Hadoop\test\4\jdk1.8\jdk1.8\jdk1.8.0_181\jre\lib\rt.jar;D:\2025.9\Hadoop\java\hadoop_project\out\production\hadoop_project;D:\2025.9\Hadoop\test\4\lib2\fst-2.50.jar;D:\2025.9\Hadoop\test\4\lib2\re2j-1.1.jar;D:\2025.9\Hadoop\test\4\lib2\asm-5.0.4.jar;D:\2025.9\Hadoop\test\4\lib2\guice-4.0.jar;D:\2025.9\Hadoop\test\4\lib2\jna-5.2.0.jar;D:\2025.9\Hadoop\test\4\lib2\avro-1.7.7.jar;D:\2025.9\Hadoop\test\4\lib2\gson-2.8.9.jar;D:\2025.9\Hadoop\test\4\lib2\okio-2.8.0.jar;D:\2025.9\Hadoop\test\4\lib2\jline-3.9.0.jar;D:\2025.9\Hadoop\test\4\lib2\jsch-0.1.55.jar;D:\2025.9\Hadoop\test\4\lib2\asm-tree-9.1.jar;D:\2025.9\Hadoop\test\4\lib2\jettison-1.1.jar;D:\2025.9\Hadoop\test\4\lib2\jsr305-3.0.2.jar;D:\2025.9\Hadoop\test\4\lib2\log4j-1.2.17.jar;D:\2025.9\Hadoop\test\4\lib2\okhttp-4.9.3.jar;D:\2025.9\Hadoop\test\4\lib2\dnsjava-2.1.7.jar;D:\2025.9\Hadoop\test\4\lib2\ehcache-3.3.1.jar;D:\2025.9\Hadoop\test\4\lib2\json-io-2.5.1.jar;D:\2025.9\Hadoop\test\4\lib2\objenesis-2.6.jar;D:\2025.9\Hadoop\test\4\lib2\paranamer-2.3.jar;D:\2025.9\Hadoop\test\4\lib2\guava-27.0-jre.jar;D:\2025.9\Hadoop\test\4\lib2\javax.inject-1.jar;D:\2025.9\Hadoop\test\4\lib2\snakeyaml-1.26.jar;D:\2025.9\Hadoop\test\4\lib2\aopalliance-1.0.jar;D:\2025.9\Hadoop\test\4\lib2\asm-commons-9.1.jar;D:\2025.9\Hadoop\test\4\lib2\commons-cli-1.2.jar;D:\2025.9\Hadoop\test\4\lib2\commons-net-3.6.jar;D:\2025.9\Hadoop\test\4\lib2\httpcore-4.4.13.jar;D:\2025.9\Hadoop\test\4\lib2\java-util-1.9.0.jar;D:\2025.9\Hadoop\test\4\lib2\jaxb-api-2.2.11.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-core-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-util-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerby-xdr-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\reload4j-1.2.22.jar;D:\2025.9\Hadoop\test\4\lib2\stax2-api-4.2.1.jar;D:\2025.9\Hadoop\test\4\lib2\zookeeper-3.5.6.jar;D:\2025.9\Hadoop\test\4\lib2\asm-analysis-9.1.jar;D:\2025.9\Hadoop\test\4\lib2\commons-io-2.8.0.jar;D:\2025.9\Hadoop\test\4\lib2\commons-text-1.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-kms-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-nfs-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\jersey-core-1.19.jar;D:\2025.9\Hadoop\test\4\lib2\jersey-json-1.19.jar;D:\2025.9\Hadoop\test\4\lib2\json-smart-2.4.7.jar;D:\2025.9\Hadoop\test\4\lib2\jsr311-api-1.1.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-admin-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerby-asn1-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerby-pkix-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerby-util-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\slf4j-api-1.7.36.jar;D:\2025.9\Hadoop\test\4\lib2\failureaccess-1.0.jar;D:\2025.9\Hadoop\test\4\lib2\guice-servlet-4.0.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-auth-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\httpclient-4.5.13.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-xc-1.9.13.jar;D:\2025.9\Hadoop\test\4\lib2\jaxb-impl-2.2.3-1.jar;D:\2025.9\Hadoop\test\4\lib2\jersey-guice-1.19.jar;D:\2025.9\Hadoop\test\4\lib2\json-simple-1.1.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-client-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-common-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-crypto-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-server-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\checker-qual-2.5.2.jar;D:\2025.9\Hadoop\test\4\lib2\commons-codec-1.15.jar;D:\2025.9\Hadoop\test\4\lib2\jersey-client-1.19.jar;D:\2025.9\Hadoop\test\4\lib2\jersey-server-1.19.jar;D:\2025.9\Hadoop\test\4\lib2\kerby-config-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\leveldbjni-all-1.8.jar;D:\2025.9\Hadoop\test\4\lib2\metrics-core-3.2.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-3.10.6.Final.jar;D:\2025.9\Hadoop\test\4\lib2\bcpkix-jdk15on-1.60.jar;D:\2025.9\Hadoop\test\4\lib2\bcprov-jdk15on-1.60.jar;D:\2025.9\Hadoop\test\4\lib2\commons-math3-3.1.1.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-common-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-core-2.12.7.jar;D:\2025.9\Hadoop\test\4\lib2\jersey-servlet-1.19.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-identity-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\protobuf-java-2.5.0.jar;D:\2025.9\Hadoop\test\4\lib2\snappy-java-1.1.8.2.jar;D:\2025.9\Hadoop\test\4\lib2\woodstox-core-5.3.0.jar;D:\2025.9\Hadoop\test\4\lib2\commons-lang3-3.12.0.jar;D:\2025.9\Hadoop\test\4\lib2\curator-client-4.2.0.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-jaxrs-1.9.13.jar;D:\2025.9\Hadoop\test\4\lib2\kerb-simplekdc-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\kotlin-stdlib-1.4.10.jar;D:\2025.9\Hadoop\test\4\lib2\token-provider-1.0.1.jar;D:\2025.9\Hadoop\test\4\lib2\zookeeper-jute-3.5.6.jar;D:\2025.9\Hadoop\test\4\lib2\accessors-smart-2.4.7.jar;D:\2025.9\Hadoop\test\4\lib2\commons-compress-1.21.jar;D:\2025.9\Hadoop\test\4\lib2\commons-daemon-1.0.13.jar;D:\2025.9\Hadoop\test\4\lib2\commons-logging-1.1.3.jar;D:\2025.9\Hadoop\test\4\lib2\curator-recipes-4.2.0.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-nfs-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-rbf-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-registry-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-api-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\HikariCP-java7-2.4.12.jar;D:\2025.9\Hadoop\test\4\lib2\mssql-jdbc-6.2.1.jre7.jar;D:\2025.9\Hadoop\test\4\lib2\nimbus-jose-jwt-9.8.1.jar;D:\2025.9\Hadoop\test\4\lib2\slf4j-reload4j-1.7.36.jar;D:\2025.9\Hadoop\test\4\lib2\j2objc-annotations-1.1.jar;D:\2025.9\Hadoop\test\4\lib2\jcip-annotations-1.0-1.jar;D:\2025.9\Hadoop\test\4\lib2\netty-all-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\commons-beanutils-1.9.4.jar;D:\2025.9\Hadoop\test\4\lib2\curator-framework-4.2.0.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-3.3.4-tests.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-core-asl-1.9.13.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-databind-2.12.7.jar;D:\2025.9\Hadoop\test\4\lib2\javax.servlet-api-3.1.0.jar;D:\2025.9\Hadoop\test\4\lib2\javax.websocket-api-1.0.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-annotations-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-client-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-httpfs-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-client-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-common-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\commons-collections-3.2.2.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-common-3.3.4-tests.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-shaded-guava-1.1.1.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-jaxrs-base-2.12.7.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-mapper-asl-1.9.13.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-io-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\netty-buffer-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-common-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\swagger-annotations-1.5.4.jar;D:\2025.9\Hadoop\test\4\lib2\audience-annotations-0.5.0.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-registry-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-annotations-2.12.7.jar;D:\2025.9\Hadoop\test\4\lib2\jakarta.xml.bind-api-2.3.2.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-xml-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\netty-handler-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-rbf-3.3.4-tests.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-http-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-jndi-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-plus-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-util-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\kotlin-stdlib-common-1.4.10.jar;D:\2025.9\Hadoop\test\4\lib2\netty-resolver-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\commons-configuration2-2.1.1.jar;D:\2025.9\Hadoop\test\4\lib2\jakarta.activation-api-1.2.1.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-dns-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-xml-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-client-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-server-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-webapp-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-http-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-mqtt-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-smtp-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-client-3.3.4-tests.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-tests-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-services-api-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\javax.websocket-client-api-1.0.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-servlet-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-http2-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-redis-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-socks-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-stomp-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\websocket-api-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\animal-sniffer-annotations-1.17.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-native-client-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-examples-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-common-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-router-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-services-core-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-security-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\netty-resolver-dns-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-hs-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-shaded-protobuf_3_7-1.1.1.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-util-ajax-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-haproxy-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-handler-proxy-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-udt-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-app-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-codec-memcache-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-rxtx-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-sctp-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\websocket-client-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\websocket-common-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\websocket-server-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-core-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-web-proxy-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-jaxrs-json-provider-2.12.7.jar;D:\2025.9\Hadoop\test\4\lib2\jetty-annotations-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\websocket-servlet-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\geronimo-jcache_1.0_spec-1.0-alpha-1.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-common-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-nodemanager-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-hdfs-native-client-3.3.4-tests.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-shuffle-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-uploader-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\jackson-module-jaxb-annotations-2.12.7.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-jobclient-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-hs-plugins-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-nativetask-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-applications-mawo-core-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-resourcemanager-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-classes-epoll-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-sharedcachemanager-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-classes-kqueue-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\javax-websocket-client-impl-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\javax-websocket-server-impl-9.4.43.v20210629.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-mapreduce-client-jobclient-3.3.4-tests.jar;D:\2025.9\Hadoop\test\4\lib2\netty-resolver-dns-classes-macos-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-applications-distributedshell-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-timeline-pluginstorage-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-native-unix-common-4.1.77.Final.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-server-applicationhistoryservice-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\hadoop-yarn-applications-unmanaged-am-launcher-3.3.4.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-native-kqueue-4.1.77.Final-osx-x86_64.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-native-epoll-4.1.77.Final-linux-x86_64.jar;D:\2025.9\Hadoop\test\4\lib2\netty-resolver-dns-native-macos-4.1.77.Final-osx-x86_64.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-native-kqueue-4.1.77.Final-osx-aarch_64.jar;D:\2025.9\Hadoop\test\4\lib2\netty-transport-native-epoll-4.1.77.Final-linux-aarch_64.jar;D:\2025.9\Hadoop\test\4\lib2\netty-resolver-dns-native-macos-4.1.77.Final-osx-aarch_64.jar;D:\2025.9\Hadoop\test\4\lib2\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar WordCount 2025-10-30 01:03:31,804 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(60)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2025-10-30 01:03:37,414 WARN [main] impl.MetricsConfig (MetricsConfig.java:loadFirst(136)) - Cannot locate configuration: tried hadoop-metrics2-jobtracker.properties,hadoop-metrics2.properties 2025-10-30 01:03:37,456 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(378)) - Scheduled Metric snapshot period at 10 second(s). 2025-10-30 01:03:37,456 INFO [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:start(191)) - JobTracker metrics system started Exception in thread "main" ExitCodeException exitCode=-1073741515: at org.apache.hadoop.util.Shell.runCommand(Shell.java:1007) at org.apache.hadoop.util.Shell.run(Shell.java:900) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1212) at org.apache.hadoop.util.Shell.execCommand(Shell.java:1306) at org.apache.hadoop.util.Shell.execCommand(Shell.java:1288) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:978) at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:660) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:700) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:672) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:699) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:677) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:336) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:162) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:113) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:148) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1571) at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1568) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1568) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1589) at WordCount.main(WordCount.java:28) 进程已结束,退出代码1
最新发布
10-31
这是一个典型的 **“代码型”** 问题：你提供了 Java 源码（`WordCount` MapReduce 程序）和运行日志，程序编译成功但执行失败，... 可通过 `HADOOP_USER_NAME` 系统属性指定运行用户，避免 UGI 获取当前 OS 用户失败。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值