eror---at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init

本文记录了一次使用Hadoop MapReduce过程中遇到的NullPointerException错误详情,包括错误堆栈跟踪及作业失败状态。
2016-05-29 23:00:09,532 WARN  [Thread-12] mapred.LocalJobRunner (LocalJobRunner.java:run(560)) - job_local1841304029_0001
java.lang.Exception: java.lang.NullPointerException
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:986)
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:391)
    at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:80)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:675)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
2016-05-29 23:00:10,298 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1355)) - Job job_local1841304029_0001 running in uber mode : false
2016-05-29 23:00:10,302 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1362)) -  map 0% reduce 0%
2016-05-29 23:00:10,308 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1375)) - Job job_local1841304029_0001 failed with state FAILED due to: NA
2016-05-29 23:00:10,325 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1380)) - Counters: 0
[root@hadoop201 lib]# ls -l /opt/bdp/hive-4.0.1/lib/orc-*.jar -rw-r--r-- 1 root root 1157143 9月 25 2024 /opt/bdp/hive-4.0.1/lib/orc-core-1.8.5.jar -rw-r--r-- 1 root root 28998 9月 25 2024 /opt/bdp/hive-4.0.1/lib/orc-shims-1.8.5.jar -rw-r--r-- 1 root root 132316 9月 25 2024 /opt/bdp/hive-4.0.1/lib/orc-tools-1.8.5.jar [root@hadoop201 lib]# ls -l /opt/bdp/hive-4.0.1/lib/protobuf-java-*.jar -rw-r--r-- 1 root root 1838876 9月 25 2024 /opt/bdp/hive-4.0.1/lib/protobuf-java-3.24.4.jar [root@hadoop201 lib]# ls -l /opt/bdp/hadoop-3.4.0/share/hadoop/common/lib/protobuf-java-*.jar ls: 无法访问/opt/bdp/hadoop-3.4.0/share/hadoop/common/lib/protobuf-java-*.jar: 没有那个文件或目录 hadoop中没有这个jar包 , 向ORC表中插入数据报错:Diagnostic Messages for this Task: Error: java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:268) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:214) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:342) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:711) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:176) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:445) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:350) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:423) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
03-28
Error: org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many counters: 121 max=120 at org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:101) at org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:108) at org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:78) at org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:95) at org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:123) at org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:113) at org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:130) at org.apache.hadoop.mapred.Counters$Group.findCounter(Counters.java:369) at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:314) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:479) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:689) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:633) at org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.getCounter(TaskAttemptContextImpl.java:76) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.getCounter(WrappedMapper.java:101) at com.sina.ud.map.keyword.GameKeywordsMapMapper.map(GameKeywordsMapMapper.java:124) at com.sina.ud.map.keyword.GameKeywordsMapMapper.map(GameKeywordsMapMapper.java:27) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at com.sina.hadoop.DelegatingMapper.run(DelegatingMapper.java:34) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:243) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1750) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:237) 这段mr报错原因是啥
最新发布
07-11
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值