hadoop2.6.0执行自带wordcount出现异常

最新推荐文章于 2024-04-05 07:00:00 发布

原创

最新推荐文章于 2024-04-05 07:00:00 发布 · 1.1k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#centos #hadoop

在CentOS上使用Hadoop 2.6.0执行WordCount示例时遇到异常，通过脱离安全模式成功运行。详细日志显示了任务的执行过程，包括读写数据的字节数、任务阶段的时间消耗以及各种计数器信息。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

[root@namenode hadoop-2.6.0]# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount hdfsInput hdfsOutput

执行上述命令，结果如下

15/08/05 00:55:05 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
15/08/05 00:55:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
15/08/05 00:55:05 INFO input.FileInputFormat: Total input paths to process : 1
15/08/05 00:55:05 INFO mapreduce.JobSubmitter: number of splits:1
15/08/05 00:55:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1009436488_0001
15/08/05 00:55:06 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
15/08/05 00:55:06 INFO mapred.LocalJobRunner: OutputCommitter set in config null
15/08/05 00:55:06 INFO mapreduce.Job: Running job: job_local1009436488_0001
15/08/05 00:55:06 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
15/08/05 00:55:06 INFO mapred.LocalJobRunner: Waiting for map tasks
15/08/05 00:55:06 INFO mapred.LocalJobRunner: Starting task: attempt_local1009436488_0001_m_000000_0
15/08/05 00:55:06 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
15/08/05 00:55:06 INFO mapred.MapTask: Processing split: hdfs://namenode:9000/user/root/hdfsInput/test:0+45
15/08/05 00:55:06 INFO mapred.MapTask: (EQUATOR) 0 kvi 2621436(10485744)
15/08/05 00:55:06 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 10
15/08/05 00:55:06 INFO mapred.MapTask: soft limit at 8388608
15/08/05 00:55:06 INFO mapred.MapTask: bufstart = 0; bufvoid = 10485760
15/08/05 00:55:06 INFO mapred.MapTask: kvstart = 2621436; length = 655360
15/08/05 00:55:06 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
15/08/05 00:55:07 INFO mapred.LocalJobRunner:
15/08/05 00:55:07 INFO mapred.MapTask: Starting flush of map output
15/08/05 00:55:07 INFO mapred.MapTask: Spilling map output
15/08/05 00:55:07 INFO mapred.MapTask: bufstart = 0; bufend = 97; bufvoid = 10485760
15/08/05 00:55:07 INFO mapred.MapTask: kvstart = 2621436(10485744); kvend = 2621388(10485552); length = 49/655360
15/08/05 00:55:07 INFO mapred.MapTask: Finished spill 0
15/08/05 00:55:07 INFO mapred.Task: Task:attempt_local1009436488_0001_m_000000_0 is done. And is in the process of committing
15/08/05 00:55:07 INFO mapred.LocalJobRunner: map <