hadoop2.2.0 mapred-site.xml

最新推荐文章于 2022-04-29 21:02:00 发布

weixin_30699955

最新推荐文章于 2022-04-29 21:02:00 发布

阅读量69

点赞数

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/weian404/p/3586068.html

本文详细介绍了MapReduce中关于作业历史记录的几个关键配置项，包括作业历史文件的存储位置、任务进度划分的数量、特定作业的历史记录存放位置以及已完成作业历史文件的位置等。此外还涉及了作业是否需要设置清理操作的配置。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

<property>
  <name>mapreduce.jobtracker.jobhistory.location</name>
  <value></value>
  <description> If job tracker is static the history files are stored 
  in this single well known place. If No value is set here, by default,
  it is in the local file system at ${hadoop.log.dir}/history.
  </description>
</property>

注释：

<property>
  <name>mapreduce.jobtracker.jobhistory.task.numberprogresssplits</name>
  <value>12</value>
  <description> Every task attempt progresses from 0.0 to 1.0 [unless
  it fails or is killed].  We record, for each task attempt, certain 
  statistics over each twelfth of the progress range.  You can change
  the number of intervals we divide the entire range of progress into
  by setting this property.  Higher values give more precision to the
  recorded data, but costs more memory in the job tracker at runtime.
  Each increment in this attribute costs 16 bytes per running task.
  </description>
</property>

注释：

<property>
  <name>mapreduce.job.userhistorylocation</name>
  <value></value>
  <description> User can specify a location to store the history files of 
  a particular job. If nothing is specified, the logs are stored in 
  output directory. The files are stored in "_logs/history/" in the directory.
  User can stop logging by giving the value "none". 
  </description>
</property>

注释：

<property>
  <name>mapreduce.jobtracker.jobhistory.completed.location</name>
  <value></value>
  <description> The completed job history files are stored at this single well 
  known location. If nothing is specified, the files are stored at 
  ${mapreduce.jobtracker.jobhistory.location}/done.
  </description>
</property>

注释：

<property>
  <name>mapreduce.job.committer.setup.cleanup.needed</name>
  <value>true</value>
  <description> true, if job needs job-setup and job-cleanup.
                false, otherwise  
  </description>
</property>