配置启动hadoop jobhistory

博客地址:http://www.fanlegefan.com
文章地址:http://www.fanlegefan.com/archives/hadoop-jobhistory

---

编辑mapred-site.xml

<property>
    <name>mapreduce.jobhistory.address</name>
    <value>0.0.0.0:10020</value>
</property>

<property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>0.0.0.0:19888</value>
</property>

<property>
    <name>mapreduce.jobhistory.done-dir</name>
    <value>${yarn.app.mapreduce.am.staging-dir}/history/done</value>
</property>

<property>
    <name>mapreduce.jobhistory.intermediate-done-dir</name>
    <value>${yarn.app.mapreduce.am.staging-dir}/history/done_intermediate</value>
</property>

<property>
    <name>yarn.app.mapreduce.am.staging-dir</name>
    <value>/tmp/hadoop-yarn/staging</value>
</property>

启动hadoop jobhistory

sbin/mr-jobhistory-daemon.sh start historyserver

history目录

hadoop dfs -ls /tmp/hadoop-yarn/staging/history/

Found 2 items
drwxrwx---   - work supergroup          0 2017-08-17 14:57 /tmp/hadoop-yarn/staging/history/done
drwxrwxrwt   - work supergroup          0 2016-05-30 15:00 /tmp/hadoop-yarn/staging/history/done_intermediate

Aggregation is not enabled. Try the nodemanager at localhost:54856

修改yarn-site.xml

<property>  
    <name>yarn.log-aggregation-enable</name>  
    <value>true</value>  
</property> 

重启yarn