yarn日志聚合historyserver参数汇总

1）yarn.log-aggregation-enable：
是否开启日志聚合。日志聚合会在应用程序完成后收集每个容器的日志，并将这些日志移动到文件系统中，例如 HDFS。 用户可以配置“yarn.nodemanager.remote-app-log-dir”和“ yarn.nodemanager.remote-app-log-dir-suffix”属性来确定将这些日志移至何处。 用户可以通过Application Timeline Server访问日志。

yarn.log-aggregation-enable false

2）yarn.nodemanager.remote-app-log-dir：
聚合日志保存路径（HDFS文件系统）

Where to aggregate logs to. yarn.nodemanager.remote-app-log-dir /tmp/logs

3）yarn.nodemanager.remote-app-log-dir-suffix：
和2)一起组成聚合日志完整保存路径。即聚合日志存放的真正路径是{yarn.nodemanager.remote-app-log-dir}/$user/yarn.nodemanager.remote-app-log-dir-suffix，默认为/tmp/logs/${user}/logs。

The remote log dir will be created at {yarn.nodemanager.remote-app-log-dir}/${user}/{thisParam} yarn.nodemanager.remote-app-log-dir-suffix logs

4）yarn.log-aggregation.retain-seconds：
聚合日志保存时间（默认-1为不删除）

How long to keep aggregation logs before deleting them. -1 disables. Be careful set this too small and you will spam the name node. yarn.log-aggregation.retain-seconds -1

5）yarn.log-aggregation.retain-check-interval-seconds：
聚合日志保存检查间隔时间。两次聚合日志保存检查之间的等待时间。如果设置为0或负值，则该值将被设置为聚合日志保存时间的十分之一。

yarn.log-aggregation.retain-check-interval-seconds -1

6）yarn.log.server.url：
自动跳转到的聚合日志的网页地址（jobhistory节点）

URL for log aggregation server yarn.log.server.url

7）yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds：
Nodemanager上传日志文件的间隔时间（默认-1表示在应用运行结束后再上传，最小可被设置为3600s。）

yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds -1

8）yarn.nodemanager.log-dirs：
容器日志本地保存路径

Where to store container logs. An application's localized log directory will be found in ${yarn.nodemanager.log-dirs}/application_${appid}. Individual containers' log directories will be below this, in directories named container_{$contid}. Each container directory will contain the files stderr, stdin, and syslog generated by that container. yarn.nodemanager.log-dirs ${yarn.log.dir}/userlogs

• 默认值为${yarn.log.dir}/userlogs

• 一个应用的本地化日志目录是这样的格式：{ yarn.nodemanager.log-dirs }/application_{appid}

• 一个独立的容器的日志文件夹会在上面的文件夹下，文件夹命名格式是这样：container_${containerid}

• 对于MapReduce的应用，每个容器目录下会包含容器生成的三个文件：stderr，stdin和syslog。

9）yarn.nodemanager.log.retain-seconds：
容器日志保留时间（在聚合日志功能启用后失效）

Time in seconds to retain user logs. Only applicable if log aggregation is disabled yarn.nodemanager.log.retain-seconds 10800

10）yarn.nodemanager.log.deletion-threads-count：
在Nodemanager中启动多少线程数去删除日志（在聚合日志功能启用后失效）

Number of threads to use in NM log cleanup. Used when log aggregation is disabled. yarn.nodemanager.log.deletion-threads-count 4

11）yarn.nodemanager.delete.debug-delay-sec：
应用结束后DeletionService删除日志需等待的时间

Number of seconds after an application finishes before the nodemanager's DeletionService will delete the application's localized file directory and log directory.

To diagnose Yarn application problems, set this property's value large
enough (for example, to 600 = 10 minutes) to permit examination of these
directories. After changing the property's value, you must restart the 
nodemanager in order for it to have an effect.

The roots of Yarn applications' work directories is configurable with
the yarn.nodemanager.local-dirs property (see below), and the roots
of the Yarn applications' log directories is configurable with the 
yarn.nodemanager.log-dirs property (see also below).

yarn.nodemanager.delete.debug-delay-sec 0