【异常】SparkStreaming长时间运行出现： Dropping SparkListenerEvent because no remaining room in event queue

问题描述

在SparkStreaming任务提交后，经过长时间的运行会在SparkUI的Jobs界面出现Job的执行时间出现几个小时的情况，点进去发现Stage已经处于unkown状态，如下图：

查看日志后发现下面ERROR异常：

2018-01-06 16:42:03  [ dag-scheduler-event-loop:600497 ] - [ ERROR ]  Dropping SparkListenerEvent because no remaining room in event queue. This likely means one of the SparkListeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.

问题分析

搜了一些关于消