Spark运行报错：ERROR CoarseGrainedExecutorBackend: Driver disassociated ! Shutting down

最新推荐文章于 2025-06-19 08:15:00 发布

原创

最新推荐文章于 2025-06-19 08:15:00 发布 · 9.8k 阅读

3 ·

CC 4.0 BY-SA版权

文章标签：

#spark #error

博客内容讲述了在Spark运行过程中遇到的一个错误：Driver disassociated，原因是内存不足。错误日志显示MemoryStore空间不足，Executor任务完成发送结果到driver。解决方案包括增加executor-memory或executor-overhead的值，建议优先考虑增加executor-memory来解决问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

报错：

INFO MemoryStore: Will not store rdd_4_5 as it would require dropping another block from the same RDD
WARN MemoryStore: Not enough space to cache rdd_4_5 in memory! (computed 68.8 MB so far)
INFO MemoryStore: Memory use = 302.6 KB (blocks) + 123.7 MB (scratch space shared across 3 tasks(s)) = 124.0 MB. Storage limit = 134.6 MB.
INFO Executor: Finished task 3.0 in stage 0.0 (TID 3). 2126 bytes result sent to driver
INFO Executor: Finished task 4.0 in stage 0.0 (TID 4). 2126 bytes result sent to driver
INFO Executor: Finished task 5.0 in stage 0.0 (TID 5). 2126 bytes result sent to driver
INFO CoarseGrainedExecutorBackend: Driver commanded a shutdown
INFO MemoryStore: MemoryStore cleared
INFO BlockManager: BlockManager stopped
WARN CoarseGrainedExecutorBackend: An unknown (leen:45970) driver disconnected.
INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.