Reducer preempted to make room for pending map attempts Container killed by the ApplicationMaster

hive mr任务

Reducer preempted to make room for pending map attempts Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143。

 

大量map在pending，running中的为0，此时reduce在copy阶段把资源耗完，导致map没资源去跑，进而导致reduce一直在copy状态等待。也就是说map需要资源去跑，reduce需要等map全部跑完才能进行下一个阶段，这样就导致相互等待，类似死锁

 

解决方案

set  mapreduce.job.reduce.slowstart.completedmaps =1