一个由java.util.ConcurrentModificationException引起的血案_caused by: java.util.concurrentmodificationexcepti-优快云博客

本文链接：https://blog.youkuaiyun.com/wstcilct/article/details/117352907

本文分析了一起因`java.util.ConcurrentModificationException`导致的Flink任务线上故障。错误源于在`collect`操作中并发修改了List。通过调用栈分析，发现可能是Flink SDK在并发环境中对list进行了修改。虽然怀疑是`Collectors.toMap`冲突解决函数引起，但多线程测试排除了这一可能性，最终推测是Flink SDK在处理数据时并发修改了list导致的异常。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

线上跑的一个flink任务突然报了异常，话不多，先把异常发出来

最近在做一个flink项目时遇到了一个线上报错，话不多说，先上代码，其中用到的第三方封装的flink sdk隐去了包前缀，不会影响分析。

Caused by: java.util.ConcurrentModificationException
   at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1388)
   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
   at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
   at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
   at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
   at com.jd.uifs.operators.JimDBBatchSinkFunction.process(JimDBBatchSinkFunction.java:84)
   at xxx.flink.sdk.batchsink2.AbstractBatchSinkFunction$BatchProcessRunnable.run(AbstractBatchSinkFunction.java:218)
   at xxx.flink.sdk.batchsink2.AbstractBatchSinkFunction.processBatch(AbstractBatchSinkFunction.java:190)
   at xxx.flink.sdk.batchsink2.AbstractBatchSinkFunction.lambda$resetBatch$0(AbstractBatchSinkFunction.java:173)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

根据错误的提示，找到BatchSinkFunction的84行，为了方便大家了解上下文，把整段代码贴出来，同时注明84行所在位置

@Override
public void process(List<Tuple3<String, String,String>> inputs, IBatchProcessCallBack callBack) {
  
    if(inputs.size() == 0 ) {
        return;
    }
    Map<String,Map<String,Tuple2<Long,Integer>>> map1 = inputs.stream()
84行所在位置            .collect(Collectors.toMap(
                    tuple3 -> (tuple3.f0)
                    ,tuple3 -> {
                        Long timeStamp = LocalDateTime.parse(tuple3.f2.substring(0,14), formatter).toInstant(ZoneOffset.of("+8")).getEpochSecond();
                        Map<String,Tuple2<Long,Integer>> map2 = new HashMap<String,Tuple2<Long,Integer>>();
                        map2.put(tuple3.f1,new Tuple2<>(timeStamp,1));
                        return map2;
                    }
                    ,(v1,v2) -> {
                       v1.forEach((key,value)->{
                           if(v2.containsKey(key)) {