Callable实现并行查询数据库，增加查询效率

最新推荐文章于 2024-07-25 17:28:19 发布

原创最新推荐文章于 2024-07-25 17:28:19 发布 · 577 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#java #多线程

页面查询速度很慢，mysql数据库400w条数据，查看代码后发现有串行执行三次，查询，每次查询大约15s左右，加起来将近50多秒了，数据库sql先不优化，只优化代码

@Slf4j
public class ThreadPoolQueryUtil {

    /**
     * 创建线程池，最大线程数量3，任务队列最大数量100
     */
    static ThreadPoolExecutor fxThreadPool = new ThreadPoolExecutor(3, 6,
            0L, TimeUnit.MILLISECONDS,
            new LinkedBlockingQueue<Runnable>(100));


    public static Future<List<UserTransFileNumStatistics>> executionTask(PageReqDto reqDto, TransFileStatisticsServiceProvider transFileStatisticsServiceProvider,boolean realTimeQuery) {
        log.info("Start executing a query [{}]", reqDto.getFilter());
        Future taskResult = fxThreadPool.submit(
                new Callable() {
                    @Override
                    public List<UserTransFileNumStatistics> call() throws Exception {
                        log.info("Thread[{}]executionQueryTask", Thread.currentThread().getId());
                        if(realTimeQuery){
                            return transFileStatisticsServiceProvider.queryUserTypeTransFileNum(reqDto);
                        }
                        return transFileStatisticsServiceProvider.queryUserTypeTransFileNumByDayTable(reqDto);
                    }
                }
        );
        log.info("Result QueryTask ----------->");
        return taskResult;

    }
}

调用处代码（请求参数，需要深拷贝，防止线程共享）

 Future<List<UserTransFileNumStatistics>> noUserListByDay = ThreadPoolQueryUtil
                .executionTask(reqDtoputFilterConditons(reqDto, null), transFileStatisticsServiceProvider, isRealTimeQuery);

        Future<List<UserTransFileNumStatistics>> apiListByDay =
                ThreadPoolQueryUtil.executionTask(reqDtoputFilterConditons(reqDto, "2"), transFileStatisticsServiceProvider, isRealTimeQuery);

        Future<List<UserTransFileNumStatistics>> normalUserListByDay =
                ThreadPoolQueryUtil.executionTask(reqDtoputFilterConditons(reqDto, "3"), transFileStatisticsServiceProvider, isRealTimeQuery);

 List<UserTransFileNumStatistics> joinList = new ArrayList<>();
        //天表和实时表+起来
        try {
            joinList.addAll(noUserListByDay.get());
            joinList.addAll(apiListByDay.get());
            joinList.addAll(normalUserListByDay.get());
        } catch (Exception e) {
            log.error("Future Task geting exception : ", e);
            e.printStackTrace();
        }