批量导入数据（saveBatch、并发编程线程池）

一个一个插入

这是一开始的代码：

@Component
public class insertUsers {
    @Resource
    private userMapper userMapper;


    public void doInsertUsers(){
        final int INSERT_NUM=50000;
        for (int i=0;i<INSERT_NUM;i++){
            user user=new user();
            user.setUsername("lilililili");
            user.setUserAccount("lilililili");
            user.setTags("");
            user.setAvatarUrl("");
            user.setGender(0);
            user.setPassword("12345678");
            user.setUserRole(0);
            user.setPhone("");
            user.setEmail("");
            user.setUserStatus(0);
            userMapper.insert(user);

        }

    }

    public static void main(String[] args) {
        long start = System.currentTimeMillis();
        ConfigurableApplicationContext context = SpringApplication.run(Usercenter2Application.class, args);
        insertUsers insertUsers = context.getBean(insertUsers.class);
        insertUsers.doInsertUsers();
        long end = System.currentTimeMillis();
        System.out.println("运行时间：" + (end - start) + " ms");
    }

}

运行时间：12676 ms

这种方式是比较慢的，每一次insert都是一次独立的数据库操作。

使用saveBatch

MyBatis-Plus 已经提供了高性能的批量插入方法。也就是saveBatch

要保证用的是IService接口

使用saveBatch的代码如下：

@Component
public class insertUsers {
    @Resource
    private userMapper userMapper;

    @Resource
    private userService userService;

    public void doInsertUsers(){
        List<user> userList=new ArrayList<>();
        final int INSERT_NUM=50000;
        for (int i=0;i<INSERT_NUM;i++){
            user user=new user();
            user.setUsername("lilililili");
            user.setUserAccount("lilililili");
            user.setTags("");
            user.setAvatarUrl("");
            user.setGender(0);
            user.setPassword("12345678");
            user.setUserRole(0);
            user.setPhone("");
            user.setEmail("");
            user.setUserStatus(0);
            userList.add(user);
        }
        userService.saveBatch(userList, 1000);

    }



    public static void main(String[] args) {
        long start = System.currentTimeMillis();
        ConfigurableApplicationContext context = SpringApplication.run(Usercenter2Application.class, args);
        insertUsers insertUsers = context.getBean(insertUsers.class);
        insertUsers.doInsertUsers();
        long end = System.currentTimeMillis();
        System.out.println("运行时间：" + (end - start) + " ms");
    }

}

运行时间：5385 ms

用并发编程、线程池

@Component
public class insertUsers {
    @Resource
    private userMapper userMapper;

    @Resource
    private userService userService;

    public void doInsertUsers(){
        List<user> userList=new ArrayList<>();
        final int INSERT_NUM=50000;
        final int THREAD_COUNT=10;
        final int BATCH_SIZE=1000;
        ExecutorService executorService = Executors.newFixedThreadPool(INSERT_NUM);
        for (int i=0;i<INSERT_NUM;i++){
            user user=new user();
            user.setUsername("lilililili");
            user.setUserAccount("lilililili");
            user.setTags("");
            user.setAvatarUrl("");
            user.setGender(0);
            user.setPassword("12345678");
            user.setUserRole(0);
            user.setPhone("");
            user.setEmail("");
            user.setUserStatus(0);
            userList.add(user);
        }
        int batch_count=(int) Math.ceil((double) INSERT_NUM / BATCH_SIZE);
        for (int i=0;i<batch_count;i++){
            int start=i*BATCH_SIZE;
            int end=Math.min(INSERT_NUM,start+BATCH_SIZE);
            List<user> subList=userList.subList(start,end);
            executorService.execute(()->userService.saveBatch(subList));
        }
        executorService.shutdown();
        try {
            executorService.awaitTermination(1, TimeUnit.HOURS);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }


    }



    public static void main(String[] args) {
        long start = System.currentTimeMillis();
        ConfigurableApplicationContext context = SpringApplication.run(Usercenter2Application.class, args);
        insertUsers insertUsers = context.getBean(insertUsers.class);
        insertUsers.doInsertUsers();
        long end = System.currentTimeMillis();
        System.out.println("运行时间：" + (end - start) + " ms");
    }

}

运行时间：2757 ms

其中，

executorService.execute(() -> userService.saveBatch(subList));

是java中使用线程池提交一个任务的写法。

executorService是一个线程池。.execute()方法用于向线程池提交一个任务，会给这个任务分配一个空闲线程。

()->userService.saveBatch(subList)是一个lamda表达式，表示要执行的任务。

()->...是一个简化写法，表示传入一个没有参数的方法体。

适合插入先后顺序没什么影响的情况。