Spring-batch（ItemReader）—数据读取从普通文件，数据库，XML，多文件数据读取

最新推荐文章于 2025-03-28 06:00:00 发布

weixin_34160277

最新推荐文章于 2025-03-28 06:00:00 发布

阅读量1.4k

点赞数

CC 4.0 BY-SA版权

文章标签：数据库 java 数据结构与算法

原文链接：http://blog.51cto.com/13501268/2298081

本文介绍了Spring-Batch中数据读取的方法，包括ItemReader接口的基本使用，从数据库、CSV文件、XML文件读取数据的示例，以及多文件读取和异常处理。示例涵盖了JDBCPagingItemReader、FlatFileItemReader、StaxEventItemReader等。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Spring-Batch学习总结（3）——如何数据输入
一．ItemReader概述
1.ItemReader:提供数据的接口
2.在这个接口中只有一个方法read(),它读取一个数据并且移动到下一个数据上去，在读取结束时必须返回一个null,否则表明数据没有读取完毕;
例：
OverViewApplication：

package com.dhcc.batch.batchDemo.input.overview;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
@EnableBatchProcessing
public class OverViewApplication {

    public static void main(String[] args) {
        SpringApplication.run(OverViewApplication.class, args);
    }
}

InputOverViewDemoJobConfiguration：

package com.dhcc.batch.batchDemo.input.overview;

import java.util.Arrays;
import java.util.List;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class InputOverViewDemoJobConfiguration {
    @Autowired
    private JobBuilderFactory jobBuilderFactory;
    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Bean
    public Job inputOverViewDemoJob() {
        return jobBuilderFactory.get("inputOverViewDemoJob").start(inputOverViewDemoJobStep()).build();

    }

    public Step inputOverViewDemoJobStep() {
        return stepBuilderFactory.get("inputOverViewDemoJobStep").<String, String>chunk(2)
                .reader(inputOverViewDemoReader()).writer(outputOverViewDemoWriter()).build();
    }

    private ItemWriter<? super String> outputOverViewDemoWriter() {
        return new ItemWriter<String>() {

            @Override
            public void write(List<? extends String> items) throws Exception {
                for (String item : items) {
                    System.out.println("output writer data: " + item);
                }
            }
        };
    }

    @Bean
    public InputOverVierDemoItemReader inputOverViewDemoReader() {
        List<String> data = Arrays.asList("dazhonghua", "xiaoriben", "meilijian", "falanxi", "deyizhi", "aierlan",
                "fandigang", "bajisitan", "baieluosi");
        return new InputOverVierDemoItemReader(data);
    }
}

InputOverVierDemoItemReader：

package com.dhcc.batch.batchDemo.input.overview;

import java.util.Iterator;
import java.util.List;

import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.NonTransientResourceException;
import org.springframework.batch.item.ParseException;
import org.springframework.batch.item.UnexpectedInputException;

public class InputOverVierDemoItemReader implements ItemReader<String> {
    private final Iterator<String> iterator;

    public InputOverVierDemoItemReader(List<String> data) {
        this.iterator = data.iterator();
    }

    @Override
    public String read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
        if (iterator.hasNext()) {
            return this.iterator.next();
        } else {
            return null;
        }
    }

}

运行结果：
Spring-batch（ItemReader）—数据读取从普通文件，数据库，XML，多文件数据读取
二．从数据库中读取数据
1.在实际应用中，我们都需要从数据库中读取数据，并且进行分页读取，在spring-batch中为我们提供了JDBCPagingItemReader这个类进行数据库数据读取
2.例：再举这个例子之前我们在数据库中建立了个person_buf表，并向表中插入了100001条数据

Spring-batch（ItemReader）—数据读取从普通文件，数据库，XML，多文件数据读取
接下来我们读取这个表中的数据为例，进行学习：
InputItemReaderJDBCApplication：

package com.dhcc.batch.batchDemo.input.db.jdbc;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
@EnableBatchProcessing
public class InputItemReaderJDBCApplication {
    public static void main(String[] args) {
        SpringApplication.run(InputItemReaderJDBCApplication.class, args);

    }
}

InputDBJdbcItemReaderConfigruation：

package com.dhcc.batch.batchDemo.input.db.jdbc;

import java.util.HashMap;
import java.util.Map;

import javax.sql.DataSource;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.database.JdbcPagingItemReader;
import org.springframework.batch.item.database.Order;
import org.springframework.batch.item.database.support.MySqlPagingQueryProvider;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class InputDBJdbcItemReaderConfigruation {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    @Qualifier("DBJdbcWriterDemo")
    private ItemWriter<? super Person> DBJbbcWriterDemo;

    @Autowired
    private DataSource dataSource;

    @Bean
    public Job DBJdbcItemReaderJob() {
        return jobBuilderFactory.get("DBJdbcItemReaderJob4")
                .start(DBJdbcItemReaderJobStep())
                .build();

    }

    @Bean
    public Step DBJdbcItemReaderJobStep() {
        return stepBuilderFactory.get("DBJdbcItemReaderJobStep4")
                .<Person, Person>chunk(100)
                .reader(DBJbbcReaderDemo())
                .writer(DBJbbcWriterDemo)
                .build();
    }

    @Bean
    @StepScope
    public JdbcPagingItemReader<Person> DBJbbcReaderDemo() {
        JdbcPagingItemReader<Person> reader = new JdbcPagingItemReader<>();
        reader.setDataSource(this.dataSource); // 设置数据源
        reader.setFetchSize(100); // 设置一次最大读取条数
        reader.setRowMapper(new PersonRowMapper()); // 把数据库中的每条数据映射到Person对中
        MySqlPagingQueryProvider queryProvider = new MySqlPagingQueryProvider();
        queryProvider.setSelectClause("id,name,per_desc,create_time,update_time,sex,score,price"); // 设置查询的列
        queryProvider.setFromClause("from person_buf"); // 设置要查询的表
        Map<String, Order> sortKeys = new HashMap<String, Order>();// 定义一个集合用于存放排序列
        sortKeys.put