starrocks采用streamload导入数据

最新推荐文章于 2025-06-04 09:07:05 发布

原创

最新推荐文章于 2025-06-04 09:07:05 发布 · 1k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#java

本文档介绍了如何通过Java配置和代码实现对StarRocks的Streamload数据导入，帮助读者理解并掌握这一过程。

1、依赖
<dependency>
    <groupId>com.starrocks</groupId>
    <artifactId>flink-connector-starrocks</artifactId>
    <version>1.2.1_flink-1.14_2.12</version>
</dependency>

2、配置

@Configuration
@ConfigurationProperties(prefix = "star")
@Data
@Slf4j
public class StarRocksConfiguration {
    private String url;
    private String port;
    private String format;
    private String user;
    private String pass;
    private String database;
    private String labelPrefix = "streamLoad";
    private String rowSep = "&";
    private Integer retryLimit;
    private Integer interval;
}

3、代码

@Override
public Map<String, Object> write(String tableName, List<String> records) throws Exception {
    StarRocksSinkBuffer bufferEntity = new StarRocksSinkBuffer(starRocksConfiguration.getDatabase(), tableName, starRocksConfiguration.getLabelPrefix());
    for (String record : records) {
        byte[] bts = record.getBytes(StandardCharsets.UTF_8);
        bufferEntity.addToBuffer(bts);
    }
    byte[] data = joinRows(bufferEntity.getBuffer(), (int) bufferEntity.getBatchSize());
    String loadUrl = new StringBuilder(starRocksConfiguration.getUrl())
            .append(":")
            .append(starRocksConfigu