data-x使用中常见报错

hcq_lxq

已于 2025-03-31 13:55:31 修改

阅读量380

点赞数 2

CC 4.0 BY-SA版权

分类专栏： ETL工具文章标签： etl

于 2025-03-31 13:54:51 首次发布

本文链接：https://blog.youkuaiyun.com/hcq_lxq/article/details/146820939

ETL工具专栏收录该内容

7 篇文章

订阅专栏

1、

java.io.IOException: Failed to flush data to StarRocks.
{"Status":"Fail","BeginTxnTimeMs":0,"Message":"too many filtered rows","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"ca799a25-8df7-4fb2-8902-fc33cd4c7b8e","LoadBytes":4452437,"StreamLoadPlanTimeMs":2,"NumberTotalRows":16758,"WriteDataTimeMs":214,"TxnId":96182,"LoadTimeMs":217,"ErrorURL":"http://xx.xx.xx.xx:xx/api/_load_error_log?file=error_log_9f44c846e8eb9ee8_b912703839eeeebc","ReadDataTimeMs":3,"NumberLoadedRows":16757,"NumberFilteredRows":1}
    at com.wgzhao.addax.plugin.writer.starrockswriter.manager.StarRocksStreamLoadVisitor.doStreamLoad(StarRocksStreamLoadVisitor.java:64)
    at com.wgzhao.addax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.asyncFlush(StarRocksWriterManager.java:162)
    at com.wgzhao.addax.plugin.writer.starrockswriter.manager.StarRocksWriterManager.access$000(StarRocksWriterManager.java:21)
    at com.wgzhao.addax.plugin.writer.starrockswriter.manager.StarRocksWriterManager$1.run(StarRocksWriterManager.java:133)
    at java.lang.Thread.run(Thread.java:748)
[36m2025-03-31 13:29:20.946[0;39m [32m[0-0-0-writer][0;39m [1;31mERROR[0;39m [35mWriterRunner        [0;39m - Writer Runner Received Exceptions:
com.wgzhao.datax.common.exception.DataXException: Code:[DBUtilErrorCode-05], Description:[往您配置的写入表中写入数据时失败.].  - java.lang.RuntimeException: Writing records to StarRocks failed

报错提示数据导入StarRocks失败，too many filtered rows，最后提示总条数NumberTotalRows":16758，加载成功16757，失败1条。所以执行失败。

解决思路：一般这种提示都是实际数据不符合要求，无法准确加载到目标库。此时检查源头数据看是否有些字段中存在空格或者回车之类的，对其字段进行替换处理regexp_replace(clom,'\t|\n','')