flink之java.lang.NumberFormatException: For input string错误

最新推荐文章于 2023-10-30 21:36:33 发布

原创

最新推荐文章于 2023-10-30 21:36:33 发布 · 784 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#flink #hudi #spark

在尝试使用Flink从一个Hudi表读取数据并写入另一个Hudi表的过程中，出现了`NumberFormatException`，原因是源代码尝试将字符串test_table转换为长整型，但该操作不合法。此问题源于Flink读取表时定义了分区字段，而创建表时未指定分区。SparkSQL查询结果显示，分区路径包含非数字的字符串。

场景：

使用flink读取一张hudi表，将数据写入到另外一张hudi表。

错误栈：

java.lang.NumberFormatException: For input string: "test_table"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.valueOf(Long.java:803)
	at org.apache.hudi.util.DataTypeUtils.resolvePartition(DataTypeUtils.java:137)
	at org.apache.hudi.table.format.mor.MergeOnReadInputFormat.lambda$getReader$0(MergeOnReadInputFormat.java:317)
	at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
	at org.apache.hudi.table.format.mor.MergeOnReadInputFormat.getReader(MergeOnReadInputFormat.java:306)
	at org.apache.hudi.table.format.mor.MergeOnReadInputFormat.getFullSchemaReader(MergeOnReadInputFormat.java:292)
	at org.apache.hudi.table.format.mor.MergeOnReadInputFormat.open(MergeOnReadInputFormat.java:209)
	at org.apache.hudi.source.StreamReadOperator.processSplits(StreamReadOperator.java:163)
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
	at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
	at org.apache.flink.streaming.runtime.tasks.mailbox.    .processMailsNonBlocking(MailboxProcessor.java:353)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:201)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:835)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:784)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:955)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:934)
	at org.apache.flink.runti

最低0.47元/天解锁文章