Flinksql bug : Temporal table join requires an equality condition on fields of table

Direction_Wind

已于 2024-01-15 10:51:10 修改

阅读量1.6k

点赞数 12

CC 4.0 BY-SA版权

文章标签： bug flink

于 2023-12-05 15:51:04 首次发布

本文链接：https://blog.youkuaiyun.com/Direction_Wind/article/details/134809353

flink/hive/iceberg 报错调试记录专栏收录该内容

16 篇文章 ¥19.90 ¥99.00

订阅专栏

超级会员免费看

在使用Flink SQL进行维表关联时遇到错误：Temporal table join需要等值条件。错误出现在特定的流式join操作与Hive表关联的场景。解决方案是确保关联条件为等号连接，且两侧数据类型一致。然而，实践中发现，即使对数字内容的字符串字段进行类型转换（如cast(cast(field as bigint) as string)），也会导致错误。正确的做法是将int类型字段直接cast为string，然后与string类型字段进行关联。

报错信息

org.apache.flink.table.api.TableException: Temporal table join requires an equality condition on fields of table *******************

发生地址

flinksql 与维表关联场景包括hive的流式join

解决

关联条件必须等值关联也就是必须是等号，并且关联条件两边要确定数据类型相同

实测

如果关联条件都是字符串，内容是数字，两测都强制类型转换，cast（feild as bigint）会报错。很奇怪做了强制类型转换反而报错了
不支持已经是int的类型再次cast(int类型字段 as int) 与 cast(string类型字段 as int) 做关联必须 string类型字段 = cast(int类型字段 as string) 这么做

报错！！！！
left join  hive_zjyprc_hadoop.china_bi.dim_sku_great_bargains /*+ OPTIONS('streaming-source.enable' = 'true','streaming-source.partition.include' = 'latest',
            'streaming-source.partition-order' = 'create-time','streaming-source.monitor-interval' = '1 h') */
             FOR SYSTEM_TIME AS OF w.proctime AS w3  on cast(w.category_id as bigint) = cast(w3.virtual_category_id as bigint)

正确!!!            left join  hive_zjyprc_hadoop.china_bi.dim_sku_great_bargains /*+ OPTIONS('streaming-source.enable' = 'true','streaming-source.partition.include' = 'latest',
            'streaming-source.partition-order' = 'create-time','streaming-source.monitor-interval' = '1 h') */
             FOR SYSTEM_TIME AS OF w.proctime AS w3  on w.category_id = cast(w3.virtual_category_id