Spark-Doris-Connecter实战

最新推荐文章于 2024-11-05 17:44:13 发布

qq_43545096

最新推荐文章于 2024-11-05 17:44:13 发布

阅读量899

点赞数 1

文章标签：大数据

本文链接：https://blog.youkuaiyun.com/qq_43545096/article/details/128875248

版权

在尝试使用Spark连接Doris时，用户遇到了一个错误：UnrecognizedPropertyException，该异常指出Schema类中无法识别fieldkeysType。错误响应包含了一个Doris表的schema信息。更新到最新版本的Spark-DorisConnector（1.1.0）解决了此问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

#Spark-Doris-Connecter实战
报错


23/02/03 19:49:47 ERROR SchemaUtils: Doris FE's response cannot map to schema. res: {"keysType":"UNIQUE_KEYS","properties":[{"name":"full_type_id","aggregation_type":"","comment":"全目录ID","type":"VARCHAR"},{"name":"full_type_name","aggregation_type":"REPLACE","comment":"全目录名称","type":"VARCHAR"},{"name":"first_type_id","aggregation_type":"REPLACE","comment":"一级目录ID","type":"VARCHAR"},{"name":"first_type_name","aggregation_type":"REPLACE","comment":"一级目录名称","type":"VARCHAR"},{"name":"second_type_id","aggregation_type":"REPLACE","comment":"二级目录ID","type":"VARCHAR"},{"name":"second_type_name","aggregation_type":"REPLACE","comment":"二级目录名称","type":"VARCHAR"},{"name":"third_type_id","aggregation_type":"REPLACE","comment":"三级目录ID","type":"VARCHAR"},{"name":"third_type_name","aggregation_type":"REPLACE","comment":"三级目录名称","type":"VARCHAR"},{"name":"fourth_type_id","aggregation_type":"REPLACE","comment":"四级目录ID","type":"VARCHAR"},{"name":"fourth_type_name","aggregation_type":"REPLACE","comment":"四级目录名称","type":"VARCHAR"}],"status":200}
org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "keysType" (Class org.apache.doris.spark.rest.models.Schema), not marked as ignorable

使用最新版本的jar会好

        <dependency>
            <groupId>org.apache.doris</groupId>
            <artifactId>spark-doris-connector-2.3_2.11</artifactId>
            <version>1.1.0</version>
        </dependency>