hive和presto的求数组长度函数区别及注意事项

最新推荐文章于 2024-07-05 16:19:30 发布

Data_IT_Farmer

最新推荐文章于 2024-07-05 16:19:30 发布

阅读量2k

点赞数

文章标签： hive presto 求数组长度

本文链接：https://blog.youkuaiyun.com/helloxiaozhe/article/details/133927849

版权

文章讲述了在Hive和SparkSQL中使用`size`函数以及Presto中使用`cardinality`函数来获取邮箱字符串数组长度的方法，并指出两者对数组索引的处理不同。还提到Presto不支持双引号作为字符串引用。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1、任务

获取邮箱字符串’@'后字符串，求长度

2、hive & spark-sql 求数组长度的函数 size


hive & spark-sql 求数组长度的函数 size


select size(split(email, '@')),split(email, '@'),split(email, '@')[0],split(email, '@')[1]
FROM 
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid;

select size(split(email, '@')),split(email, '@'),split(email, '@')[0],split(email, '@')[1]
FROM 
(select 'jack@126.com' as email union select 'tom@126.com.cn' as email) tb_mid;


2	["tom","126.com.cn"]	tom	126.com.cn
2	["jack","126.com"]	jack	126.com
Time taken: 0.723 seconds, Fetched 2 row(s)

3、presto 求数组长度的函数 cardinality

presto  求数组长度的函数 cardinality

select cardinality(split(email, '@')),split(email, '@'),split(email, '@')[1],split(email, '@')[2]
FROM 
(select 'jack@126.com' as email union select 'tom@126.com.cn' as email) tb_mid;

_col0 |       _col1       | _col2 |   _col3    
-------+-------------------+-------+------------
     2 | [tom, 126.com.cn] | tom   | 126.com.cn 
     2 | [jack, 126.com]   | jack  | 126.com    
(2 rows)


select cardinality(split(email, '@')),split(email, '@'),split(email, '@')[1],split(email, '@')[2]
FROM 
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid;


Query 20231019_070945_20009_n9u2s failed: line 3:9: Column 'jack@126.com' cannot be resolved
select cardinality(split(email, '@')),split(email, '@'),split(email, '@')[1],split(email, '@')[2]
FROM
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid

4、注意事项

1）、在计算数组长度的时候，hive和presto的函数不同
其中hive的size函数默认数组的下标从0开始
presto的cardinality函数默认数组的下标从1开始

2）、presto 不支持双引号，而hive 既支持单引号，也支持双引号

presto> SELECT 
     -> email,
     -> (case when cardinality(split(email, '@')) = 2 then split(email, '@')[1] else '' end ) as email_suffix
     -> FROM 
     -> (select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid;
Query 20231016_070153_17958_p9f2s failed: line 5:9: Column 'jack@126.com' cannot be resolved
SELECT
email,
(case when cardinality(split(email, '@')) = 2 then split(email, '@')[1] else '' end ) as email_suffix
FROM
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid