hive处理json和url

2019独角兽企业重金招聘Python工程师标准>>>

HIVE直接读入json

样例：表test,字段json_str存储了json字符串

{ "firstName": "Brett", "lastName":"McLaughlin", "email": "aaaa" }
{ "firstName": "jesse", "lastName":"AJ", "email": "bb" }

• get_json_object(jsonStr, kn)

        返回值: 一个key对应的一个字段

        说明：解析json的字符串jsonStr,若输入的json字符串无效，则返回NULL。

Hive处理语句：select get_json_object(json_str,'$.firstName') from test t

• json_tuple(jsonStr, k1, k2, ...)

        返回值：多个key对应的多个字段

        说明：比 get_json_object 高效，可以在一次调用中输入多个键

HIVE处理语句：select json_tuple(json_str,'firstName','lastName','email') from test t

 

 

HIVE直接读入url

样例：表test1,字段url_str存储了url字符串

http://192.168.111.23:8888/filebrowser/view=/DB/lefuwx/test/part-m-00000

• parse_url(url, partToExtract[, key])

        partToExtract选项[HOST,PATH,QUERY,REF,PROTOCOL,FILE,AUTHORITY,USERINFO]

Hive处理语句：select parse_url(url_str,'HOST') from .test1 t

• parse_url_tuple(url, k1,k2..)

        partToExtract选项[HOST,PATH,QUERY,REF,PROTOCOL,FILE,AUTHORITY,USERINFO]

Hive处理语句：select parse_url_tuple(url_str,'HOST','PATH') from test1 t

转载于:https://my.oschina.net/puwenchao/blog/876278