1.sparksql读取sql的相关处理操作

qq_35561207

于 2019-11-12 20:35:04 发布

阅读量531

点赞数

分类专栏：大数据组件

本文链接：https://blog.youkuaiyun.com/qq_35561207/article/details/103038062

版权

本文介绍了如何使用SparkSQL读取和处理JSON数据，包括解析JSON文件、查询JSON字段以及进行相关数据操作。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

19/11/12 20:32:53 INFO CodeGenerator: Code generated in 12.9619 ms
19/11/12 20:32:53 INFO CodeGenerator: Code generated in 11.6207 ms
19/11/12 20:32:53 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 220.0 KB, free 4.1 GB)
19/11/12 20:32:53 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 20.6 KB, free 4.1 GB)
19/11/12 20:32:53 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on dst001825.cn1.global.ctrip.com:53907 (size: 20.6 KB, free: 4.1 GB)
19/11/12 20:32:53 INFO SparkContext: Created broadcast 2 from show at sparksql_json.scala:13
19/11/12 20:32:53 INFO FileSourceScanExec: Planning scan with bin packing, max size: 4194377 bytes, open cost is considered as scanning 4194304 bytes.
19/11/12 20:32:53 INFO SparkContext: Starting job: show at sparksql_json.scala:13
19/11/12 20:32:53 INFO DAGScheduler: Got job 1 (show at sparksql_json.scala:13) with 1 output partitions

最低0.47元/天解锁文章