spark-sql-readme

最新推荐文章于 2024-08-23 07:45:00 发布

原创最新推荐文章于 2024-08-23 07:45:00 发布 · 448 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#sql #readme

spark 专栏收录该内容

23 篇文章

订阅专栏

本文详细阐述了Spark SQL模块的四个子项目：Catalyst、Execution、HiveSupport和HiveServer & CLI支持，重点介绍了Catalyst和Execution组件的功能与作用，以及它们在Spark生态系统中的地位。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

spark-sql-readme

@(spark)[sql][readme]

This module provides support for executing relational queries expressed in either SQL or a LINQ-like Scala DSL.

Spark SQL is broken up into four subprojects:
- Catalyst (sql/catalyst) - An implementation-agnostic framework for manipulating trees of relational operators and expressions.
- Execution (sql/core) - A query planner / execution engine for translating Catalyst’s logical query plans into Spark RDDs. This component also includes a new publiciinterface, SQLContext, that allows users to execute SQL or LINQ statements against existing RDDs and Parquet files. - Hive Support (sql/hive) - Includes an extension of SQLContext called HiveContext that allows users to write queries using a subset of HiveQL and access data from a Hive Metastore using Hive SerDes. There are also wrappers that allows users to run queries that include Hive UDFs, UDAFs, and UDTFs.
- HiveServer and CLI support (sql/hive-thriftserver) - Includes support for the SQL CLI (bin/spark-sql) and a HiveServer2 (for JDBC/ODBC) compatible server.

限于精力和水平，重点关注Catalyst和Execution。