How-to: Enabled hive job running support on hive

最新推荐文章于 2025-08-02 10:19:30 发布

chenfangfang_2015

最新推荐文章于 2025-08-02 10:19:30 发布

阅读量952

点赞数

CC 4.0 BY-SA版权

文章标签：基础架构 spark hive HiveFromSpark

本文链接：https://blog.youkuaiyun.com/lulynn/article/details/47320749

本文解决了在使用 Spark 运行 Hive 示例时遇到的多个问题，包括配置文件 hive-site.xml 的正确路径设置、通过 spark-submit 命令指定 JAR 文件路径及内存选项，以及如何解决 Java SQL 异常等问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

There are multiple issues during running spark hive example HiveFromSpark:

Copy hive-site.xml to ${SPARK_HOME}/conf/
Correct command:
${SPARK_HOME}/bin/spark-submit --master yarn-cluster --driver-java-options "-XX:MaxPermSize=1G" --jars ../lib/datanucleus-api-jdo-3.2.1.jar,../lib/datanucleus-rdbms-3.2.1.jar,../lib/datanucleus-core-3.2.2.jar,../lib/mysql-connector-java-5.1.31-bin.jar --files ../conf/hive-site.xml --class org.apache.spark.examples.sql.hive.HiveFromSpark ../lib/spark-examples-1.3.0-cdh5.4.1-hadoop2.6.0-cdh5.4.1.jar
java.sql.SQLException: No suitable driver found for jdbc:mysql:
Add mysql-connector*.jar into hadoop classpath. One way is locate this jar in ${HADOOP_HOME}/share/hadoop/yarn/lib/
java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Driver":
To add driver memory size via increasing "--driver-java-options "-XX:MaxPermSize=1G"" option in spark-submit command.