环境:win7 64bit
hadoop: hadoop2.0.0-cdh3u6
spark2.3.1
问题现象:
Exception in thread "main" java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:868)...
附:(spark1.6中这个问题表现为URL找不到,其实还是classes are not found问题)
参考原因:
1. 依赖缺失: https://stackoverflow.com/questions/39444493/how-to-create-sparksession-with-hive-support-fails-with-hive-classes-are-not-f
2. cdh低版本不支持(我的根本原因) https://www.cloudera.com/documentation/enterprise/5-5-x/topics/admin_hos_config.html#hos_config__hos_enable_disable
spark官网介绍连接hive(没说cdh版本问题) http://spark.apache.org/docs/latest/sql-programming-guide.html