spark集成Hive报错error: not found: value sqlContext

最新推荐文章于 2020-03-06 10:04:39 发布

原创最新推荐文章于 2020-03-06 10:04:39 发布 · 1.2k 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#spark #hive

本文详细介绍了在使用Spark时遇到的“找不到MySQL驱动”错误，并提供了具体的解决方案，包括如何在启动Spark时指定正确的MySQL驱动路径。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
  at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:259)
  at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:131)
  at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFactoryImpl.java:85)
  ... 143 more
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
  at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
  at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
  at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238)
  ... 145 more
<console>:14: error: not found: value spark
       import spark.implicits._
              ^
<console>:14: error: not found: value spark
       import spark.sql
              ^
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.1
      /_/
         
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144)
Type in expressions to have them evaluated.
Type :help for more information.

仔细观察日志开始以为是Hive的配置文件问题，可是Hive使用正常，又检查的mysql(元数据库)也是正常的，结果是mysql驱动没有被读取，启动spark时指定驱动可以正常启动。

[spark-2.1.1-bin-hadoop2.7]$ bin/spark-shell --jars /opt/software/mysql-libs/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar //你自己的mysql驱动

然后就可以正常启动了！