Spark SQL连接Hive失败 org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

• 我的需求

  • 使用Spark连接Hive，访问Hive表
  • 语法是Spark SQL

• 上代码

  ConnectHiveDemo.scala

  /**
   * 本地虚拟机的地址是：10.211.55.101；
   * 需要根据自己的IP地址修改；9083是Hive的访问端口
   * 我的Hive表是t_data
   */
  object ConnectHiveDemo {
    def main(args: Array[String]): Unit = {
      //1.创建对象
      val spark: SparkSession = SparkSession.builder()
        .master("local[*]")
        .appName("app")
        .config("hive.metastore.uris", "thrift://10.211.55.101:9083")
        .enableHiveSupport()
        .getOrCreate()
      //2.执行SQL语句
      spark.sql("select * from t_data").show(10)
    }
  }
  

  POM.xml[注意版本问题]

      <properties>
          <spark.version>2.2.1</spark.version>
          <scala.version>2.11</scala.version>
      </properties>
  
      <dependencies>
          <dependency>
              <groupId>org.scala-lang</groupId>
              <artifactId>scala-compiler</artifactId>
              <version>2.11.8</version>
          </dependency>
  
          <dependency>
              <groupId>org.apache.spark</groupId>
              <artifactId>spark-core_2.11</artifactId>
              <version>${spark.version}</version>
          </dependency>
  
          <dependency>
              <groupId>org.apache.spark</groupId>
              <artifactId>spark-sql_2.11</artifactId>
              <version>${spark.version}</version>
          </dependency>
  
          <dependency>
              <groupId>org.apache.spark</groupId>
              <artifactId>spark-streaming_2.11</artifactId>
              <version>${spark.version}</version>
              <scope>provided</scope>
          </dependency>
  
          <dependency>
              <groupId>org.apache.spark</groupId>
              <artifactId>spark-hive_2.11</artifactId>
              <version>${spark.version}</version>
          </dependency>
  
          <dependency>
              <groupId>log4j</groupId>
              <artifactId>log4j</artifactId>
              <version>1.2.17</version>
          </dependency>
      </dependencies>
  
      <build>
          <sourceDirectory>src/main/scala</sourceDirectory>
          <testSourceDirectory>src/test/scala</testSourceDirectory>
          <plugins>
              <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-compiler-plugin</artifactId>
                  <version>3.1</version>
                  <configuration>
                      <source>1.8</source>
                      <target>1.8</target>
                      <encoding>utf-8</encoding>
                  </configuration>
              </plugin>
          </plugins>
      </build>=
  

• 报错定位

  Trying to connect to metastore with URI thrift://10.211.55.101:9083
  Failed to connect to the MetaStore Server...
  Failed to access metastore. This class should not accessed in runtime.
  org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
  

• 问题分析

  根据控制台提示的信息：访问Hive的元数据失败

  访问不了Hive的元数据？那怎么办？

  这里打一个比方，比如小明放学回家看电视，把插头插上，小明做在椅子上等了2分钟也没有看到电视有画面，为什么呢？

  是因为小明只是插上了电源，但是并没有用遥控器打开电视(缺少了一个步骤罢了)

  言归正传，意思是Hive的元数据访问不到，是因为我们没有打开访问的开关，怎么打开呢？

  # 首先进入到Hive的安装目录下
  # 我本地虚拟机的路径是/opt/module/hive/
  # 执行 bin/hive --service metastore &
  # 再次执行代码，出现预期结果
  

  

• 春江花朝秋月夜，往往取酒还独倾