初始化hive元数据

最新推荐文章于 2024-07-20 14:06:12 发布

原创最新推荐文章于 2024-07-20 14:06:12 发布 · 1.2k 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#hive #hadoop #数据仓库

文章描述了在执行Hive的schematool命令时遇到的SSL通信异常，提示存在多个SLF4J绑定和SSL协议问题。通过添加配置到hive-site.xml，将useSSL设置为false来禁用SSL，解决了连接失败的问题。

第一：执行命令

   bin/schematool -dbType mysql -initSchema

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive-3.1.3/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop-3.3.3/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:   jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :   com.mysql.jdbc.Driver
Metastore connection User:   root
Tue Jun 27 15:09:50 CST 2023 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure

第二：增加 --verbose 参数

bin/schematool -dbType mysql -initSchema --verbose

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
   at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:94)
   at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169)
   at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475)
   at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581)
   at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)
   at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 176 milliseconds ago. The last packet sent successfully to the server was 170 milliseconds ago.
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
   at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:988)
   at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:164)
   at com.mysql.jdbc.MysqlIO.negotiateSSLConnection(MysqlIO.java:4894)
   at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1661)
   at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1228)
   at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2253)
   at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2284)
   at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2083)
   at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:806)
   at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
   at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:410)
   at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:328)
   at java.sql.DriverManager.getConnection(DriverManager.java:664)
   at java.sql.DriverManager.getConnection(DriverManager.java:247)
   at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)
   ... 11 more
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
   at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
   at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:106)
   at sun.security.ssl.TransportContext.kickstart(TransportContext.java:238)
   at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:405)
   at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:384)
   at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:149)

第三：问题排查

可能和SSL有关

第四：解决方案

在hive-site.xml 中增加配置

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>
</property>