原文链接:http://www.cnblogs.com/zwgblog/p/6063993.html
启动hive-metastore和hive-server2 用beeline连接hive报错
1
2
3
4
5
6
7
8
|
[root @node04 hive]#
beeline Beeline
version 0.13 . 1 -cdh5. 3.0 by
Apache Hive beeline>
!connect jdbc:hive2: //172.16.145.114:10000
hive scan
complete in 3ms Connecting
to jdbc:hive2: //172.16.145.114:10000 Enter
password for jdbc:hive2: //172.16.145.114:10000:
**** Error:
Could not open connection to jdbc:hive2: //172.16.145.114:10000:
java.net.ConnectException: Connection refused (state=08S01,code=0) 0 :
jdbc:hive2: //172.16.145.114:10000
(closed)> !connect jdbc:hive2://172.16.145.114:10000 hive |
确认已经赋予了hive用户在当前主机链接数据库的权限:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
mysql>
use mysql; Reading
table information for completion
of table and column names You
can turn off this feature
to get a quicker startup with -A Database
changed mysql>
select host,user from user; +-----------+------+ |
host | user | +-----------+------+ |
% | hive | | 127.0 . 0.1 |
root | |
localhost | hive | |
localhost | root | |
node04 | root | +-----------+------+ 5 rows
in set ( 0.00 sec) |
查看metastore的日志,报错信息如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
|
... 47 more 2016 - 11 - 14 16 : 10 : 00 , 881 ERROR
[main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>( 135 ))
- Self-test query [select "DB_ID" from "DBS" ]
failed; direc t
SQL is disabled javax.jdo.JDODataStoreException:
Error executing SQL query "select
" DB_ID "
from " DBS "" . at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java: 451 ) at
org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java: 230 ) at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java: 131 ) at
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java: 253 ) at
org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java: 224 ) at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java: 73 ) at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java: 133 ) at
org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java: 58 ) at
org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java: 67 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java: 506 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java: 484 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java: 532 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java: 406 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java: 365 ) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java: 55 ) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java: 60 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java: 4953 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java: 5174 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java: 5093 ) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 57 ) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 43 ) at
java.lang.reflect.Method.invoke(Method.java: 606 ) at
org.apache.hadoop.util.RunJar.main(RunJar.java: 212 ) NestedThrowablesStackTrace: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:
Table 'metastore.DBS' doesn't
exist at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java: 57 ) at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java: 45 ) at
java.lang.reflect.Constructor.newInstance(Constructor.java: 526 ) at
com.mysql.jdbc.Util.handleNewInstance(Util.java: 411 ) at
com.mysql.jdbc.Util.getInstance(Util.java: 386 ) at
com.mysql.jdbc.SQLError.createSQLException(SQLError.java: 1052 ) at
com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java: 3597 ) at
com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java: 3529 ) at
com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java: 1990 ) at
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java: 2151 ) at
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java: 2625 ) at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java: 2119 ) at
com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java: 2283 ) at
com.jolbox.bonecp.PreparedStatementHandle.executeQuery(PreparedStatementHandle.java: 174 ) at
org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeQuery(ParamLoggingPreparedStatement.java: 381 ) at
org.datanucleus.store.rdbms.SQLController.executeStatementQuery(SQLController.java: 504 ) at
org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java: 280 ) at
org.datanucleus.store.query.Query.executeQuery(Query.java: 1786 ) at
org.datanucleus.store.query.AbstractSQLQuery.executeWithArray(AbstractSQLQuery.java: 339 ) at
org.datanucleus.store.query.Query.execute(Query.java: 1654 ) at
org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java: 221 ) at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java: 131 ) at
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java: 253 ) at
org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java: 224 ) at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java: 73 ) at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java: 133 ) at
org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java: 58 ) at
org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java: 67 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java: 506 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java: 484 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java: 532 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java: 406 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java: 365 ) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java: 55 ) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java: 60 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java: 4953 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java: 5174 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java: 5093 ) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 57 ) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 43 ) at
java.lang.reflect.Method.invoke(Method.java: 606 ) at
org.apache.hadoop.util.RunJar.main(RunJar.java: 212 ) 2016 - 11 - 14 16 : 10 : 01 , 019 WARN
[main]: metastore.ObjectStore (ObjectStore.java:checkSchema( 6304 ))
- Version information not found in metastore. hive.metastore.schema .verification
is not enabled so recording the schema version 0.13 . 0 2016 - 11 - 14 16 : 10 : 01 , 063 ERROR
[main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore( 5215 ))
- javax.jdo.JDODataStoreException: Required table missing : "`VERSION`" in
Catalog "" Schema "" .
DataNucleus requires this table
to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.autoCreateTables" at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java: 461 ) at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java: 732 ) at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java: 752 ) at
org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java: 6398 ) at
org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java: 6308 ) at
org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java: 6286 ) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 57 ) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 43 ) at
java.lang.reflect.Method.invoke(Method.java: 606 ) at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java: 108 ) at
com.sun.proxy.$Proxy0.verifySchema(Unknown Source) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java: 485 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java: 532 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java: 406 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java: 365 ) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java: 55 ) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java: 60 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java: 4953 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java: 5174 ) at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java: 5093 ) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 57 ) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 43 ) at
java.lang.reflect.Method.invoke(Method.java: 606 ) at
org.apache.hadoop.util.RunJar.main(RunJar.java: 212 ) NestedThrowablesStackTrace:| |
由报错信息猜想hive元数据表创建失败,通过查看mysql中hive的元数据库,果真里面一张表都没有。
通过”Either your MetaData is incorrect, or you need to enable "datanucleus.autoCreateTables" 初步判断可能是因为这个属性没有设置。一顿百度后,看到一个帖子http://blog.youkuaiyun.com/eason_oracle/article/details/52273954,说是修改hive-site.xml:

照着做修改完后重启hive-metastore和hiveserver2,还是没有生成元数据表。
纠结了好久,确定了hive元数据库没有初始化是罪魁祸首,最后,得知hive可以通过命令初始化元数据库
schematool -dbType mysql -initSchema
查看那初始化后的信息 schematool -dbType mysql -info
进入hive的bin目录下执行schematool脚本 :./schematool -dbType mysql -initSchema
1
2
3
4
5
6
7
8
|
[root @node04 bin]#
./schematool -dbType mysql -initSchema Metastore
connection URL: jdbc:mysql: //node04:3306/metastore?createDatabaseIfNotExist=true&characterEncoding=UTF-8 Metastore
Connection Driver : com.mysql.jdbc.Driver Metastore
connection User: hive Starting
metastore schema initialization to 0.13 . 0 Initialization
script hive-schema- 0.13 . 0 .mysql.sql Initialization
script completed schemaTool
completed |
查看初始化信息:
1
2
3
4
5
6
7
|
[root @node04 bin]#
./schematool -dbType mysql -info Metastore
connection URL: jdbc:mysql: //node04:3306/metastore?createDatabaseIfNotExist=true&characterEncoding=UTF-8 Metastore
Connection Driver : com.mysql.jdbc.Driver Metastore
connection User: hive Hive
distribution version: 0.13 . 0 Metastore
schema version: 0.13 . 0 schemaTool
completed |
查看hive的元数据库,发现表已经生成了
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
|
mysql>
show tables; +---------------------------+ |
Tables_in_metastore | +---------------------------+ |
BUCKETING_COLS | |
CDS | |
COLUMNS_V2 | |
COMPACTION_QUEUE | |
COMPLETED_TXN_COMPONENTS | |
DATABASE_PARAMS | |
DBS | |
DB_PRIVS | |
DELEGATION_TOKENS | |
FUNCS | |
FUNC_RU | |
GLOBAL_PRIVS | |
HIVE_LOCKS | |
IDXS | |
INDEX_PARAMS | |
MASTER_KEYS | |
NEXT_COMPACTION_QUEUE_ID | |
NEXT_LOCK_ID | |
NEXT_TXN_ID | |
NUCLEUS_TABLES | |
PARTITIONS | |
PARTITION_EVENTS | |
PARTITION_KEYS | |
PARTITION_KEY_VALS | |
PARTITION_PARAMS | |
PART_COL_PRIVS | |
PART_COL_STATS | |
PART_PRIVS | |
ROLES | |
ROLE_MAP | |
SDS | |
SD_PARAMS | |
SEQUENCE_TABLE | |
SERDES | |
SERDE_PARAMS | |
SKEWED_COL_NAMES | |
SKEWED_COL_VALUE_LOC_MAP | |
SKEWED_STRING_LIST | |
SKEWED_STRING_LIST_VALUES | |
SKEWED_VALUES | |
SORT_COLS |
重启metastore和hiveserver2
beeline连接hiveserve2成功
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[root @node05 ~]#
beeline Beeline
version 0.13 . 1 -cdh5. 3.0 by
Apache Hive beeline>
!connect jdbc:hive2: //node04:10000
hive scan
complete in 4ms Connecting
to jdbc:hive2: //node04:10000 Enter
password for jdbc:hive2: //node04:10000:
**** Connected
to: Apache Hive (version 0.13 . 1 -cdh5. 3.0 ) Driver:
Hive JDBC (version 0.13 . 1 -cdh5. 3.0 ) Transaction
isolation: TRANSACTION_REPEATABLE_READ 0 :
jdbc:hive2: //node04:10000>
show tables; +-----------+--+ |
tab_name | +-----------+--+ +-----------+--+ No
rows selected ( 0.261 seconds) |
参考:http://blog.youkuaiyun.com/diudiu2025/article/details/51890673
http://blog.youkuaiyun.com/eason_oracle/article/details/52273954