- 在上一篇文章中已经安装好了Anaconda https://blog.youkuaiyun.com/redhat1986/article/details/87824229
- 需要安装的组件有(安装之前先把pip装好,这里不说pip的安装了)
模块 | 版本 |
---|---|
pure_sasl | 0.5.1 |
thrift_sasl | 0.2.1 |
thrift | 0.9.3 |
thriftpy | 0.3.9 |
impyla | 0.14.1 |
- 安装各个组件:
pip install pure-sasl
pip install thrift_sasl==0.2.1 --no-deps
pip install thrift==0.9.3
conda install -c anaconda impyla
- 安装完成测试连接
[root@xxxx-35-12 ~]# python
Python 3.7.1 (default, Dec 14 2018, 19:28:38)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> from impala.dbapi import connect
>>>
>>>
>>> conn = connect(host='你的hiveserver2 IP',port=10000,database='default',auth_mechanism='NOSASL')
>>>
>>>
>>> cur = conn.cursor()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 125, in cursor
session = self.service.open_session(user, configuration)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 995, in open_session
resp = self._rpc('OpenSession', req)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 923, in _rpc
response = self._execute(func_name, request)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 940, in _execute
return func(request)
File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/thrift.py", line 198, in _req
return self._recv(_api)
File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/thrift.py", line 210, in _recv
fname, mtype, rseqid = self._iprot.read_message_begin()
File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/protocol/binary.py", line 372, in read_message_begin
self.trans, strict=self.strict_read)
File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/protocol/binary.py", line 178, in read_message_begin
message='No protocol version header')
thriftpy.protocol.exc.TProtocolException: TProtocolException(type=4)
>>>
>>>
>>>
- thriftpy.protocol.exc.TProtocolException: TProtocolException(type=4)问题解决
pip install git+https://github.com/laserson/python-sasl.git@cython
有关于sasl报错的信息可以忽略
- 继续测试连接
[root@xxx35-12 ~]# python
Python 3.7.1 (default, Dec 14 2018, 19:28:38)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>>
>>>
>>> from impala.dbapi import connect
>>> conn = connect(host='xx.xx.xx.xx',port=10000,database='default',auth_mechanism='PLAIN')
>>> cur = conn.cursor()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 139, in cursor
cursor.execute('USE %s' % self.default_db)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 302, in execute
configuration=configuration)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
self._execute_async(op)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
operation_fn()
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 340, in op
_async=True)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 1027, in execute
return self._operation('ExecuteStatement', req)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 957, in _operation
resp = self._rpc(kind, request)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
err_if_rpc_not_ok(response)
File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
raise HiveServer2Error(resp.status.errorMessage)
impala.error.HiveServer2Error: Error while compiling statement: FAILED: SemanticException No valid privileges
User root does not have privileges for SWITCHDATABASE
The required privileges: Server=server1->Db=*->Table=+->Column=*->action=select;Server=server1->Db=*->Table=+->Column=*->action=insert;
>>>
>>>
>>>
>>> exit()
报错是因为权限问题,切换个有权限连接impala的用户执行:
(base) [lixianwei@xxxxx35-12 ~]$ python
Python 3.7.1 (default, Dec 14 2018, 19:28:38)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>>
>>>
>>> from impala.dbapi import connect
>>> conn = connect(host='xx.xx.xx.xx',port=10000,database='default',auth_mechanism='PLAIN')
>>> cur = conn.cursor()
>>> cur.execute('SHOW DATABASES')
>>> print(cur.fetchall())
[('default',), ('tmp',)]
>>> cur.execute('SHOW Tables')
>>>
>>>
>>> exit()
- 连接成功!