Centos 6.5 安装impyla

最新推荐文章于 2023-05-12 23:05:57 发布

柔情岁月-万圣节

最新推荐文章于 2023-05-12 23:05:57 发布

阅读量825

点赞数

CC 4.0 BY-SA版权

分类专栏：机器学习

本文链接：https://blog.youkuaiyun.com/redhat1986/article/details/87875109

机器学习专栏收录该内容

2 篇文章

订阅专栏

本文档介绍了在Centos 6.5上安装impyla的步骤，包括先安装Anaconda，然后安装必要的组件，并解决在安装过程中遇到的thrift协议异常和权限问题。通过切换到有权限的用户，最终成功连接到impala。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在上一篇文章中已经安装好了Anaconda https://blog.youkuaiyun.com/redhat1986/article/details/87824229
需要安装的组件有（安装之前先把pip装好，这里不说pip的安装了）

模块	版本
pure_sasl	0.5.1
thrift_sasl	0.2.1
thrift	0.9.3
thriftpy	0.3.9
impyla	0.14.1

安装各个组件：

pip install pure-sasl

在这里插入图片描述

pip install thrift_sasl==0.2.1 --no-deps

在这里插入图片描述

 pip install thrift==0.9.3

在这里插入图片描述

 conda install -c anaconda impyla

在这里插入图片描述

安装完成测试连接

[root@xxxx-35-12 ~]# python
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> 
>>> from impala.dbapi import connect
>>> 
>>> 
>>> conn = connect(host='你的hiveserver2  IP',port=10000,database='default',auth_mechanism='NOSASL')
>>> 
>>> 
>>> cur = conn.cursor()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 125, in cursor
    session = self.service.open_session(user, configuration)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 995, in open_session
    resp = self._rpc('OpenSession', req)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 923, in _rpc
    response = self._execute(func_name, request)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 940, in _execute
    return func(request)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/thrift.py", line 198, in _req
    return self._recv(_api)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/thrift.py", line 210, in _recv
    fname, mtype, rseqid = self._iprot.read_message_begin()
  File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/protocol/binary.py", line 372, in read_message_begin
    self.trans, strict=self.strict_read)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/thriftpy/protocol/binary.py", line 178, in read_message_begin
    message='No protocol version header')
thriftpy.protocol.exc.TProtocolException: TProtocolException(type=4)
>>> 
>>> 
>>>

thriftpy.protocol.exc.TProtocolException: TProtocolException(type=4)问题解决

pip install git+https://github.com/laserson/python-sasl.git@cython

在这里插入图片描述
有关于sasl报错的信息可以忽略

继续测试连接

[root@xxx35-12 ~]# python
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> 
>>> 
>>> 
>>> from impala.dbapi import connect                                                         
>>> conn = connect(host='xx.xx.xx.xx',port=10000,database='default',auth_mechanism='PLAIN') 
>>> cur = conn.cursor()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 139, in cursor
    cursor.execute('USE %s' % self.default_db)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 302, in execute
    configuration=configuration)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 343, in execute_async
    self._execute_async(op)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 362, in _execute_async
    operation_fn()
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 340, in op
    _async=True)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 1027, in execute
    return self._operation('ExecuteStatement', req)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 957, in _operation
    resp = self._rpc(kind, request)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 925, in _rpc
    err_if_rpc_not_ok(response)
  File "/export/tools/anaconda3/lib/python3.7/site-packages/impala/hiveserver2.py", line 704, in err_if_rpc_not_ok
    raise HiveServer2Error(resp.status.errorMessage)
impala.error.HiveServer2Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User root does not have privileges for SWITCHDATABASE
 The required privileges: Server=server1->Db=*->Table=+->Column=*->action=select;Server=server1->Db=*->Table=+->Column=*->action=insert;
>>> 
>>> 
>>> 
>>> exit()

报错是因为权限问题，切换个有权限连接impala的用户执行：

(base) [lixianwei@xxxxx35-12 ~]$ python
Python 3.7.1 (default, Dec 14 2018, 19:28:38) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> 
>>> 
>>> 
>>> from impala.dbapi import connect
>>> conn = connect(host='xx.xx.xx.xx',port=10000,database='default',auth_mechanism='PLAIN')
>>> cur = conn.cursor()
>>> cur.execute('SHOW DATABASES')
>>> print(cur.fetchall())
[('default',), ('tmp',)]
>>> cur.execute('SHOW Tables')
>>> 
>>> 
>>> exit()

连接成功!