Windows impyla 踩坑

安装各种包

conda install impyla
pip install thriftpy2
#建议使用此版本，高版本出现数据量大读不出来的问题
pip install --no-deps thrift-sasl==0.2.1
conda install pure-sasl

报错

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "D:\pycharm\PyCharm Community Edition 2020.3.2\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "D:\pycharm\PyCharm Community Edition 2020.3.2\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/project/project_python/saim2/hive_test.py", line 5, in <module>
    conn = connect(host='10.1.3.18', port=10000, auth_mechanism='PLAIN', database='saim_test', user='hive',password='hive')
  File "D:\anaconda\envs\saim2_hivetest\lib\site-packages\impala\dbapi.py", line 144, in connect
    service = hs2.connect(host=host, port=port,
  File "D:\anaconda\envs\saim2_hivetest\lib\site-packages\impala\hiveserver2.py", line 826, in connect
    transport.open()
  File "D:\anaconda\envs\saim2_hivetest\lib\site-packages\thrift_sasl\__init__.py", line 75, in open
    self._send_message(self.START, chosen_mech)
  File "D:\anaconda\envs\saim2_hivetest\lib\site-packages\thrift_sasl\__init__.py", line 94, in _send_message
    self._trans.write(header + body)
TypeError: can't concat str to bytes

解决方法

更改thrift_sasl下的__init__.py，在第93行94行之间加入

    if (type(body) is str):
      body = body.encode()

加入之后代码

  def _send_message(self, status, body):
    header = struct.pack(">BI", status, len(body))
    if (type(body) is str):
      body = body.encode()
    self._trans.write(header + body)
    self._trans.flush()

参考

https://github.com/cloudera/impyla/issues/238