python 访问hbase

最新推荐文章于 2024-11-18 20:40:50 发布

guoqiangma

最新推荐文章于 2024-11-18 20:40:50 发布

阅读量1k

点赞数

分类专栏： HBase

本文链接：https://blog.youkuaiyun.com/guoqiangma/article/details/18353271

版权

HBase 专栏收录该内容

16 篇文章

订阅专栏

本文介绍了如何使用Thrift框架在Python中访问HBase。首先，详细说明了Thrift的下载、安装及配置过程，然后讲解了安装Python的Thrift包的步骤，接着展示了如何生成Python和HBase通讯的Thrift包，最后提供了启动HBase Thrift服务的方法及编写测试程序的示例代码。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

通过thrift，我们可以使用python访问hbase。

关于thrift

thrift是一个跨语言服务的软件开发框架（Thrift is a software framework for scalable cross-language services development.）。

它的官方网站是：http://incubator.apache.org/thrift/

下载thrift

svn co http://svn.apache.org/repos/asf/incubator/thrift/trunk thrift

基本安装工具环境yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel./configure 配置，看到如下信息：

thrift 0.9.1
Building C++ Library ......... : no
Building C (GLib) Library .... : no
Building Java Library ........ : yes
Building C# Library .......... : no
Building Python Library ...... : yes
Building Ruby Library ........ : no
Building Haskell Library ..... : no
Building Perl Library ........ : no
Building PHP Library ......... : no
Building Erlang Library ...... : no
Building Go Library .......... : no
Building D Library ........... : no
thrift 0.9.1

Building C++ Library ......... : no

Building C (GLib) Library .... : no

Building Java Library ........ : yes

Building C# Library .......... : no

Building Python Library ...... : yes

Building Ruby Library ........ : no

Building Haskell Library ..... : no

Building Perl Library ........ : no

Building PHP Library ......... : no

Building Erlang Library ...... : no

Building Go Library .......... : no

Building D Library ........... : no

Java Library:

Using javac ............... : javac

Using java ................ : java

Using ant ................. : /usr/bin/ant

Python Library:

Using Python .............. : /usr/bin/python

make

make install

查看thrift 版本

thrift -version

2. 安装python 的thrift 包

[root@mgq-lenovo py]# cd /opt/thrift-0.9.1/lib/py/

[root@mgq-lenovo py]# ls

build compat dist Makefile Makefile.am Makefile.in README setup.cfg setup.py src thrift.egg-info

[root@mgq-lenovo py]# python setup.py build

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.6

....................
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c src/protocol/fastbinary.c -o build/temp.linux-x86_64-2.6/src/protocol/fastbinary.o
gcc -pthread -shared build/temp.linux-x86_64-2.6/src/protocol/fastbinary.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/thrift/protocol/fastbinary.so

[root@mgq-lenovo py]# python setup.py install
running install
running bdist_egg
running egg_info
......
Adding thrift 0.9.1 to easy-install.pth file
Installed /usr/lib64/python2.6/site-packages/thrift-0.9.1-py2.6-linux-x86_64.egg
Processing dependencies for thrift==0.9.1
Finished processing dependencies for thrift==0.9.1

3 生成python和hbase通讯的thrift包：

[root@mgq-lenovo thrift]# cd /opt/hbase/src/main/resources/org/apache/hadoop/hbase/thrift

[root@mgq-lenovo1 thrift]# ls

Hbase.thrift

[root@mgq-lenovo thrift]# thrift --gen py:new_style Hbase.thrift

[root@mgq-lenovo thrift]# ls

gen-py Hbase.thrift

[root@mgq-lenovo thrift]# ll -t

总用量 28

drwxr-xr-x 3 root root 4096 1月 14 21:11 gen-py.bak

-rwxr-xr-x 1 hbase hbase 23434 8月 28 2012 Hbase.thrift

[root@mgq-lenovo thrift]# mv gen-py /usr/lib/python2.6/site-packages/