hive 倒数到HBase

最新推荐文章于 2022-04-08 13:39:29 发布

唐-coffee

最新推荐文章于 2022-04-08 13:39:29 发布

阅读量174

点赞数

分类专栏： hbase

本文链接：https://blog.youkuaiyun.com/m0_38102941/article/details/88876185

版权

hbase 专栏收录该内容

2 篇文章

订阅专栏

版本说明

		版本
1	hadoop	2.x
2	hive	0.11
3	hbase	0.98.6

https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration hive wiki

安装hbase客户端

下载hbase0.98.6版本，将以下内容覆盖　到HBASE_HOME/conf/hbase-site.xml

<name>zookeeper.znode.parent</name>

<value>/hbase/wap_marmot</value>

</property>

<name>hbase.zookeeper.quorum</name>

<value>l-zk2.dba.cn6.qunar.com</value>

</property>

<name>hbase.zookeeper.property.clientPort</name>

</property>

覆盖对应版本jar包

将HBase_home/lib/中一下jar包覆盖hive_home/lib 中对应jar包

hbase-client-0.98.6-hadoop2.jar

hbase-common-0.98.6-hadoop2-tests.jar

hbase-common-0.98.6-hadoop2.jar

hbase-protocol-0.98.6-hadoop2.jar

hbase-server-0.98.6-hadoop2.jar

htrace-core-2.04.jar

替换掉　hive_home/lib/中以下jar：

hbase-0.94.6.1-tests.jar

hbase-0.94.6.1.jar

访问hive创建表

hive.shell　-- 线上环境

hive.java

sudo -u wirelessdev hive --auxpath ${HIVE_HOME}/lib/hive-hbase-handler-0.11.0.jar,${HIVE_HOME}/lib/hbase-client-0.98.6-hadoop2.jar,${HIVE_HOME}/lib/hbase-common-0.98.6-hadoop2.jar,${HIVE_HOME}/lib/hbase-server-0.98.6-hadoop2.jar,${HIVE_HOME}/lib/hbase-protocol-0.98.6-hadoop2.jar,${HIVE_HOME}/lib/zookeeper-3.4.6.jar,${HIVE_HOME}/lib/htrace-core-2.04.jar --hiveconf hbase.zookeeper.quorum=l-zk1.dba.cn6.qunar.com,l-zk2.dba.cn6.qunar.com,l-zk3.dba.cn8.qunar.com,l-zk4.dba.cn8.qunar.com,l-zk5.dba.cn1.qunar.com --hiveconf zookeeper.znode.parent=/hbase/wap_marmot

hive.shell　-- 测试环境

hive.java

创建表

create table

CREATE TABLE table_name (
uid STRING,
platform STRING,
channel STRING,
chan_value STRING,
host INT,
logtime STRING,
bd_source STRING,
action STRING,
refer STRING,
back_url STRING,
browser STRING,
mobile STRING,
server_ip STRING,
ip STRING,
url STRING,
post STRING,
handle_time STRING,
loghost STRING COMMENT 'log host name'
)
PARTITIONED BY (
dt STRING,
HOUR STRING,
ctype STRING)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES
("hbase.columns.mapping"=":key,info:platform,info:channel,info:chan_value,info:host,info:logtime,info:bd_source,info:action,info:refer,info: back_url,info:browser,info:mobile,info:server_ip,info:ip,info:url,info:post,info:handle_time,info:loghost")
TBLPROPERTIES("hbase.table.name" = "marmot:orig_flight_touch");

导入数据

loaddata

INSERT overwrite TABLE table_name PARTITION (dt='2015-05-13', HOUR ='20', ctype='tuan')
SELECT
uid,
platform,
channel,
chan_value,
host,
logtime,
bd_source,
action,
refer,
back_url,
browser,
mobile,
server_ip,
ip,
url,
post,
handle_time,
loghost
FROM orig_touch
WHERE dt = '2015-05-13'
AND hour = '20'
AND ctype = 'tuan';