hive 插入数据映射到hbase_求助 hive导入数据到HBase,一直100%,又变为0%

用户尝试将Hive表中的数据插入到HBase表中遇到了问题,即使减少数据量到仅一条记录也无法成功。系统显示了长时间的map任务状态停留在100%,最终任务失败并返回了错误代码。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

该楼层疑似违规已被系统折叠 隐藏此楼查看此楼

一开始使用十万条数据出了以下类似信息,后来把表的数据减到20条出了以下信息,后来按网上hive hbase整合的例子,只插入一条数据也不行,求问原因和解决方法,谢谢。Hadoop2.7.3 hive2.1.1 hbase1.2.6 vmware 12 centos7

hive> insert overwrite table table1 select * from table2;

WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.

Query ID = root_20170822141000_95274e7c-1c76-4e20-990d-b9c872a06464

Total jobs = 1

Launching Job 1 out of 1

Number of reduce tasks is set to 0 since there's no reduce operator

Starting Job = job_1503380316328_0001, Tracking URL = http://localhost:8088/proxy/application_1503380316328_0001/

Kill Command = /usr/local/hadoop/hadoop-2.7.3/bin/hadoop job -kill job_1503380316328_0001

Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0

2017-08-22 14:16:07,563 Stage-1 map = 0%, reduce = 0%

2017-08-22 14:16:42,618 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:17:43,077 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:18:44,127 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:19:44,506 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:20:45,304 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:21:45,946 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:22:46,026 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:23:46,369 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:24:49,887 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:25:50,058 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:26:50,427 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.58 sec

2017-08-22 14:27:26,439 Stage-1 map = 0%, reduce = 0%

2017-08-22 14:28:27,219 Stage-1 map = 0%, reduce = 0%

2017-08-22 14:28:54,512 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:29:54,748 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:30:55,535 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:31:56,262 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:32:57,403 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:33:57,973 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:34:58,303 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:35:58,841 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:36:59,447 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:37:59,522 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:38:59,712 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.01 sec

2017-08-22 14:39:20,359 Stage-1 map = 0%, reduce = 0%

2017-08-22 14:40:20,765 Stage-1 map = 0%, reduce = 0%, Cumulative CPU 3.47 sec

2017-08-22 14:40:43,370 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:41:44,146 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:42:45,035 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:43:45,057 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:44:45,986 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:45:46,513 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:46:47,254 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:47:48,279 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:48:48,284 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:49:48,353 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:50:48,394 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 4.59 sec

2017-08-22 14:50:49,541 Stage-1 map = 0%, reduce = 0%

2017-08-22 14:51:36,852 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:52:37,837 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:53:38,823 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:54:38,900 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:55:38,962 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:56:39,042 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:57:39,612 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:58:40,396 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 14:59:42,840 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 15:00:43,872 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 15:01:44,439 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 3.51 sec

2017-08-22 15:02:45,871 Stage-1 map = 100%, reduce = 0%

2017-08-22 15:03:46,482 Stage-1 map = 100%, reduce = 0%

MapReduce Total cumulative CPU time: 3 seconds 510 msec

Ended Job = job_1503380316328_0001 with errors

Error during job, obtaining debugging information...

Job Tracking URL: http://localhost:8088/proxy/application_1503380316328_0001/

Examining task ID: task_1503380316328_0001_m_000000 (and more) from job job_1503380316328_0001

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

MapReduce Jobs Launched:

Stage-Stage-1: Map: 1 Cumulative CPU: 3.51 sec HDFS Read: 0 HDFS Write: 0 FAIL

Total MapReduce CPU Time Spent: 3 seconds 510 msec

例子:

hive> CREATE TABLE hbase_table_1(key int, value string)

> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'

> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")

> TBLPROPERTIES ("hbase.table.name" = "xyz");

hive> create table ccc(foo int,bar string) row format delimited fields terminated by '\t' lines terminated by '\n' stored as textfile;

load data local inpath '/root/kv1.txt' overwrite into table ccc;

insert overwrite table hbase_table_1 select * from ccc where foo=1;

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值