操作hive报Application xxx submitted by user hadoop to unknown queue: default

在使用beeline操作Hive时,遇到一个问题:select语句能够正常执行,但create table...as select...语句失败,报错提示应用由用户hadoop提交到未知队列default。原因是未指定正确的mapreduce.job.queuename。解决方案是在Hive会话中设置队列名称为所需队列,如test001。对于不同的引擎,如Spark和Thriftserver,队列设置需求也不同。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

今天用beeline去操作hive,简单的语句能执行成功,复杂的就不行了,记录在此望下次不再入坑。

select 语句能出数据

执行select c_13930,c_45365,c_std,c_cv,c_22599,c_opp,c_cs,c_is_outnet
from devtest.yangtest001 a;能出数据

create 语句报错

执行create table devtest.t1 as select c_13930,c_45365,c_std,c_cv,c_22599,c_opp,c_cs,c_is_outnet from devtest.yangtest001 a;就会报错,但是看信息是队列名有问题

报错信息如下

ERROR : Job Submission failed with exception 'java.io.IOException(org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1624689709041_1440 to YARN : Application application_1624689709041_1440 submitted by user hadoop to unknown queue: default)'
java.io.IOException: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1624689709041_1440 to YARN : Application application_1624689709041_1440 submitted by user hadoop to unknown queue: default
	at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:316)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:242)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341)
	at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338)
	at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
	at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:411)
	at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:151)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2183)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1839)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1526)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1232)
	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:255)
	at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1624689709041_1440 to YARN : Application application_1624689709041_1440 submitted by user hadoop to unknown queue: default
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:276)
	at org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:296)
	at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
	... 35 more

原因

我这番操作是在默认的队列default上,需要设置成本hive指定的队列

解决办法

连上hive后,输入以下命令:

set mapreduce.job.queuename=test001

其他解释

使用spark引擎(连10001端口)时,它已经指定了队列名,所以不需要设置;
使用thriftserver引擎(连10000端口)时,默认使用的是default队列名,由于每个队列的权限不同,有需要的话需要指定队列名。

至于为什么我在执行select语句可以,create语句不行呢?hive里执行sql都是会提交job任务的,但因为create会需要往hdfs上落数据,当遇到队列的权限不匹配,就报错了;select的权限存在,就没有报错。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值