mapreduce mysql hdfs_sqoop导mysql数据到hdfs报错，mapreduce任务失败

最新推荐文章于 2025-01-21 22:38:54 发布

摩羯啪啦啪啦

最新推荐文章于 2025-01-21 22:38:54 发布

阅读量431

点赞数

CC 4.0 BY-SA版权

文章标签： mapreduce mysql hdfs

本文链接：https://blog.youkuaiyun.com/weixin_26870313/article/details/113317171

在尝试使用MapReduce和Sqoop将MySQL数据导入到HDFS时遇到错误，作业job_1462370365985_0020因应用程序失败两次而终止。错误日志显示Container退出代码为1，具体原因包括ExitCodeException，堆栈跟踪涉及多个Hadoop和YARN组件。MapReduce作业完成0%的映射和减少操作，并且没有计数器信息。最后，警告提示Counters的某些组已过时。总之，数据迁移过程中出现了严重的问题，导致任务失败。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

16/09/09 15:22:22 INFO mapreduce.Job: Running job: job_1462370365985_0020

16/09/09 15:22:40 INFO mapreduce.Job: Job job_1462370365985_0020 running in uber mode : false

16/09/09 15:22:40 INFO mapreduce.Job: map 0% reduce 0%

16/09/09 15:22:40 INFO mapreduce.Job: Job job_1462370365985_0020 failed with state FAILED due to: Application application_1462370365985_0020 failed 2 times due to AM Container for appattempt_1462370365985_0020_000002 exited with exitCode: 1

For more detailed output, check application tracking page:http://liujunjie-1:8088/proxy/ ... /Then, click on links to logs of each attempt.

Diagnostics: Exception from container-launch.

Container id: container_1462370365985_0020_02_000001

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:543)

at org.apache.hadoop.util.Shell.run(Shell.java:460)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:720)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

Failing this attempt. Failing the application.

16/09/09 15:22:40 INFO mapreduce.Job: Counters: 0

16/09/09 15:22:40 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead

16/09/09 15:22:40 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 24.0458 seconds (0 bytes/sec)

16/09/09 15:22:40 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead

16/09/09 15:22:40 INFO mapreduce.ImportJobBase: Retrieved 0 records.

16/09/09 15:22:40 ERROR tool.ImportTool: Error during import: Import job failed!