sqoop1.4.4导出mysql数据到hadoop1.2.1集群出现的问题

在使用Hadoop 1.2.1集群进行sqoop导入MySQL数据时遇到错误提示需要设置HADOOP_HOME、HBASE_HOME和HCAT_HOME,以及解决版本不兼容导致的IncompatibleClassChangeError问题。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

改为hadoop1.2.1的集群后。还是提示拒绝连接的错误,但是我发现错误提示里说需要HADOOP_HOME。所以我export了HADOOP_HOME。然后再次运行报了下面的错误。

hadoop1@slave02 sqoop-1.4.4]$ ./bin/sqoop import --connect jdbc:mysql://localhost/hadoopguide --table widgets -username root -password root

Warning: /usr/lib/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: $HADOOP_HOME is deprecated.

14/08/16 12:43:21 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/08/16 12:43:21 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/08/16 12:43:21 INFO tool.CodeGenTool: Beginning code generation
14/08/16 12:43:23 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
14/08/16 12:43:23 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
14/08/16 12:43:23 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop1/hadoop-1.2.1
注: /tmp/sqoop-hadoop1/compile/4c6868cf30f6d1678ba33b8bde05d9a2/widgets.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
14/08/16 12:43:27 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop1/compile/4c6868cf30f6d1678ba33b8bde05d9a2/widgets.jar
14/08/16 12:43:27 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/08/16 12:43:27 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/08/16 12:43:27 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/08/16 12:43:27 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/08/16 12:43:27 INFO mapreduce.ImportJobBase: Beginning import of widgets
14/08/16 12:43:38 INFO mapred.JobClient: Cleaning up the staging area hdfs://master:9000/home/hadoop1/hadoop-1.2.1/tmp/mapred/staging/hadoop1/.staging/job_201408161228_0002
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.JobContext, but interface was expected
    at org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:53)
    at com.cloudera.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:36)
    at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:121)
    at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)
    at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
    at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
    at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239)
    at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:600)
    at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)

    at org.apache.sqoop.Sqoop.


百度了一下大家说是版本不兼容的问题。sqoop-1.4.4不支持hadoop-1.2.1

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值