java操作sqoop在本地测试正常打包发布到阿里云服务器报错

在将sqoop与mapreduce集成到SpringBoot项目并打包上传到阿里云服务器时遇到问题。首先,由于服务器环境变量不完整导致Hadoop相关问题,通过在/etc/profile中添加Hadoop环境变量解决。其次,报错显示找不到sqoop的类,检查jar包发现存在该库,但依然报错。最后,为解决打包问题,将项目改为单体SpringBoot应用并打包为war,问题得以解决。作者寻求关于打包问题的解决方案。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

在做项目时,把sqoop和mapreduce集成到springboot中做数据抽取和计算,并打成jar包上传到服务器,项目是使用eureka进行注册提供服务,父项目下创建子模块,通过maven的package打包子模块。
通过以下命令在服务器运行

# nohup和&组合在后台运行  运行时会在当前目录下生成一个nohup.out日志文件
[root@fda ~]# nohup java -jar XXX.jar &

Sqoop版本1.4.7
Hadoop版本2.7.7
当进行sqoop的数据抽取时报错信息如下

2021-02-27 19:24 | [34mINFO [0;39m | [1;33mtask-1[0;39m | [1;32mo.a.sqoop.orm.CompilationManager[0;39m | $HADOOP_MAPRED_HOME is not set
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:7: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.BytesWritable;
                           ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:8: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Text;
                           ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:9: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Writable;
                           ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:10: error: package org.apache.hadoop.mapred.lib.db does not exist
import org.apache.hadoop.mapred.lib.db.DBWritable;
                                      ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:11: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.JdbcWritableBridge;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:12: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.DelimiterSet;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:13: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.FieldFormatter;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:14: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.RecordParser;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:15: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.BooleanParser;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:16: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.BlobRef;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:17: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.ClobRef;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:18: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.LargeObjectLoader;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:19: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.SqoopRecord;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:37: error: cannot find symbol
public class Users extends SqoopRecord  implements DBWritable, Writable {
                           ^
  symbol: class SqoopRecord
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:37: error: cannot find symbol
public class Users extends SqoopRecord  implements DBWritable, Writable {
                                                   ^
  symbol: class DBWritable
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:37: error: cannot find symbol
public class Users extends SqoopRecord  implements DBWritable, Writable {
                                                               ^
  symbol: class Writable
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:137: error: cannot find symbol
  public void loadLargeObjects(LargeObjectLoader __loader)
                               ^
  symbol:   class LargeObjectLoader
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:140: error: cannot find symbol
  public void loadLargeObjects0(LargeObjectLoader __loader)
                                ^
  symbol:   class LargeObjectLoader
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:217: error: cannot find symbol
  private static final DelimiterSet __outputDelimiters = new DelimiterSet((char) 9, (char) 10, (char) 0, (char) 0, false);
                       ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:221: error: cannot find symbol
  public String toString(DelimiterSet delimiters) {
                         ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:227: error: cannot find symbol
  public String toString(DelimiterSet delimiters, boolean useRecordDelim) {
                         ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:240: error: cannot find symbol
  public void toString0(DelimiterSet delimiters, StringBuilder __sb, char fieldDelim) {
                        ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:247: error: cannot find symbol
  private static final DelimiterSet __inputDelimiters = new DelimiterSet((char) 9, (char) 10, (char) 0, (char) 0, false);
                       ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:248: error: cannot find symbol
  private RecordParser __parser;
          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:249: error: cannot find symbol
  public void parse(Text __record) throws RecordParser.ParseError {
                    ^
  symbol:   class Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:249: error: package RecordParser does not exist
  public void parse(Text __record) throws RecordParser.ParseError {
                                                      ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:257: error: package RecordParser does not exist
  public void parse(CharSequence __record) throws RecordParser.ParseError {
                                                              ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:265: error: package RecordParser does not exist
  public void parse(byte [] __record) throws RecordParser.ParseError {
                                                         ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:273: error: package RecordParser does not exist
  public void parse(char [] __record) throws RecordParser.ParseError {
                                                         ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:281: error: package RecordParser does not exist
  public void parse(ByteBuffer __record) throws RecordParser.ParseError {
                                                            ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:289: error: package RecordParser does not exist
  public void parse(CharBuffer __record) throws RecordParser.ParseError {
                                                            ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:128: error: cannot find symbol
    this.mobile = JdbcWritableBridge.readString(1, __dbResults);
                  ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:129: error: cannot find symbol
    this.userName = JdbcWritableBridge.readString(2, __dbResults);
                    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:130: error: cannot find symbol
    this.password = JdbcWritableBridge.readString(3, __dbResults);
                    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:133: error: cannot find symbol
    this.mobile = JdbcWritableBridge.readString(1, __dbResults);
                  ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:134: error: cannot find symbol
    this.userName = JdbcWritableBridge.readString(2, __dbResults);
                    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:135: error: cannot find symbol
    this.password = JdbcWritableBridge.readString(3, __dbResults);
                    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:148: error: cannot find symbol
    JdbcWritableBridge.writeString(mobile, 1 + __off, 12, __dbStmt);
    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:149: error: cannot find symbol
    JdbcWritableBridge.writeString(userName, 2 + __off, 12, __dbStmt);
    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:150: error: cannot find symbol
    JdbcWritableBridge.writeString(password, 3 + __off, 12, __dbStmt);
    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:154: error: cannot find symbol
    JdbcWritableBridge.writeString(mobile, 1 + __off, 12, __dbStmt);
    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:155: error: cannot find symbol
    JdbcWritableBridge.writeString(userName, 2 + __off, 12, __dbStmt);
    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:156: error: cannot find symbol
    JdbcWritableBridge.writeString(password, 3 + __off, 12, __dbStmt);
    ^
  symbol:   variable JdbcWritableBridge
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:164: error: cannot find symbol
    this.mobile = Text.readString(__dataIn);
                  ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:169: error: cannot find symbol
    this.userName = Text.readString(__dataIn);
                    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:174: error: cannot find symbol
    this.password = Text.readString(__dataIn);
                    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:182: error: cannot find symbol
    Text.writeString(__dataOut, mobile);
    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:188: error: cannot find symbol
    Text.writeString(__dataOut, userName);
    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:194: error: cannot find symbol
    Text.writeString(__dataOut, password);
    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:202: error: cannot find symbol
    Text.writeString(__dataOut, mobile);
    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:208: error: cannot find symbol
    Text.writeString(__dataOut, userName);
    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:214: error: cannot find symbol
    Text.writeString(__dataOut, password);
    ^
  symbol:   variable Text
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:217: error: cannot find symbol
  private static final DelimiterSet __outputDelimiters = new DelimiterSet((char) 9, (char) 10, (char) 0, (char) 0, false);
                                                             ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:230: error: cannot find symbol
    __sb.append(FieldFormatter.escapeAndEnclose(mobile==null?"\\N":mobile, delimiters));
                ^
  symbol:   variable FieldFormatter
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:232: error: cannot find symbol
    __sb.append(FieldFormatter.escapeAndEnclose(userName==null?"\\N":userName, delimiters));
                ^
  symbol:   variable FieldFormatter
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:234: error: cannot find symbol
    __sb.append(FieldFormatter.escapeAndEnclose(password==null?"\\N":password, delimiters));
                ^
  symbol:   variable FieldFormatter
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:241: error: cannot find symbol
    __sb.append(FieldFormatter.escapeAndEnclose(mobile==null?"\\N":mobile, delimiters));
                ^
  symbol:   variable FieldFormatter
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:243: error: cannot find symbol
    __sb.append(FieldFormatter.escapeAndEnclose(userName==null?"\\N":userName, delimiters));
                ^
  symbol:   variable FieldFormatter
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:245: error: cannot find symbol
    __sb.append(FieldFormatter.escapeAndEnclose(password==null?"\\N":password, delimiters));
                ^
  symbol:   variable FieldFormatter
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:247: error: cannot find symbol
  private static final DelimiterSet __inputDelimiters = new DelimiterSet((char) 9, (char) 10, (char) 0, (char) 0, false);
                                                            ^
  symbol:   class DelimiterSet
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:251: error: cannot find symbol
      this.__parser = new RecordParser(__inputDelimiters);
                          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:259: error: cannot find symbol
      this.__parser = new RecordParser(__inputDelimiters);
                          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:267: error: cannot find symbol
      this.__parser = new RecordParser(__inputDelimiters);
                          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:275: error: cannot find symbol
      this.__parser = new RecordParser(__inputDelimiters);
                          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:283: error: cannot find symbol
      this.__parser = new RecordParser(__inputDelimiters);
                          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:291: error: cannot find symbol
      this.__parser = new RecordParser(__inputDelimiters);
                          ^
  symbol:   class RecordParser
  location: class Users
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:363: error: cannot find symbol
    Users o = (Users) super.clone();
                      ^
  symbol:   variable super
  location: class Users
67 errors
2021-02-27 19:24 | [1;31mERROR[0;39m | [1;33mtask-1[0;39m | [1;32morg.apache.sqoop.tool.ImportTool[0;39m | Import failed: java.io.IOException: Error returned by javac
	at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:226)
	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at zut.edu.service.impl.DataExtractionServiceImpl.sqoopTask(DataExtractionServiceImpl.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

这里一共有两个问题第一个问题

/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:7: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.BytesWritable;
                           ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:8: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Text;
                           ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:9: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Writable;

这是由于阿里云的环境变量没有配置完整Hadoop的环境变量。在服务器的 /etc/profile 中添加如下配置
#Hadoop环境变量

[root@fda ~]# vim /etc/profile
#添加下面的内容
export HADOOP_HOME=/opt/module/Hadoop/hadoop-2.7.7
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
export YARN_HOME=$HADOOP_HOME
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_LOG_DIR=$HADOOP_LOG_DIR
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

第一个问题解决。
第二个问题是

/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:11: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.JdbcWritableBridge;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:12: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.DelimiterSet;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:13: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.FieldFormatter;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:14: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.RecordParser;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:15: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.BooleanParser;

报的sqoop的找不到sqoop-1.4.7.jar下面的类,打开打成的可执行jar包,在BOOT-INF\lib下确实有sqoop-1.4.7.jar,找了好几天最终发现问题,是因为打包的原因,但是又找不到怎样去打包,索性直接把原来的后台项目架构改成了单体的springboot项目,打成war包部署到服务器,问题解决。

原来打包问题待进一步研究……如果小伙伴有解决方案,请与我分享一下,感谢!!!

解决打成jar包找不到依赖

原因是找不到classpath
报错信息

06:26 16:01:23.177 [task-1] INFO  o.a.sqoop.orm.CompilationManager - HADOOP_MAPRED_HOME is /opt/module/Hadoop/hadoop-2.7.7
06:26 16:01:24.048 [task-1] ERROR org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: Error returned by javac
	at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:217)
	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:108)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at zut.edu.service.impl.DataExtractionServiceImpl.sqoopTask(DataExtractionServiceImpl.java:124)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

动态编译的日志如果没有特殊配置的话,是无法通过log4j进行输出的,因此,编译错误需要到系统日志里查找:

/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:11: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.JdbcWritableBridge;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:12: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.DelimiterSet;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:13: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.FieldFormatter;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:14: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.RecordParser;
                             ^
/tmp/sqoop-root/compile/f677c4605f731efc068e40b146260436/Users.java:15: error: package com.cloudera.sqoop.lib does not exist
import com.cloudera.sqoop.lib.BooleanParser;

在用到依赖之前的代码添加以下代码

String curClasspath = System.getProperty ("java.class.path");
        curClasspath = curClasspath
                + File.pathSeparator
                + "/opt/module/Hadoop/hadoop-2.7.7/share/hadoop/common/hadoop-common-2.7.7.jar"
                + File.pathSeparator
                + "/opt/module/Sqoop/sqoop-1.4.7/lib/sqoop-1.4.7.jar"
                + File.pathSeparator
                + "/opt/module/Hadoop/hadoop-2.7.7/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.7.jar";
        System.setProperty ("java.class.path", curClasspath);
评论 7
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值