使用命令行:sqoop import --connect jdbc:mysql://*.*.*.*:3306/test?characterEncoding=UTF-8 --username aaa--password 'bbb' -table lll
尝试将mysql的数据导入到hdfs中,结果一直导入失败,并显示:
Warning: /usr/local/sqoop/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/local/sqoop/sqoop/../hcatalog does not exist! HCatalog jobs will fa il.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop/sqoop/../zookeeper does not exist! Accumulo imports wil l fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
2019-06-11 02:41:42,287 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2019-06-11 02:41:42,500 WARN tool.BaseSqoopTool: Setting your password on the com mand-line is insecure. Consider using -P instead.
2019-06-11 02:41:42,753 INFO manager.MySQLManager: Preparing to use a MySQL strea ming resultset.
2019-06-11 02:41:42,753 INFO tool.CodeGenTool: Beginning code generation
Tue Jun 11 02:41:43 EDT 2019 WARN: Establishing SSL connection without server's i dentity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit opt ion isn't set. For compliance with existing applications not using SSL the verify ServerCertificate property is set to 'false'. You need either to explicitly disab le SSL by setting useSSL=false, or set useSSL=true and provide truststore for ser ver certificate verification.
2019-06-11 02:41:45,892 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `data_weilai_pressure` AS t LIMIT 1
2019-06-11 02:41:47,326 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `data_weilai_pressure` AS t LIMIT 1
2019-06-11 02:41:48,010 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/l ocal/hadoop
Note: /tmp/sqoop-myy/compile/abd370f29650c545e0e82607b93e8cf0/data_weilai_pressur e.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
2019-06-11 02:41:51,300 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop -myy/compile/abd370f29650c545e0e82607b93e8cf0/data_weilai_pressure.jar
2019-06-11 02:41:51,434 WARN manager.MySQLManager: It looks like you are importin g from mysql.
2019-06-11 02:41:51,440 WARN manager.MySQLManager: This transfer can be faster! U se the --direct
2019-06-11 02:41:51,440 WARN manager.MySQLManager: option to exercise a MySQL-spe cific fast path.
2019-06-11 02:41:51,440 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
2019-06-11 02:41:52,127 INFO mapreduce.ImportJobBase: Beginning import of data_we ilai_pressure
2019-06-11 02:41:52,129 INFO Configuration.deprecation: mapred.job.tracker is dep recated. Instead, use mapreduce.jobtracker.address
2019-06-11 02:41:52,386 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
2019-06-11 02:41:53,971 INFO Configuration.deprecation: mapred.map.tasks is depre cated. Instead, use mapreduce.job.maps
2019-06-11 02:41:54,129 INFO client.RMProxy: Connecting to ResourceManager at mas ter/10.141.200.75:8032
2019-06-11 02:41:55,024 INFO mapreduce.JobResourceUploader: Disabling Erasure Cod ing for path: /tmp/hadoop-yarn/staging/myy/.staging/job_1560165070962_0011
Tue Jun 11 02:42:06 EDT 2019 WARN: Establishing SSL connection without server's i dentity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit opt ion isn't set. For compliance with existing applications not using SSL the verify ServerCertificate property is set to 'false'. You need either to explicitly disab le SSL by setting useSSL=false, or set useSSL=true and provide truststore for ser ver certificate verification.
2019-06-11 02:42:07,808 INFO db.DBInputFormat: Using read commited transaction is olation
2019-06-11 02:42:08,425 INFO mapreduce.JobSubmitter: number of splits:1
2019-06-11 02:42:08,530 INFO Configuration.deprecation: yarn.resourcemanager.syst em-metrics-publisher.enabled is deprecated. Instead, use yarn.system-metrics-publ isher.enabled
2019-06-11 02:42:08,879 INFO mapreduce.JobSubmitter: Submitting tokens for job: j ob_1560165070962_0011
2019-06-11 02:42:08,882 INFO mapreduce.JobSubmitter: Executing with tokens: []
2019-06-11 02:42:09,245 INFO conf.Configuration: resource-types.xml not found
2019-06-11 02:42:09,246 INFO resource.ResourceUtils: Unable to find 'resource-typ es.xml'.
2019-06-11 02:42:09,391 INFO impl.YarnClientImpl: Submitted application applicati on_1560165070962_0011
2019-06-11 02:42:09,507 INFO mapreduce.Job: The url to track the job: http://mast er:8088/proxy/application_1560165070962_0011/
2019-06-11 02:42:09,509 INFO mapreduce.Job: Running job: job_1560165070962_0011
2019-06-11 02:42:21,686 INFO mapreduce.Job: Job job_1560165070962_0011 running in uber mode : false
2019-06-11 02:42:21,688 INFO mapreduce.Job: map 0% reduce 0%
2019-06-11 02:44:35,691 INFO mapreduce.Job: Task Id : attempt_1560165070962_0011_ m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exc eptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The drive r has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.ja va:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java :161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77 )
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.jav a:137)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat ion.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.Communicat ionsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The drive r has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputForma t.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.ja va:168)
... 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communication s link failure
The last packet sent successfully to the server was 0 milliseconds ago. The drive r has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructo rAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCo nstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:99 0)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:342)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2197)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:22 30)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2025)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:778)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructo rAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingCo nstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java: 330)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfigur ation.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputForma t.java:216)
... 11 more
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.jav a:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketI mpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java: 188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.jav a:211)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:301)
... 27 more
2019-06-11 02:46:50,403 INFO mapreduce.Job: Task Id : attempt_1560165070962_0011_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)
... 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:342)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2197)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2230)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2025)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:778)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:216)
... 11 more
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:301)
... 27 more
2019-06-11 02:49:02,941 INFO mapreduce.Job: Task Id : attempt_1560165070962_0011_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:223)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:168)
... 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:342)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2197)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2230)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2025)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:778)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:216)
... 11 more
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:301)
... 27 more
目前还没有找到原因,好气!!!!初学者用linux简直像是在炼丹!!