CDH配置sqoop

最新推荐文章于 2024-08-11 14:57:23 发布

原创最新推荐文章于 2024-08-11 14:57:23 发布 · 1.1k 阅读

2 ·

CC 4.0 BY-SA版权

文章标签：

#sqoop #hadoop #cloudera #大数据 #mysql

本文介绍了如何在CDH集群上添加并配置Sqoop服务，包括安装sqoop2服务以及下载并配置MySQL驱动，以便进行大数据与MySQL之间的数据迁移操作。

添加sqoop服务

打开cluster
添加服务
选择sqoop2，然后一路继续即可

在这里插入图片描述
4. 安装完成后查看服务

在这里插入图片描述

配置mysql驱动

下载mysql驱动到cdh的jars路径下

mysql驱动下载地址

root@cdh1:/opt/cloudera/parcels/CDH/jars# ll mysql-connector-java-5.1.49-bin.jar 
-rw-r--r-- 1 root root 1006906 1月  21 09:17 mysql-connector-java-5.1.49-bin.jar

给sqoop配置mysql驱动，创建软连接

root@cdh8:/opt/cloudera/parcels/CDH/lib/sqoop/lib# ln -s ../../../jars/mysql-connector-java-5.1.49-bin.jar mysql-connector-java-5.1.49-bin.jar

4.连接mysql查看数据库

root@cdh3:~# sqoop list-databases --connect jdbc:mysql://cdh3:3306/ --username xxx --password xxxx
Warning: /opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
21/01/21 13:31:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.14.2
21/01/21 13:31:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/01/21 13:31:41 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Thu Jan 21 13:31:41 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
information_schema