大数据之SQOOP1安装部署

ln -s /usr/local/mysql/lib/libmysqlclient.so.18 /usr/local/lib64

三台机器都装
由于sqoop2不太好用所以安装sqoop1系列
sqoop-cdh安装：

mkdir -p /opt/sqoop
1、上传sqoop-1.4.6-cdh5.12.0.tar.gz mysql-connector-java.jar, ojdbc7.jar包到/opt/sqoop目录
2、解压缩文件： tar -zxf sqoop-1.4.6-cdh5.12.0.tar.gz
3、设置环境变量：
vi  /etc/profile
#添加一下内容
#Hadoop
export HADOOP_HOME=/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop
export HADOOP_CMD=/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/bin/hadoop
export PATH=$PATH:$HADOOP_HOME:$HADOOP_HOME/bin
#hbasae
export HBASE_HOME=/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase
export PATH=$PATH:$HBASE_HOME:$HBASE_HOME/bin
#hive
export HIVE_HOME=/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hive
export HCAT_HOME=/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hive-hcatalog
export PATH=$PATH:$HIVE_HOME:$HIVE_HOME/bin
#sqoop
export SQOOP_HOME=/opt/sqoop/sqoop-1.4.6-cdh5.12.0
export PATH=$PATH:$SQOOP_HOME/bin

 

4： source  /etc/profile
5、修改sqoop配置文件：
进入到conf目录：cd /opt/sqoop/sqoop-1.4.6-cdh5.12.0/conf
修改文件名：cp sqoop-env-template.sh sqoop-env.sh
修改文件内容：vim sqoop-env.sh
#修改内容如下：
# Set Hadoop-specific environment variables here.
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME= /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME= /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop
#set the path to where bin/hbase is available
export HBASE_HOME= /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase
#Set the path to where bin/hive is available
export HIVE_HOME= /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hive
#Set the path for where zookeper config dir is
export ZOOCFGDIR= /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/zookeeper
5、导入驱动包

sqoop安装完成后,根据连接方式不同，选择导入jar包：

1、mysql连接导入jar:

将jar包上传到/opt/sqoop目录
执行命令：
拷贝到$SQOOP_HOME/lib目录 cp /opt/sqoop/mysql-connector-java.jar /opt/sqoop/sqoop-1.4.6-cdh5.12.0/lib
拷贝到$HADOOP_HOME/lib目录 cp /opt/sqoop/mysql-connector-java.jar /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/lib

拷贝到$SQOOP_HOME/lib目录 cp /opt/sqoop/ojdbc7.jar  /opt/sqoop/sqoop-1.4.6-cdh5.12.0/lib
拷贝到$HADOOP_HOME/lib目录 cp /opt/sqoop/ojdbc7.jar  /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hadoop/lib

 

最后

rm -rf /usr/bin/sqoop

ln -s /opt/sqoop/sqoop-1.4.6-cdh5.12.1/bin/sqoop  /usr/bin

验证  

改你自己的ip

sqoop list-databases --connect jdbc:mysql://10.195.237.109:3306/mysql --username root --password bsoft2019