学习笔记:从0开始学习大数据-11. sqoop安装部署

环境：centos7 已安装java和hadoop

1.下载
wget http://archive.cloudera.com/cdh5/cdh/5/sqoop2-1.99.5-cdh5.16.0.tar.gz

2.解压 tar -zxvf sqoop2-1.99.5-cdh5.16.0.tar.gz

3. 修改 /etc/profile
增加环境变量

export SQOOP_HOME=/home/linbin/software/sqoop2-1.99.5-cdh5.16.0
export SQOOP_SERVER_EXTRA_LIB=$SQOOP_HOME/extra
export CATALINA_BASE=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs/
export PATH=$PATH:$SQOOP_HOME/bin

环境变量生效
source  /etc/profile

4.  修改配置文件 
/home/linbin/software/sqoop2-1.99.5-cdh5.16.0/server/conf/sqoop.propertions

org.apache.sqoop.submission.engine.mapreduce.configuration.directory=
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/etc/hadoop

把所有 @BASEDIR@ 替换成 /home/linbin/software/sqoop2-1.99.5-cdh5.16.0/

把所有@LOGDIR@替换成 /home/linbin/software/sqoop2-1.99.5-cdh5.16.0/logs

 

5. 修改配置文件
/home/linbin/software/sqoop2-1.99.5-cdh5.16.0/server/conf/catalina.propertions

以下在同一行
common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/common/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/common/lib/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/hdfs/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/hdfs/lib/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/mapreduce/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/mapreduce/lib/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/tools/lib/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/yarn/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/yarn/lib/*.jar,
/home/linbin/software/hadoop-2.6.0-cdh5.16.0/share/hadoop/httpfs/tomcat/lib/*.jar

6. 启动服务
#sqoop.sh server start

7. 启动客户端
#sqoop.sh server client

8.可以使用sqoop在不同数据库系统间导入导出数据了

这是另外一台虚拟机安装的是 sqoop2-1.99.5-cdh5.15.1版本

9. 使用sqoop2导入导出数据实践 参考网址

https://blog.youkuaiyun.com/HG_Harvey/article/details/77986531  Sqoop2 MySQL与HDFS导入导出

https://blog.youkuaiyun.com/xuxudede1989/article/details/81700251   用sqoop2从mysql导入数据到hbase