Sqoop1.4.7安装

一.简介

Sqoop 是一款进行数据传输的工具, 可在hadoop 的 hdfs 和关系型数据库之间传输数据
可以使用Sqoop把数据从MySQL 或 Oracle导入到hdfs中, 也可以把数据从hdfs导入到MySQL或Oracle中

二.安装与配置

基于<<centos7 Hive2.3.6安装与配置>>

1. 下载与解压

cd /usr/local

wget http://www.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

2. 配置环境变量

# 配置环境变量
vi /etc/profile
export SQOOP_HOME=/usr/local/sqoop-1.4.7.bin__hadoop-2.6.0
export PATH=$PATH:$SQOOP_HOME/bin

# 使环境变量生效
source /etc/profile

3. $SQOOP_HOME/conf 修改sqoop_env.sh配置

cd /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/conf
cp sqoop-env-template.sh sqoop-env.sh

vim sqoop-env.sh
#在sqoop_env.sh中
export HADOOP_COMMON_HOME=/usr/local/hadoop-2.7.7
export HADOOP_MAPRED_HOME=/usr/local/hadoop-2.7.7
export HIVE_HOME=/usr/local/apache-hive-2.3.6-bin
export HBASE_HOME=/usr/local/hbase-2.1.1
# export ZOOCFGDIR=

4. 拷贝 mysql驱动到$SQOOP_HOME/lib目录下

cp $HIVE_HOME/lib/mysql-connector-java-5.1.48.jar /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/lib

5. 测试sqoop环境

sqoop version

出现以下内容表示安装成功

Warning: /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
19/12/06 20:05:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

列举所有的数据库

sqoop list-databases --connect jdbc:mysql://localhost:3306/ --username root --password Cocktail_py