Sqoop2 将hdfs中的数据导出到MySQL

最新推荐文章于 2024-09-02 10:05:29 发布

weixin_30580943

最新推荐文章于 2024-09-02 10:05:29 发布

阅读量301

点赞数

CC 4.0 BY-SA版权

文章标签：数据库 java 网络

原文链接：http://www.cnblogs.com/chuijingjing/p/10728749.html

本文详细介绍了Sqoop2的操作流程，包括进入终端、配置服务器、查看连接和配置信息，还阐述了创建HDFS和MySQL的Link、创建作业、查看作业以及启动作业等步骤，涉及数据库和HDFS的数据交互，为相关操作提供了具体指导。

1.进入sqoop2终端：

[root@master /]# sqoop2

2.为客户端配置服务器：

sqoop:000> set server --host master --port 12000 --webapp sqoop

3.查看服务器配置：

sqoop:000> show version --all

4. 查看sqoop的所有连接：

sqoop 所有的连接固定为四个，如下：

sqoop:000> show connector

5.创建hdfs的link：

sqoop:000> create link --cid 3

sqoop:000> create link --cid 3
Creating link for connector with id 3
Please fill following values to create new link object
Name: hdfs_link

Link configuration

HDFS URI: hdfs://master:8020  // 这个地方一定要和hdfs中的地址对应
New link was successfully created with validation status OK and persistent id 2

5.创建MySQL的Link

sqoop:000> create link --cid 1
Creating link for connector with id 1
Please fill following values to create new link object
Name: mysql_link

Link configuration

JDBC Driver Class: com.mysql.jdbc.Driver
JDBC Connection String: jdbc:mysql://master:3306/app        
Username: root
Password: ****
JDBC Connection Properties: 
There are currently 0 values in the map:
entry# protocol=tcp
There are currently 1 values in the map:
protocol = tcp
entry# 
New link was successfully created with validation status OK and persistent id 3

6.查看当前的所有link

sqoop:000> show link

7.创建job

sqoop:000> create job -f 1 -t 2
Creating job for links with from id 1 and to id 2
Please fill following values to create new job object
Name: hdfs_to_mysql

From Job configuration

Input directory: /yk/dl/GetAlarmHis/
Override null value: null 
Null value: null

To database configuration

Schema name: app
Table name: alarm_his
Table SQL statement: 
Table column names: 
Stage table name: 
Should clear stage table: 

Throttling resources

Extractors: 1
Loaders: 1
New job was successfully created with validation status OK  and persistent id 1