- 简介
DataX 是一个异构数据源离线同步工具,致力于实现包括关系型数据库(MySQL、Oracle等)、HDFS、Hive、ODPS、HBase、FTP等各种异构数据源之间稳定高效的数据同步功能。 - Datax下载
1.安装包下载地址:http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz
2.源码下载地址:https://github.com/alibaba/DataX - 源码编译环境安装,前置条件大家可以自行安装jdk、python、maven并配置环境变量。
对应版本如下图所示:
- 源码编译
修改maven安装包下conf/settings.xml
1.增加本地仓库目录配置
2.增加淘宝镜像配置
<!-- localRepository
| The path to the local repository maven will use to store artifacts.
|
| Default: ${user.home}/.m2/repository
<localRepository>/path/to/local/repo</localRepository>
-->
<localRepository>F:\apache-maven-3.6.1-bin\repository</localRepository>
<mirrors>
<!-- mirror
| Specifies a repository mirror site to use instead of a given repository. The repository that
| this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
| for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
|
<mirror>
<id>mirrorId</id>
<mirrorOf>repositoryId</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://my.repository.com/repo/path</url>
</mirror>
-->
<mirror>
<id>nexus-aliyun</id>
<mirrorOf>central</mirrorOf>
<name>Nexus aliyun</name>
<url>https://maven.aliyun.com/repository/central</url>
</mirror>
</mirrors>
3.修改odps读和写的相关pom.xml
<dependency>
<groupId>com.aliyun.odps</groupId>
<artifactId>odps-sdk-core</artifactId>
<!--<version>0.19.3-public</version>-->
<version>0.20.7-public</version>
</dependency>
编译成功截图:
编译后文件和生成的压缩包如下图:
7. 参考链接
参考链接:https://blog.youkuaiyun.com/qq_32447301/article/details/82909326
参考链接:https://blog.youkuaiyun.com/baidu_30851231/article/details/79863779
8. Datax使用