DataX二次开发——（8）rdbms的达梦8数据库的支持

原创已于 2024-01-04 11:16:16 修改 · 8.5k 阅读

24 ·

CC 4.0 BY-SA版权

文章标签：

#数据库 #DataX #数据同步 #ETL

于 2022-10-27 13:53:54 首次发布

大数据同时被 2 个专栏收录

40 篇文章

订阅专栏

笔记

11 篇文章

订阅专栏

1 前言

目前GitHub上的DataX3.0开源版本，rdbms里面默认是达梦7的驱动，因此，如果像链接达梦8需要替换驱动。

2 驱动替换

1、下载Dm8JdbcDriver18-xxxx.jar，放到libs下面（rdbmsreader和rdbmswriter同理）
在这里插入图片描述
2、修改maven依赖，注释dm7的驱动，改成达梦8

        <dependency>
            <groupId>com.dameng</groupId>
            <artifactId>Dm8JdbcDriver18</artifactId>
            <version>8.1.1.49</version>
            <scope>system</scope>
            <systemPath>${basedir}/src/main/libs/Dm8JdbcDriver18-8.1.1.49.jar</systemPath>
        </dependency>

        <!--		<dependency>-->
        <!--			<groupId>com.dm</groupId>-->
        <!--			<artifactId>dm</artifactId>-->
        <!--			<scope>system</scope>-->
        <!--			<systemPath>${basedir}/src/main/libs/Dm7JdbcDriver16.jar</systemPath>-->
        <!--		</dependency>-->

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.dameng</groupId>
                <artifactId>Dm8JdbcDriver18</artifactId>
                <version>8.1.1.49</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

3、重新打包编译

mvn -U clean package assembly:assembly -Dmaven.test.skip=true

3 读写达梦8的任务模板json

1、从达梦8读取写入到MySQL5.7模板

{
    "job": {
        "setting": {
            "speed": {
                 "channel": 3
            },
            "errorLimit": {
                "record": 0,
                "percentage": 0.02
            }
        },
        "content": [
            {

                "reader": {
                    "name": "rdbmsreader",
                     "parameter": {
                        "column": ["ID","USERNAME","PASSWORD"],
                        "connection": [
                            {
                                "jdbcUrl": ["jdbc:dm://10.252.xx.xxx:15236?schema=SYSDBA"],
                                "table": ["TEST_DATAX"]
                            }
                        ], 
                        "password": "", 
                        "username": "SYSDBA"
                    }
                },
               "writer": {
                    "name": "mysqlwriter",
                    "parameter": {
                        "print":true,   
                        "column": [
                            "id",
                            "username",
                            "password"
                        ], 
                        "connection": [
                            {
                                "jdbcUrl": "jdbc:mysql://10.252.xxx.xx:30006/test_datax_dm?useSSL=false&allowPublicKeyRetrieval=true&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai",      
                                "table": ["test_datax"]    
                            }
                        ], 
                        "password": "", 
                        "username": "root"
                    }
                }
            }
        ]
    }
}

2、从MySQL5.7读取写入到达梦8模板

{
    "job": {
        "setting": {
            "speed": {
                 "channel": 3
            },
            "errorLimit": {
                "record": 0,
                "percentage": 0.02
            }
        },
        "content": [
            {

                "reader": {
                    "name": "mysqlreader",
                    "parameter": {
                        "username": "root",
                        "password": "",
                        "column": [
                            "id",
                            "username",
                            "password"
                        ],
                        "connection": [
                            {
                                "table": [
                                    "test_datax"    
                                ],
                                "jdbcUrl": [
                                    "jdbc:mysql://10.252.xxx.xx:30006/test_datax_dm?useSSL=false&allowPublicKeyRetrieval=true&useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai"
                                ]
                            }
                        ]
                    }
                },
               "writer": {
                    "name": "rdbmswriter", 
                    "parameter": {
                        "column": ["ID","USERNAME","PASSWORD"], 
                        "connection": [
                            {
                                "jdbcUrl": "jdbc:dm://10.252.xx.xxx:15236?schema=SYSDBA", 
                                "table": ["TEST_DATAX"]
                            }
                        ], 
                        "password": "", 
                        "username": "SYSDBA",
                        "preSql": [], 
                        "session": []
                    }
                }
            }
        ]
    }
}