批处理数据同步脚本

批处理真不好写~,现把成果粘贴出来留个纪念~~~

@echo off
rem =======================================================================
rem = ① 本批处理实现目录数据的导出                        ---20160816    =
rem =======================================================================
cd..
set OrgID=83a9280f-2b19-4bc9-ab37-78f94b64dc76
set DumpFolder=d:
set LogFolder=d:
set FileDate=%Date:~0,4%%Date:~5,2%%Date:~8,2%
set UserName=datacenter
set Password=datacenter
set NetServer=orcl
rem ===================
rem =   备份源库      =
rem ===================
cd %DumpFolder%
echo 开始备份源库数据......
start /b /wait exp %UserName%/%Password%@%NetServer% tables=(rc_catalog_basic,rc_resource_info,rc_dynamic_authority) file=%DumpFolder%\%OrgID%_%FileDate%.dmp log=%LogFolder%\%OrgID%_exp_%FileDate%.log
echo 源库数据备份完毕!
pause
exit

@echo off
rem =======================================================================
rem = ② 本批处理实现目录数据同步到中心                     ---20160816	  =
rem =======================================================================
set DumpFolder=d:\dmp
set LogFolder=d:\log
set FileDate=%Date:~0,4%%Date:~5,2%%Date:~8,2%
set UserName=datacenter
set Password=datacenter
set NetServer=orcl
for /f "tokens=1 delims=_" %%i in ('dir /a-d /b *.dmp') do (
echo insert into rc_catalog_basic_bak select * from rc_catalog_basic where organization_id = '%%i';>%DumpFolder%\pre.sql
echo insert into rc_resource_info_bak select * from rc_resource_info t where (t.catalog_resource_id,t.version^) in (select catalog_resource_id,version from rc_catalog_basic where organization_id = '%%i'^);>>%DumpFolder%\pre.sql
echo insert into rc_dynamic_authority_bak select * from rc_dynamic_authority t where (t.catalog_resource_id,t.version^) in (select catalog_resource_id,version from rc_catalog_basic where organization_id = '%%i'^);>>%DumpFolder%\pre.sql
echo commit;>>%DumpFolder%\pre.sql
echo delete from rc_catalog_basic where organization_id = '%%i';>>%DumpFolder%\pre.sql
echo delete from rc_resource_info t where (t.catalog_resource_id,t.version^) in (select catalog_resource_id,version from rc_catalog_basic where organization_id = '%%i'^);>>%DumpFolder%\pre.sql
echo delete from rc_dynamic_authority t where (t.catalog_resource_id,t.version^) in (select catalog_resource_id,version from rc_catalog_basic where organization_id = '%%i'^);>>%DumpFolder%\pre.sql
echo commit;>>%DumpFolder%\pre.sql
echo exit;>>%DumpFolder%\pre.sql
echo 前置机%%i数据环境准备初始化。。。
start /b /wait sqlplus %UserName%/%Password%@%NetServer% @%DumpFolder%\pre.sql >%LogFolder%\%%i_pre_%FileDate%.log
echo 前置机%%i数据环境初始化完毕!
echo 前置机%%i准备同步数据。。。
imp %UserName%/%Password%@%NetServer% fromuser=%UserName% touser=%UserName% ignore=y file=%DumpFolder%\%%i_%FileDate%.dmp log=%LogFolder%\%%i_imp_%FileDate%.log
echo 前置机%%i同步数据完成!
echo 前置机%%iDMP文件准备删除。。。
del %DumpFolder%\%%i_%FileDate%.dmp
echo 前置机%%iDMP文件删除完成!
)
pause



Flink SQL(也称为Apache Flink SQL)是一种用于处理流数据和批处理数据的强大工具,它允许用户通过SQL语法编写数据同步脚本。Flink SQL支持从多种数据源读取数据,并将结果写入到其他存储系统,如HDFS、MySQL、Kafka等。编写Flink SQL数据同步脚本主要包括以下几个步骤: 1. **连接源和目标**: 使用`CREATE TABLE`或`INSERT INTO`语句定义源表(通常是从外部数据源)和目标表(通常是内部或持久化的表)。 ```sql CREATE TABLE source_table ( column1 STRING, column2 INT, //... ) WITH ( 'connector' = 'jdbc', -- 或者 'kafka', 'hdfs' 'url' = '<source_url>', 'table-name' = '<source_table_name>' ); INSERT INTO sink_table SELECT * FROM source_table; ``` 2. **设置模式和转换**: 可能需要对源数据进行过滤、映射、聚合或其他操作。Flink SQL提供了丰富的函数和窗口功能。 ```sql SELECT column1, SUM(column2) as total_sum FROM source_table GROUP BY window(start_time, INTERVAL '5' MINUTE) ``` 3. **配置作业**: 设置并行度、检查点策略和错误恢复选项等,以优化性能和容错性。 ```sql SET parallelism = <parallelism>; SET checkpoint.interval = <interval>; ``` 4. **运行任务**: 调用`executeSql()`或`submitJob()`来启动Flink SQL作业。 ```java StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); TableEnvironment tEnv = StreamTableEnvironment.create(env); tEnv.executeSql("your Flink SQL script"); ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值