在mysql上套上impala中间件后,能够实现mysql的覆盖更新功能,原理如下:通过jddl往impala中load数据时候,impala根据mysql表的主键,如果已存在,则update;如果不存在,则insert。其中关键点是mysql主键和load语句中需要填上replace参数。
例如如下load语句
{"load": "load data local infile 'jingo.csv' replace into table th_sku_change_info fields terminated by '\\t' enclosed by '' lines terminated by '\\n' (id,cate,item_sku_id,main_sku_id,item_id,sku_name,item_name,data_type,brand_code,barndname_en,barndname_cn,barndname_full,item_first_cate_cd,item_first_cate_name,item_second_cate_cd,item_second_cate_name,item_third_cate_cd,item_third_cate_name,work_post_cd,purchaser_erp_acct,purchaser_name,saler_erp_acct,sale_staf_name,pop_operator_erp_acct,pop_operator_name,pop_vender_id,pop_vender_name,shop_id,shop_name,dept_id_3,dept_name_3,dept_id_2,dept_name_2,dept_id_1,dept_name_1,bu_id,bu_name)","cnum":36}
load语句如下
其中replace和ignore参数说明如下:
The REPLACE and IGNORE keywords control handling of input rows that duplicate existing rows on unique key values:
If you specify REPLACE, input rows replace existing rows. In other words, rows that have the same value for a primary key or unique index as an existing row. See Section 14.2.8, “REPLACE Syntax”.
If you specify IGNORE, rows that duplicate an existing row on a unique key value are discarded. For more information, see Comparison of the IGNORE Keyword and Strict SQL Mode.
If you do not specify either option, the behavior depends on whether the LOCAL keyword is specified. Without LOCAL, an error occurs when a duplicate key value is found, and the rest of the text file is ignored. With LOCAL, the default behavior is the same as if IGNORE is specified; this is because the server has no way to stop transmission of the file in the middle of the operation.
加replace
the default behavior is the same as if IGNORE is specified;