背景
根据doris官网描述使用Select Into Outfile方式导出时,如果需要并发导出,只需要设置
SET enable_parallel_outfile = true;
即可
文档链接:Select Into Outfile
但是实际操作中,这样并不会生效,可能还需要在会话中额外设置其他参数来解决
正确的使用姿势
SET enable_parallel_outfile = true;
set enable_pipeline_engine = false;
SELECT * FROM `tmpdb`.tmp_table
INTO OUTFILE "s3:/bucket_name/doris_export/ods.db/table_name/file_name"
FORMAT AS ORC
PROPERTIES(
"s3.endpoint" = "",
"s3.region" = "",
"s3.secret_key"="",
"s3.access_key" = "",
"file_suffix" = ".orc",
"max_file_size" = "200MB",
"delete_existing_files" = "true"
);