grep: Invalid range end

本文介绍了一种使用awk命令解决大型CSV文件差异对比的问题,避免了因文件过大而导致的grep命令失效的情况。通过具体实例,展示了如何利用awk找出两个文件间的不同项并输出到第三个文件。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

报错发生的情景介绍: 我要筛选出biao1.csv和biao2.csv的不同项,并把不同的内容输出到另外一个文件中; 
[root@localhost data]# grep -vwf biao1.csv biao2.csv < ./diff.csv

grep: Invalid range end
最开始我以为是命令用错了,于是把biao1.csv和biao2.csv这两个文件的位置在命令行中对调了一下,但情况还是一样,均出现如上‘grep: Invalid range end’报错。

最后,发现是因为grep命令后面比较的两个文件太大了,如果文件小的话,可以还是可以用grep命令的。但是,如果文件大的话建议用awk命令,反正我这里做比较的文件大小为19M都不行。 

如下是用awk命令解决此问题的,命令如下:

awk 'NR==FNR{a[$0]}NR>FNR{ if(!($1 in a)) print $0}' biao1.csv biao2.csv < ./diff.csv

此命令会找出文件 biao1与biao2中不同的值 ,其中 biao2是参考值;

 

#!/bin/bash # 定义目标目录 base_dir="/home/trade/product_env/backup" output_file="${base_dir}/fout.txt" # 确保输出目录存在 mkdir -p "$(dirname "$output_file")" # 清空或创建输出文件 > "$output_file" # 动态计算日期范围:从指定日期到上个月的第一天 get_date_range() { local current_date="$1" local day=$(date -d "$current_date" +%d) local last_month_first_day # 显式将 day 转换为十进制 last_month_first_day=$(date -d "$current_date -$(($10#$day - 1)) days -1 month" +"%Y%m%d") echo "$current_date $last_month_first_day" } # 将 out.sh 的功能封装为函数 generate_output() { local log_path="$1" local output_file="$2" cd "$log_path" || return # 创建一个临时文件,用于保存经过处理后的唯一行 temp_file=$(mktemp) # 提取目录名作为当前日期(假设目录名格式为 YYYYMMDD) current_date=$(basename "$(dirname "$(dirname "$log_path")")") # 获取动态日期范围 read -r start_date end_date <<< "$(get_date_range "$current_date")" # 查找指定日期范围内的Log.txt文件 find . -name "*$start_date*Log.txt" -o -name "*$end_date*Log.txt" | while IFS= read -r file; do # 在文件中查找特定字段,并处理数据 grep -E "172.22.88.*result=okay:user=" "$file" | cut -d':' -f6,8,9 | sort | uniq >> "$temp_file" done # 将临时文件内容写入输出文件 cat "$temp_file" > out.txt rm "$temp_file" # 合并并去重输出 sort out.txt | uniq >> "$output_file" rm "out.txt" } # 查找符合每月一号格式的目录(如 20250401) for dir in $(find "$base_dir" -mindepth 1 -maxdepth 1 -type d -name "[0-9][0-9][0-9][0-9][0-1][0-9][0-3][0-9]" | sort); do # 遍历 day 和 night 目录 for sub_dir in day night; do log_path="${dir}/${sub_dir}/log" if [ -d "$log_path" ]; then # 调用封装好的函数替代 out.sh generate_output "$log_path" "$output_file" fi done done # 最终去重和排序 sort -u "$output_file" -o "$output_file"./output.sh: line 20: 202503100#10: invalid arithmetic base (error token is "202503100#10") ./output.sh: line 20: 202503110#11: invalid arithmetic base (error token is "202503110#11") ./output.sh: line 20: 202503110#11: invalid arithmetic base (error token is "202503110#11") ./output.sh: line 20: 202503120#12: invalid arithmetic base (error token is "202503120#12") ./output.sh: line 20: 202503120#12: invalid arithmetic base (error token is "202503120#12") ./output.sh: line 20: 202503130#13: invalid arithmetic base (error token is "202503130#13") ./output.sh: line 20: 202503130#13: invalid arithmetic base (error token is "202503130#13") ./output.sh: line 20: 202503140#14: invalid arithmetic base (error token is "202503140#14") ./output.sh: line 20: 202503140#14: invalid arithmetic base (error token is "202503140#14") ./output.sh: line 20: 202503170#17: invalid arithmetic base (error token is "202503170#17") ./output.sh: line 20: 202503170#17: invalid arithmetic base (error token is "202503170#17") ./output.sh: line 20: 202503180#18: invalid arithmetic base (error token is "202503180#18") ./output.sh: line 20: 202503180#18: invalid arithmetic base (error token is "202503180#18") ./output.sh: line 20: 202503190#19: invalid arithmetic base (error token is "202503190#19") ./output.sh: line 20: 202503190#19: invalid arithmetic base (error token is "202503190#19") ./output.sh: line 20: 202503200#20: invalid arithmetic base (error token is "202503200#20") ./output.sh: line 20: 202503200#20: invalid arithmetic base (error token is "202503200#20") ./output.sh: line 20: 202503210#21: invalid arithmetic base (error token is "202503210#21") ./output.sh: line 20: 202503210#21: invalid arithmetic base (error token is "202503210#21") ./output.sh: line 20: 202503240#24: invalid arithmetic base (error token is "202503240#24") ./output.sh: line 20: 202503240#24: invalid arithmetic base (error token is "202503240#24") ./output.sh: line 20: 202503250#25: invalid arithmetic base (error token is "202503250#25") ./output.sh: line 20: 202503250#25: invalid arithmetic base (error token is "202503250#25") ./output.sh: line 20: 202503260#26: invalid arithmetic base (error token is "202503260#26") ./output.sh: line 20: 202503260#26: invalid arithmetic base (error token is "202503260#26") ./output.sh: line 20: 202503270#27: invalid arithmetic base (error token is "202503270#27") ./output.sh: line 20: 202503270#27: invalid arithmetic base (error token is "202503270#27") ./output.sh: line 20: 202503280#28: invalid arithmetic base (error token is "202503280#28") ./output.sh: line 20: 202503280#28: invalid arithmetic base (error token is "202503280#28") ./output.sh: line 20: 202503310#31: invalid arithmetic base (error token is "202503310#31") ./output.sh: line 20: 202503310#31: invalid arithmetic base (error token is "202503310#31") ./output.sh: line 20: 202504010#01: invalid arithmetic base (error token is "202504010#01") ./output.sh: line 20: 202504010#01: invalid arithmetic base (error token is "202504010#01") ./output.sh: line 20: 202504020#02: invalid arithmetic base (error token is "202504020#02") ./output.sh: line 20: 202504020#02: invalid arithmetic base (error token is "202504020#02") ./output.sh: line 20: 202504030#03: invalid arithmetic base (error token is "202504030#03") ./output.sh: line 20: 202504030#03: invalid arithmetic base (error token is "202504030#03") ./output.sh: line 20: 202504070#07: invalid arithmetic base (error token is "202504070#07") ./output.sh: line 20: 202504080#08: invalid arithmetic base (error token is "202504080#08") ./output.sh: line 20: 202504080#08: invalid arithmetic base (error token is "202504080#08") ./output.sh: line 20: 202504090#09: invalid arithmetic base (error token is "202504090#09") ./output.sh: line 20: 202504090#09: invalid arithmetic base (error token is "202504090#09") ./output.sh: line 20: 202504100#10: invalid arithmetic base (error token is "202504100#10") ./output.sh: line 20: 202504100#10: invalid arithmetic base (error token is "202504100#10") ./output.sh: line 20: 202504110#11: invalid arithmetic base (error token is "202504110#11") ./output.sh: line 20: 202504110#11: invalid arithmetic base (error token is "202504110#11") ./output.sh: line 20: 202504140#14: invalid arithmetic base (error token is "202504140#14") ./output.sh: line 20: 202504140#14: invalid arithmetic base (error token is "202504140#14") ./output.sh: line 20: 202504150#15: invalid arithmetic base (error token is "202504150#15") ./output.sh: line 20: 202504150#15: invalid arithmetic base (error token is "202504150#15") ./output.sh: line 20: 202504160#16: invalid arithmetic base (error token is "202504160#16") ./output.sh: line 20: 202504160#16: invalid arithmetic base (error token is "202504160#16") ./output.sh: line 20: 202504170#17: invalid arithmetic base (error token is "202504170#17") ./output.sh: line 20: 202504170#17: invalid arithmetic base (error token is "202504170#17") ./output.sh: line 20: 202504180#18: invalid arithmetic base (error token is "202504180#18") ./output.sh: line 20: 202504180#18: invalid arithmetic base (error token is "202504180#18") ./output.sh: line 20: 202504210#21: invalid arithmetic base (error token is "202504210#21") ./output.sh: line 20: 202504210#21: invalid arithmetic base (error token is "202504210#21") ./output.sh: line 20: 202504220#22: invalid arithmetic base (error token is "202504220#22") ./output.sh: line 20: 202504220#22: invalid arithmetic base (error token is "202504220#22") ./output.sh: line 20: 202504230#23: invalid arithmetic base (error token is "202504230#23") ./output.sh: line 20: 202504230#23: invalid arithmetic base (error token is "202504230#23") ./output.sh: line 20: 202504240#24: invalid arithmetic base (error token is "202504240#24") ./output.sh: line 20: 202504240#24: invalid arithmetic base (error token is "202504240#24") ./output.sh: line 20: 202504250#25: invalid arithmetic base (error token is "202504250#25") ./output.sh: line 20: 202504250#25: invalid arithmetic base (error token is "202504250#25") ./output.sh: line 20: 202504280#28: invalid arithmetic base (error token is "202504280#28") ./output.sh: line 20: 202504280#28: invalid arithmetic base (error token is "202504280#28") ./output.sh: line 20: 202504290#29: invalid arithmetic base (error token is "202504290#29") ./output.sh: line 20: 202504290#29: invalid arithmetic base (error token is "202504290#29") ./output.sh: line 20: 202504300#30: invalid arithmetic base (error token is "202504300#30") ./output.sh: line 20: 202504300#30: invalid arithmetic base (error token is "202504300#30") ./output.sh: line 20: 202505060#06: invalid arithmetic base (error token is "202505060#06") ./output.sh: line 20: 202505070#07: invalid arithmetic base (error token is "202505070#07") ./output.sh: line 20: 202505070#07: invalid arithmetic base (error token is "202505070#07") ./output.sh: line 20: 202505080#08: invalid arithmetic base (error token is "202505080#08") ./output.sh: line 20: 202505080#08: invalid arithmetic base (error token is "202505080#08") ./output.sh: line 20: 202505090#09: invalid arithmetic base (error token is "202505090#09") ./output.sh: line 20: 202505090#09: invalid arithmetic base (error token is "202505090#09") ./output.sh: line 20: 202505120#12: invalid arithmetic base (error token is "202505120#12") ./output.sh: line 20: 202505120#12: invalid arithmetic base (error token is "202505120#12") ./output.sh: line 20: 202505130#13: invalid arithmetic base (error token is "202505130#13") ./output.sh: line 20: 202505130#13: invalid arithmetic base (error token is "202505130#13") ./output.sh: line 20: 202505140#14: invalid arithmetic base (error token is "202505140#14") ./output.sh: line 20: 202505140#14: invalid arithmetic base (error token is "202505140#14") ./output.sh: line 20: 202505150#15: invalid arithmetic base (error token is "202505150#15") ./output.sh: line 20: 202505150#15: invalid arithmetic base (error token is "202505150#15") ./output.sh: line 20: 202505160#16: invalid arithmetic base (error token is "202505160#16") ./output.sh: line 20: 202505160#16: invalid arithmetic base (error token is "202505160#16") ./output.sh: line 20: 202505190#19: invalid arithmetic base (error token is "202505190#19") ./output.sh: line 20: 202505190#19: invalid arithmetic base (error token is "202505190#19") ./output.sh: line 20: 202505200#20: invalid arithmetic base (error token is "202505200#20") ./output.sh: line 20: 202505200#20: invalid arithmetic base (error token is "202505200#20") ./output.sh: line 20: 202505210#21: invalid arithmetic base (error token is "202505210#21") ./output.sh: line 20: 202505210#21: invalid arithmetic base (error token is "202505210#21") ./output.sh: line 20: 202505220#22: invalid arithmetic base (error token is "202505220#22") ./output.sh: line 20: 202505220#22: invalid arithmetic base (error token is "202505220#22") ./output.sh: line 20: 202505230#23: invalid arithmetic base (error token is "202505230#23") ./output.sh: line 20: 202505230#23: invalid arithmetic base (error token is "202505230#23") ./output.sh: line 20: 202505260#26: invalid arithmetic base (error token is "202505260#26") ./output.sh: line 20: 202505260#26: invalid arithmetic base (error token is "202505260#26") ./output.sh: line 20: 202505270#27: invalid arithmetic base (error token is "202505270#27") ./output.sh: line 20: 202505270#27: invalid arithmetic base (error token is "202505270#27") ./output.sh: line 20: 202505280#28: invalid arithmetic base (error token is "202505280#28") ./output.sh: line 20: 202505280#28: invalid arithmetic base (error token is "202505280#28") ./output.sh: line 20: 202505290#29: invalid arithmetic base (error token is "202505290#29") ./output.sh: line 20: 202505290#29: invalid arithmetic base (error token is "202505290#29") ./output.sh: line 20: 202505300#30: invalid arithmetic base (error token is "202505300#30") ./output.sh: line 20: 202505300#30: invalid arithmetic base (error token is "202505300#30") ./output.sh: line 20: 202506030#03: invalid arithmetic base (error token is "202506030#03") ./output.sh: line 20: 202506040#04: invalid arithmetic base (error token is "202506040#04") ./output.sh: line 20: 202506040#04: invalid arithmetic base (error token is "202506040#04") ./output.sh: line 20: 202506050#05: invalid arithmetic base (error token is "202506050#05") ./output.sh: line 20: 202506050#05: invalid arithmetic base (error token is "202506050#05") ./output.sh: line 20: 202506060#06: invalid arithmetic base (error token is "202506060#06") ./output.sh: line 20: 202506060#06: invalid arithmetic base (error token is "202506060#06") ./output.sh: line 20: 202506090#09: invalid arithmetic base (error token is "202506090#09") ./output.sh: line 20: 202506090#09: invalid arithmetic base (error token is "202506090#09") ./output.sh: line 20: 202506100#10: invalid arithmetic base (error token is "202506100#10") ./output.sh: line 20: 202506100#10: invalid arithmetic base (error token is "202506100#10") ./output.sh: line 20: 202506110#11: invalid arithmetic base (error token is "202506110#11") ./output.sh: line 20: 202506110#11: invalid arithmetic base (error token is "202506110#11") ./output.sh: line 20: 202506120#12: invalid arithmetic base (error token is "202506120#12") ./output.sh: line 20: 202506120#12: invalid arithmetic base (error token is "202506120#12") ./output.sh: line 20: 202506130#13: invalid arithmetic base (error token is "202506130#13") ./output.sh: line 20: 202506130#13: invalid arithmetic base (error token is "202506130#13") ./output.sh: line 20: 202506160#16: invalid arithmetic base (error token is "202506160#16") ./output.sh: line 20: 202506160#16: invalid arithmetic base (error token is "202506160#16") ./output.sh: line 20: 202506170#17: invalid arithmetic base (error token is "202506170#17") ./output.sh: line 20: 202506170#17: invalid arithmetic base (error token is "202506170#17") ./output.sh: line 20: 202506180#18: invalid arithmetic base (error token is "202506180#18") ./output.sh: line 20: 202506180#18: invalid arithmetic base (error token is "202506180#18") ./output.sh: line 20: 202506190#19: invalid arithmetic base (error token is "202506190#19") ./output.sh: line 20: 202506190#19: invalid arithmetic base (error token is "202506190#19") ./output.sh: line 20: 202506200#20: invalid arithmetic base (error token is "202506200#20") ./output.sh: line 20: 202506200#20: invalid arithmetic base (error token is "202506200#20") ./output.sh: line 20: 202506230#23: invalid arithmetic base (error token is "202506230#23") ./output.sh: line 20: 202506230#23: invalid arithmetic base (error token is "202506230#23") ./output.sh: line 20: 202506240#24: invalid arithmetic base (error token is "202506240#24") ./output.sh: line 20: 202506240#24: invalid arithmetic base (error token is "202506240#24") ./output.sh: line 20: 202506250#25: invalid arithmetic base (error token is "202506250#25") ./output.sh: line 20: 202506250#25: invalid arithmetic base (error token is "202506250#25") ./output.sh: line 20: 202506260#26: invalid arithmetic base (error token is "202506260#26") ./output.sh: line 20: 202506260#26: invalid arithmetic base (error token is "202506260#26") ./output.sh: line 20: 202506270#27: invalid arithmetic base (error token is "202506270#27") ./output.sh: line 20: 202506270#27: invalid arithmetic base (error token is "202506270#27") ./output.sh: line 20: 202506300#30: invalid arithmetic base (error token is "202506300#30") ./output.sh: line 20: 202506300#30: invalid arithmetic base (error token is "202506300#30") ./output.sh: line 20: 202507010#01: invalid arithmetic base (error token is "202507010#01") ./output.sh: line 20: 202507010#01: invalid arithmetic base (error token is "202507010#01") ./output.sh: line 20: 202507020#02: invalid arithmetic base (error token is "202507020#02") ./output.sh: line 20: 202507020#02: invalid arithmetic base (error token is "202507020#02") ./output.sh: line 20: 202507030#03: invalid arithmetic base (error token is "202507030#03") ./output.sh: line 20: 202507030#03: invalid arithmetic base (error token is "202507030#03") ./output.sh: line 20: 202507040#04: invalid arithmetic base (error token is "202507040#04") ./output.sh: line 20: 202507040#04: invalid arithmetic base (error token is "202507040#04") ./output.sh: line 20: 202507070#07: invalid arithmetic base (error token is "202507070#07") ./output.sh: line 20: 202507070#07: invalid arithmetic base (error token is "202507070#07") ./output.sh: line 20: 202507080#08: invalid arithmetic base (error token is "202507080#08") ./output.sh: line 20: 202507080#08: invalid arithmetic base (error token is "202507080#08") ./output.sh: line 20: 202507090#09: invalid arithmetic base (error token is "202507090#09") ./output.sh: line 20: 202507090#09: invalid arithmetic base (error token is "202507090#09") ./output.sh: line 20: 202507100#10: invalid arithmetic base (error token is "202507100#10") ./output.sh: line 20: 202507100#10: invalid arithmetic base (error token is "202507100#10") ./output.sh: line 20: 202507110#11: invalid arithmetic base (error token is "202507110#11") ./output.sh: line 20: 202507110#11: invalid arithmetic base (error token is "202507110#11") ./output.sh: line 20: 202507140#14: invalid arithmetic base (error token is "202507140#14") ./output.sh: line 20: 202507140#14: invalid arithmetic base (error token is "202507140#14") ./output.sh: line 20: 202507150#15: invalid arithmetic base (error token is "202507150#15") ./output.sh: line 20: 202507150#15: invalid arithmetic base (error token is "202507150#15") ./output.sh: line 20: 202507160#16: invalid arithmetic base (error token is "202507160#16") ./output.sh: line 20: 202507160#16: invalid arithmetic base (error token is "202507160#16") ./output.sh: line 20: 202507170#17: invalid arithmetic base (error token is "202507170#17") ./output.sh: line 20: 202507170#17: invalid arithmetic base (error token is "202507170#17") ./output.sh: line 20: 202507180#18: invalid arithmetic base (error token is "202507180#18") ./output.sh: line 20: 202507180#18: invalid arithmetic base (error token is "202507180#18") ./output.sh: line 20: 202507210#21: invalid arithmetic base (error token is "202507210#21") ./output.sh: line 20: 202507210#21: invalid arithmetic base (error token is "202507210#21") ./output.sh: line 20: 202507220#22: invalid arithmetic base (error token is "202507220#22") ./output.sh: line 20: 202507220#22: invalid arithmetic base (error token is "202507220#22") ./output.sh: line 20: 202507230#23: invalid arithmetic base (error token is "202507230#23") ./output.sh: line 20: 202507230#23: invalid arithmetic base (error token is "202507230#23") ./output.sh: line 20: 202507240#24: invalid arithmetic base (error token is "202507240#24") ./output.sh: line 20: 202507240#24: invalid arithmetic base (error token is "202507240#24") ./output.sh: line 20: 202507250#25: invalid arithmetic base (error token is "202507250#25") ./output.sh: line 20: 202507250#25: invalid arithmetic base (error token is "202507250#25") ./output.sh: line 20: 202507280#28: invalid arithmetic base (error token is "202507280#28") ./output.sh: line 20: 202507280#28: invalid arithmetic base (error token is "202507280#28") ./output.sh: line 20: 202507290#29: invalid arithmetic base (error token is "202507290#29") ./output.sh: line 20: 202507290#29: invalid arithmetic base (error token is "202507290#29") ./output.sh: line 20: 202507300#30: invalid arithmetic base (error token is "202507300#30") ./output.sh: line 20: 202507300#30: invalid arithmetic base (error token is "202507300#30") ./output.sh: line 20: 202507310#31: invalid arithmetic base (error token is "202507310#31") ./output.sh: line 20: 202507310#31: invalid arithmetic base (error token is "202507310#31") ./output.sh: line 20: 202508010#01: invalid arithmetic base (error token is "202508010#01") ./output.sh: line 20: 202508010#01: invalid arithmetic base (error token is "202508010#01") ./output.sh: line 20: 202508040#04: invalid arithmetic base (error token is "202508040#04") ./output.sh: line 20: 202508040#04: invalid arithmetic base (error token is "202508040#04") ./output.sh: line 20: 202508050#05: invalid arithmetic base (error token is "202508050#05") ./output.sh: line 20: 202508050#05: invalid arithmetic base (error token is "202508050#05") ./output.sh: line 20: 202508060#06: invalid arithmetic base (error token is "202508060#06") ./output.sh: line 20: 202508060#06: invalid arithmetic base (error token is "202508060#06") ./output.sh: line 20: 202508070#07: invalid arithmetic base (error token is "202508070#07") ./output.sh: line 20: 202508070#07: invalid arithmetic base (error token is "202508070#07") ./output.sh: line 20: 202508080#08: invalid arithmetic base (error token is "202508080#08") ./output.sh: line 20: 202508080#08: invalid arithmetic base (error token is "202508080#08")输出完整代码 echo "操作已完成,结果已汇总至 ${output_file}"这么修改这段这段代码不会输出
最新发布
08-10
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值