shell脚本求hdfs文件小于20M的文件、文件大小最大值和最小值

最新推荐文章于 2023-10-18 17:16:37 发布

qq_23519469

最新推荐文章于 2023-10-18 17:16:37 发布

阅读量486

点赞数

文章标签： hdfs 大数据

本文链接：https://blog.youkuaiyun.com/qq_23519469/article/details/125749177

版权

hdfs dfs -du -h hdfs://sfbd/user/hive/warehouse/dm_gis.db/temp_tel_addr_unionpay_tmp1/action_date=20220318/ | awk ‘{print $1$2 “\t” $5}’ | awk ‘{if($1 ~/K/ || $1 ~/M/) print $0}’ | awk ‘{if($1 ~/M/) print $0}’|sed ‘s/M//g’| awk ‘{if($1 <20) print $1"M"“\t”$2}’
hdfs dfs -du -h hdfs://sfbd/user/hive/warehouse/dm_gis.db/temp_tel_addr_unionpay_tmp1/action_date=20220318/ | awk ‘{print $1$2 “\t” $5}’ | awk ‘{if($1 ~/K/ || $1 ~/M/) print $0}’ | awk ‘{if($1 ~/M/) print $0}’|sed ‘s/M//g’| awk ‘BEGIN {max = 0} {if ($1+0 > max+0) max=$1} END {print “Max=”, max}’
hdfs dfs -du -h hdfs://sfbd/user/hive/warehouse/dm_gis.db/temp_tel_addr_unionpay_tmp1/action_date=20220318/ | awk ‘{print $1$2 “\t” $5}’ | awk ‘{if($1 ~/K/ || $1 ~/M/) print $0}’ | awk ‘{if($1 ~/M/) print $0}’|sed ‘s/M//g’| awk ‘BEGIN {min = 65536} {if ($1+0 < min+0) min=$1} END {print “Min=”, min}’