目录
shell 24 nginx日志分析2-统计某个时间段的IP
shell 25 nginx日志分析3-统计访问3次以上的IP
shell 26 nginx日志分析4-查询某个IP的详细访问情况
shell 27 nginx日志分析5-爬虫爬取404的次数
shell 28 nginx 日志分析6-统计每分钟的请求数
shell 29 netstat练习1-查看各个状态的连接数
shell 30 netstat练习2-查看和3306端口建立的连接
shell 31 netstat练习3-输出每个IP的连接数
shell 32 netstat练习4-输出和3306端口建立连接总的各个状态数目
shell 1 统计文件的行数
描述
写一个 bash脚本以输出一个文本文件 nowcoder.txt中的行数
示例:
假设 nowcoder.txt 内容如下:#include <iostream>
using namespace std;
int main()
{
int a = 10;
int b = 100;
cout << "a + b:" << a + b << endl;
return 0;
}
你的脚本应当输出:
9
题解:
使用linux内置命令解决
wc -l < nowcoder.txt
- c 统计字节数;- l 统计行数;- w 统计字数
awk ‘END{print NR}’ #END在结尾输出 NR记录行号(每行)
sed -n ‘$=’ nowcoder.txt
基础代码实现(空间和时间最优化)
line=0
while read p
do
((line++))
done < ./nowcoder.txt #将nowcoder.txt作为输入重定向到p中 read命令一次读一行直到无数据可读后会以非零状态退出(注意:shell中除了0以外的数都会认定为false)这种方式等同于 cat nowcoder.txt | while read p 方式,另外shell与其他与语言不一样,0一般表示代码完全执行成功所以为true。
echo $line
示例:shell中除了0以外的数都会认定为false
shell 2 打印文件的最后5行
描述
经常查看日志的时候,会从文件的末尾往前查看,于是请你写一个 bash脚本以输出一个文本文件 nowcoder.txt中的最后5行
示例:
假设 nowcoder.txt 内容如下:#include<iostream>
using namespace std;
int main()
{
int a = 10;
int b = 100;
cout << "a + b:" << a + b << endl;
return 0;
}
你的脚本应当输出:int a = 10;
int b = 100;
cout << "a + b:" << a + b << endl;
return 0;
}
题解
内置命令实现
tail -n5 ./nowcoder.txt
awk ‘NR>=4{print $0}’ nowcoder.txt \ awk 'NR>=5{print}' nowcoder.txt
$0 代表整行内容 $0 一般读取是当前文件名
基础代码实现
while read line
do
row=$((row+1))
done<nowcoder.txt
while read line
do
if [ $row -lt 6 -a $row -gt 0 ]
then
echo $line
fi
row=$((row-1))
done<nowcoder.txt
\\\
row=0
while read line
do
((row++))
if [ row -gt 4 ];then
echo $line
fi
done < nowcoder.txt
下面代码虽然与上面代码输出结果相同但是不符合题意。上面代码先统计代码数后对将其—直到最后5行然后输出。
shell 3 输出7的倍数
描述
写一个 bash脚本以输出数字 0 到 500 中 7 的倍数(0 7 14 21...)的命令
题解
内置命令实现
seq 0 7 500
seq 用于生成从一个整数到另一个整数之间的所有整数
用法:seq [选项]...尾数 或者seq [选项]...首数 尾数 或者seq [选项]...首数 增数 尾数
基础代码常规思路:
for ((i=0;i<=500;i++))
do
if [[ $i%7 -eq 0 ]];then
echo "$i"
fi
done
将500中7倍数输出
思路2:
for (( i=0;i<=500;i=i+7))
do echo $i
done
思路3:
for i in {0..500..7}
do
echo $i
done
思路2和3都一样是以7为跨度输出
shell 4 输出第5行的内容
描述
写一个 bash脚本以输出一个文本文件 nowcoder.txt 中第5行的内容。
示例:
假设 nowcoder.txt 内容如下:
welcome
to
nowcoder
this
is
shell
code
你的脚本应当输出:
is
题解
内置命令实现
head -n5 nowcoder.txt | tail -n1
awk ‘NR==5{print}’ \awk ‘NR==5{print $0}’
基础代码实现
line=1
while read value
do
if [ $line -eq 5 ]
then echo $value
fi
((line++))
done < nowcoder.txt
shell 5 打印空行的行号
描述
写一个 bash脚本以输出一个文本文件 nowcoder.txt中空行的行号,可能连续,从1开始
示例:
假设 nowcoder.txt 内容如下:a
b
c
d
e
f
你的脚本应当输出:
3
5
7
9
10
题解
内置命令实现
1、grep -n ‘^$’ nowcoder.txt | cut -d ‘:’ -f 1
2、awk ‘{if($0 == “”){print NR}}’ ./nowcoder.txt #“”代表空行==^$ 但是适用范围没有^$广
3、awk ‘/^$/{print NR}’
基础代码实现
row=0
while read line
do
((row++))
#if [ -z $line];then
#if [ ! $line];then
if [ "$line" = "" ];then
echo $row
fi
done < nowcoder.txt
shell 6 去掉空行
描述
写一个 bash脚本以去掉一个文本文件 nowcoder.txt中的空行
示例:
假设 nowcoder.txt 内容如下:abc
567
aaa
bbb
ccc
你的脚本应当输出:
abc
567
aaa
bbb
ccc
题解
内置命令实现
1、awk ‘/.../{print}’ nowcoder.txt #.代表一个或多个随机字符不包括空
2、grep . nowcoder.txt
3、awk NF nowcoder.txt #NF记录有数据的行 本身是用来记录切割后的域个数 空行相当于无域与awk ‘NF!=0{print}’原理一样
基础代码实现 -n(字符串) \ $(字符) 不为空返回true
while read line
do
#if [ -n “$line” ];then
#if [ ! -z $line ];then
if [ $line ];then
echo $line
fi
done < nowcoder.txt
\\\
while read line
do
if [ -z $line ]
then
continue
fi
done < nowcoder.txt
shell 7 打印字母数小于8的单词
描述
写一个 bash脚本以统计一个文本文件 nowcoder.txt中字母数小于8的单词。
示例:
假设 nowcoder.txt 内容如下:
how they are implemented and applied in computer
你的脚本应当输出:
how
they
areand
applied
in
说明:
不要担心你输出的空格以及换行的问题
题解
内置命令实现
awk -F “ ” ‘{for(i=0;i<=NF;i++){if(lengh($i) < 8){print $i}}}’ nowcoder.txt
基础代码实现
#read line < nowcoder.txt
for i in `cat nowcoder.txt`
do
if [ ${#i} -lt 8 ];then # ${#i}==lengh($i)
echo $i
fi
done
唯一需要解决的问题就是如何将文档中字符串一个一个的读取并进行判断
shell 8 统计所有进程占用内存大小的和
描述
假设 nowcoder.txt 内容如下:
root 2 0.0 0.0 0 0 ? S 9月25 0:00 [kthreadd]
root 4 0.0 0.0 0 0 ? I< 9月25 0:00 [kworker/0:0H]
web 1638 1.8 1.8 6311352 612400 ? Sl 10月16 21:52 test
web 1639 2.0 1.8 6311352 612401 ? Sl 10月16 21:52 test
tangmiao-pc 5336 0.0 1.4 9100240 238544 ?? S 3:09下午 0:31.70 /Applications
以上内容是通过ps aux | grep -v 'RSS TTY' 命令输出到nowcoder.txt文件下面的
请你写一个脚本计算一下所有进程占用内存大小的和:
题解
内置命令实现
awk '{(sum+=$6)}END{print sum}' nowcoder.txt\ awk -F " " '{sum+=$6}END{print sum }'\
基础代码实现
方式1:利用awk将第6行分割出来作为一个数组
sum=0
for i in `awk ‘{print $6}’ nocoder.txt` #awk 默人分隔符为空格
do
((sum+=$i))
done
echo $sum
方式2:利用read函数每次只读一行的特性将每读出一行的数据作为数组
sum=0
while read -a txt #-a 变量会被认为是个数组,然后给其赋值,默认是以空格为分割符
do
((sum+=txt[5]))
done < nowcoder.txt
echo $sum
shell 9 统计每个单词出现的个数
描述
写一个 bash脚本以统计一个文本文件 nowcoder.txt 中每个单词出现的个数。
为了简单起见,你可以假设:
nowcoder.txt只包括小写字母和空格。
每个单词只由小写字母组成。
单词间由一个或多个空格字符分隔。
示例:
假设 nowcoder.txt 内容如下:
welcome nowcoder
welcome to nowcoder
nowcoder
你的脚本应当输出(以词频升序排列):
to 1
welcome 2
nowcoder 3
说明:
不要担心个数相同的单词的排序问题,每个单词出现的个数都是唯一的。
题解
内置命令实现
awk '{for(i=1;i<=NF;i++){a[$i]+=1}}END{for(j in a) print j,a[j]}' nowcoder.txt #每次只读一行 $i会随着i的增加对应到i列上 每出现一次加一次
基础代码实现
declare -A map #关联数组
while read line #记录每个单词出现次数\没考虑空字符串情况
do
for i in $line
do
((map[$i]+=1))
done
done < nowcoder.txt
mm=()
for a in ${map[@]} #将单词出现次数传递给数组用于排序
do
mm[${#mm[@]}]=$a
done
for ((i=0;i<${#mm[@]};i++)) #${#mm[@]}数组内元素个数\排序
do
for ((j=$i+1;j<${#mm[@]};j++))
do
if [ ${mm[$i]} -gt ${mm[$j]} ]
then
q=${mm[$i]}
mm[$i]=${mm[$j]}
mm[$j]=$q
fi
done
done
for ((k=0;k<${#mm[*]};k++))#将数组内的值与关联数组中值进行比较
do
for d in ${!map[@]}
do
if [ ${map[$d]} -eq ${mm[$k]} ]
then
echo "$d ${mm[$k]}"
fi
done
done
考虑空字符串
while read line
do
for I in $line
do
if [ -z ${map[$i]} ]
then
map[$i]=1
else
((map[$i]++))
fi
done
done < nowcoder.txt
shell 10 第二列是否有重复
描述
给定一个 nowcoder.txt文件,其中有3列信息,如下实例,编写一个shell脚本来检查文件第二列是否有重复,且有几个重复,并提取出重复的行的第二列信息:
实例:
20201001 python 99
20201002 go 80
20201002 c++ 88
20201003 php 77
20201001 go 88
20201005 shell 89
20201006 java 70
20201008 c 100
20201007 java 88
20201006 go 97
结果:
2 java
3 go
题解
内置命令实现
awk '{a[$2]++}END{for(i in a){if(a[i]>1){print a[i]" "i}}}' nowcoder.txt
#a[$2]++ =((a[$2]+=1)) 当a[i]>1时输出
基础代码实现
方式1
declare -A map
while read line
do
a=(${line[@]})
map[${a[1]}]=$((${map[${a[1]}]}+1))
done
for j in ${!map[@]}
do
if [[ ${map[$j]} -gt 1 ]]
then
echo ${map[$j]}" "$j
fi
done
方式2
declare -A map
while read line
do
i=(${line[@]})
j=${i[1]}
if [ -z ${map[${j}]} ]
then
map[$j]=1
else
((map[$j]++))
fi
done < nowcoder.txt
mm=()
for i in ${!map[@]}
do
if [ ${map[$i]} -gt 1 ];then
mm[${#mm[@]}]=${map[$i]}
fi
done
for (( i=0;i<${#mm[@]};i++ ))
do
for (( j=$i+1;j<${#mm[@]};j++ ))
do
if [ ${mm[$i]} -gt ${mm[$j]} ]
then
a=${mm[$j]}
mm[$j]=${mm[$i]}
mm[$i]=a
fi
done
done
for (( i=0;i<${#mm[@]};i++ ))
do
for j in ${!map[@]}
do
if [ ${map[$j]} -eq ${mm[$i]} ]
then
echo ${map[$j]}" "$j
fi
done
done
shell 11 转置文件的内容
描述
写一个 bash脚本来转置文本文件nowcoder.txt中的文件内容。
为了简单起见,你可以假设:
你可以假设每行列数相同,并且每个字段由空格分隔
示例:
假设 nowcoder.txt 内容如下:
job salary
c++ 13
java 14
php 12
你的脚本应当输出(以词频升序排列):
job c++ java php
salary 13 14 12
题解
内置命令实现
awk '{printf $1" "}' nowcoder.txt #print是按照每一行打印遵循文档格式 printf则是不换行打印
awk '{printf $2" "}' nowcoder.txt
awk '{for(i=1;i<=NF;i++){if(NR==1){a[i]=$i}else{a[i]=a[i] $i}}}END{for(j in a){print a[j]}}' nowcoder.txt #a[i]定义一个二维数组 a[$i]是一个关联数组 数组在读取字符串通过拼接的方式拼接成一个字符串
基础代码实现
方式1
declare -A map
declare -A nm
while read line
do
i=(${line[@]})
nm[${#nm[@]}]=${i[0]}
map[${#map[@]}]=${i[1]}
done < nowcoder.txt
echo ${nm[@]}
echo ${map[@]}
方式2
lines=()
while read line
do
j=(${line[@]})
for ((i=0;i<2;i++))
do
lines[$i]=${lines[$i]}" "${j[$i]}
done
done<nowcoder.txt
echo ${lines[0]}
echo ${lines[1]}
shell 12 打印每一行出现的数字个数
描述
写一个 bash脚本以统计一个文本文件 nowcoder.txt中每一行出现的1,2,3,4,5数字个数并且要计算一下整个文档中一共出现了几个1,2,3,4,5数字数字总数。
示例:
假设 nowcoder.txt 内容如下:
a12b8
10ccc
2521abc
9asf
你的脚本应当输出:
line1 number: 2
line2 number: 1
line3 number: 4
line4 number: 0
sum is 7
说明:
不要担心你输出的空格以及换行的问题
题解
内置命令实现
awk -F "[1 2 3 4 5]" 'BEGIN{su=0}{print "line"NR" number:"(NF-1);su+=(NF-1)}END{print "sum is "su}' nowcoder.txt #利用字符串中的数字将字符串分割多个域 NF-1就等于数字数量
awk -F "" 'BEGIN{su=0}{c=0;for(i=1;i<=NF;i++){if ($i ~ /[1-5]/){c+=1}}su+=c;print "line"NR" number:"c"\n"}END{print "sum is "su}' nowcoder.txt #先将字符串分割成多个域,利用正则表达式对比字符串中的每一个元素
基础代码实现
sum=0
count=0
while read line
do
for ((j=0;j<${#line};j++))
do
if [[ ${line:$j:1} =~ [1-5] ]] #[[ ]]支持正则表达式 if [[ ${tmp} -eq 1 || ${tmp} -eq 2 || ${tmp} -eq 3 || ${tmp} -eq 4 || ${tmp} -eq 5 ]]在运行速度上比正则快
then
((count++))
fi
done
((linecount+=1))
echo "line$linecount number:$count"
((sum+=$count))
count=0
done < nowcoder.txt
echo "sum is $sum"
shell 13 去掉所有包含this的句子
描述
写一个 bash脚本以实现一个需求,去掉输入中含有this的语句,把不含this的语句输出
示例:
假设输入如下:
that is your bag
is this your bag?
to the degree or extent indicated.
there was a court case resulting from this incident
welcome to nowcoder
你的脚本获取以上输入应当输出:
that is your bag
to the degree or extent indicated.
welcome to nowcoder
说明:
你可以不用在意输出的格式,包括空格和换行
题解
内置命令实现
grep -v "this" nowcoder.txt
awk '$0!~ /this/{print $0}' nowcoder.txt
awk '!/this/ {print $0}'
基础代码实现
方式1 #直接利用正则表达式
while read line
do
if [[ $line =~ "this" ]]
then
continue
else
echo $line
fi
done < nowcoder.txt
方式2 #对比筛选
while read line
do
flag=0
for i in $line
do
if [[ $i = "this" ]]
then
flag=1
break
fi
done
if [ $flag -eq 0 ]
then
echo $line
fi
done < nowcoder.txt
shell 14 求平均值
描述
写一个bash脚本以实现一个需求,求输入的一个的数组的平均值
第1行为输入的数组长度N
第2~N行为数组的元素,如以下为:
数组长度为4,数组元素为1 2 9 8
示例:
4
1
2
9
8
那么平均值为:5.000(保留小数点后面3位)
你的脚本获取以上输入应当输出:
5.000
题解
内置命令实现 scale=2小数点2位 echo "scale=2;$a/$b" | bc
awk '{if (NR!=1){sum+=$1}}END{printf ("%0.3f\n",sum/(NR-1))}' nowcoder.txt
#printf 才能使用%0.3f等 print不能使用
基础代码实现
sum=0
i=0
while read line
do
if [ $i -eq 0 ]
then
((i+=$line))
else
((sum+=$line))
fi
done
echo "scale=3;$sum/$i" | bc
shell 15 去掉不需要的单词
描述
写一个 bash脚本以实现一个需求,去掉输入中的含有B和b的单词
示例:
假设输入如下:
big
nowcoder
Betty
basic
test
你的脚本获取以上输入应当输出:
nowcoder test
说明:
你可以不用在意输出的格式,空格和换行都行
题解
内置命令实现
grep -v '[b B]' nowcoder.txt
grep -iv 'b' nowcoder.txt #-i 忽略英文大小写
awk '$0 !~ /b|B/{printf $0}' nowcoder.txt
awk '!/[b B]/{printf $0}' nowcoder.txt
基础代码实现
方式1 #逐个元素对比
while read line
do
i=${#line}
flag=0
for ((j=0;j<$i;j++))
do
if [ ${line:j:1} = "B" -o ${line:j:1} = "b" ]
then
flag=1
fi
done
if [ $flag != 1 ]
then
echo "$line"
fi
done < nowcoder.txt
方式2 #利用正则对比
while read line
do
if [[ $line =~ [bB] ]]
then
continue
else
echo "$line"
fi
done < nowcoder.txt
方式3 #不理解
while read line
do
my_array=("${my_array[@]}" $line)
done
declare -a pattern=(${my_array[@]/*[Bb]*/})
echo ${pattern[@]}
shell 16 判断输入的是否为IP地址
描述
写一个脚本统计文件nowcoder.txt中的每一行是否是正确的IP地址。
如果是正确的IP地址输出:yes
如果是错误的IP地址,四段号码的话输出:no,否则的话输出:error
假设nowcoder.txt内容如下
192.168.1.1
192.168.1.0
300.0.0.0
123你的脚本应该输出
yes
yes
no
error
题解
内置命令实现
awk -F "." '{if(NF==4){for(i=1;i<5;i++){if($i >255 || $i <0){print("no");break}}if(i==5){print("yes")}else{print("error")}}}' nowcoder.txt #纯awk
awk -F "." '{if(NF == 4){if($0 !~ /^(([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4])\.){3}([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4])$/){print("no")}else{print("yes")}}else{print("error")}}' nowcoder.txt
#正则
此题关键在于正则匹配
0-255数字匹配
- 0-99匹配表达式:[1-9]?[0-9] #?表示前面字符匹配一次或者0次
- 100-199匹配表达式:1[0-9]{2} #{2}表示前一个匹配方式重复两次
- 200-249匹配表达式:2[0-4][0-9]
- 250-255匹配表达式:25[0-4]
- 0-255 匹配表达式:[1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4]
正确ip的匹配表达式:^(([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4])\.){3}([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4])$ #^匹配开头 $匹配结尾
基础代码实现
方式1
IFS='.' #IFS 内部字段分隔符(分割字符串为数值 需要先定义数组
while read line
do
arr=(${line})
# arr=(${line//./ }) #用空格分隔符替换.
if [ ${#arr[@]} -ne 4 ]
then
echo "error"
else
for ((i=0;i < ${#arr[@]};i++))
do
if [ ${arr[$i]} -gt 255 -o ${arr[$i]} -lt 0 ]
then
echo "no"
break
fi
done
[ $i == 4] && echo "yes"
# if [ $i == 4 ]
# then
# echo "yes"
# fi
fi
done < nowcoder.txt
方式2 #正则
while read line
do
if [[ $line =~ ^([0-9]+\.){3}([0-9]+)$ ]] #在判断框内‘’中间的语句会被判定为字符串 例如:‘^([0-9]+\.){3}([0-9]+)$’会被作为字符串处理
then
if [[ $line =~ ^(([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4])\.){3}([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-4])$ ]]
then
echo "yes"
else
echo "no"
fi
else
echo "error"
fi
done < nowcoder.txt
shell 17 将字段逆序输出文件的每行
描述
将字段逆序输出文件nowcoder.txt的每一行,其中每一字段都是用英文冒号: 相分隔。
假设nowcoder.txt内容如下:
nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false
root:*:0:0:System Administrator:/var/root:/bin/sh
你的脚本应当输出
/usr/bin/false:/var/empty:Unprivileged User:-2:-2:*:nobody
/bin/sh:/var/root:System Administrator:0:0:*:root
题解
内置命令实现
awk -F ':' '{for(i=NF;i>1;i--){printf "%s:",$i}}{print $1}' nowcoder.txt
基础代码实现
IFS=':'
while read -a line
do
for ((i=${#line[@]}-1;i>-1;i--))
do
if [ $i -gt 0 ]
then
echo -n "${line[$i]}:" #-n 不输出新行= -e “sda\c”
#echo -e “${line[$i]}:\c”
#printf "${line[$i]}:"
else
echo "${line[$i]}"
#printf "${line[$i]}"
fi
done
done < nowcoder.txt
shell 18 域名进行计数排序处理
描述
假设我们有一些域名,存储在nowcoder.txt里,现在需要你写一个脚本,将域名取出并根据域名进行计数排序处理。
假设nowcoder.txt内容如下:
http://www.nowcoder.com/index.html
http://www.nowcoder.com/1.html
http://m.nowcoder.com/index.html
你的脚本应该输出:
1 m.nowcoder.com
题解
内置命令实现
awk -F '/' '{a[$3]+=1}END{for(i in a){print a[i],i}}' nowcoder.txt | sort -r #sort -r 以相反的顺序排序
基础代码实现
declare -A map
IFS='/'
while read -a line
do
((map[${line[2]}]+=1))
done < nowcoder.txt
arr=()
for i in ${!map[@]}
do
arr[${#arr[@]}]=${map[$i]}
done
for ((i=0;i<${#arr[@]};i++))
do
for ((j=i+1;j<${#arr[@]};j++))
do
if [ ${arr[$i]} -lt ${arr[$j]} ]
then
f=${arr[$i]}
arr[$i]=${arr[$j]}
arr[$j]=$f
fi
done
done
for ((k=0;k<${#arr[@]};k++))
do
for h in ${!map[@]}
do
if [ ${arr[$k]} -eq ${map[$h]} ]
then
echo "${map[$h]} ${h}"
fi
done
done
shell 19 打印等腰三角形
描述
打印边长为5的等腰三角形。
你的脚本应该输出
*
* *
* * *
* * * *
* * * * *
题解
内置命令实现
awk '{for(i=1;i<=5;i++){str="";for(j=5-i;j>=1;j--){str=str " "}for(a=1;a<=i;a++){str=str "*" " "}print str}}'
基础代码实现
for ((i=1;i<=5;i++))
do
for ((j=1;j<=5-$i;j++))
do
echo -e " \c"
done
for ((a=1;a<=$i;a++))
do
echo -e "* \c"
done
echo ""
done
shell 20 打印只有一个数字的行
描述
假设我们有一个nowcoder.txt,现在需要你写脚本,打印只有一个数字的行。
假设nowcoder.txt内容如下
haha
1
2ab
cd
77
那么你的脚本应该输出
1
2ab
题解
内置命令实现
awk -F '[0-9]' '{if(NF==2){print $0}}' nowcoder.txt
基础代码实现
while read line
do
a=0
for ((i=0;i<${#line};i++))
do
[[ ${line:$i:1} =~ [0-9] ]] && ((a+=1))
done
if [ $a -eq 1 ]
then
echo "${line}"
fi
done < nowcoder.txt
shell 21 格式化输出
描述
我们有一个文件nowcoder.txt,里面的每一行都是一个数字串,假设数字串为“123456789”,那么我们要输出为123,456,789。
假设nowcoder.txt内容如下
1
12
123
1234
123456
那么你的脚本输出如下
1
12
123
1,234
123,456
题解
内置命令实现
awk -F "" '{for(j=1;j<=NF;j++){if((NF-j)%3==0&&j!=NF){printf $j","}else{printf $j}}printf "\n"}' nowcoder.txt #NF-i 等于剩余字符数量;剩余字符数量等于3的倍数时需要加点
awk -F "" '{k=0;for(i=NF;i>0;i--){k++;str=sprintf("%s%s",$i,str);if(k%3==0&&NF>3&&i>=2){str=sprintf(",%s",str)}}print str;str=""}' nowcoder.txt #逆序 每三个字符打印一个,排除NF<=3的情况和剩余字符为3的情况且(i>=2 剩余1个字符时不打印)
基础代码实现
方式1 #利用printf 实现格式化输出
while read line
do
printf "%'d\n" $line
done < nowcoder.txt
方式2
while read line
do
len=${#line}
k=0
res=""
for ((i=$len-1;i>=0;i--))
do
((k++))
res=${line:$i:1}$res
(($k%3==0)) && (($i!=0)) && res=","$res
done
echo $res
done < nowcoder.txt
shell 22 处理文本
描述
假设我们有一个nowcoder.txt,假设里面的内容如下
111:13443
222:13211
111:13643
333:12341
222:12123
现在需要你写一个脚本按照以下的格式输出
[111]
13443
13643
[222]
13211
12123
[333]
12341
题解
内置命令实现
awk -F ":" '{a[$1]=a[$1] $2 "\n"}END{for (i in a){printf "[%s]\n%s",i,a[i]}}' nowcoder.txt #拼接字符串 输出即可 “\n“控制换行
基础代码实现
IFS=':'
declare -A map
while read -a line
do
map[${line[0]}]=${map[${line[0]}]}${line[1]}"\n"
done < nowcoder.txt
k=0
for i in ${!map[@]}
do
((k==0)) && k=1 && a="[${i}]\n${map[$i]}" && continue #将第一个字符串输出调换至最后
printf "[${i}]\n${map[$i]}"
done
printf $a
shell 23 nginx日志分析1-IP统计
描述
假设nginx的日志我们存储在nowcoder.txt里,格式如下:
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
现在需要你统计出2020年4月23号的访问ip次数,并且按照次数降序排序。你的脚本应该输出:
5 192.168.1.22
4 192.168.1.21
3 192.168.1.20
2 192.168.1.25
1 192.168.1.24
题解
内置命令实现
grep "23/Apr/2020" nowcoder.txt | awk '{a[$1]++}END{for (i in a){print a[i]" "i}}' | sort -r
基础代码实现
declare -A map
while read -a line
do
[[ ${line[3]} =~ 23/Apr/2020 ]] && ((map[${line[0]}]++))
done < nowcoder.txt
function shellsort(){
tmp=()
for i in ${map[@]}
do
tmp[${#tmp[@]}]=${map[$i]}
done
len=${#tmp[@]}
for ((gap=len/2;gap > 0;gap/=2))
do
for ((j=$gap;j<$len;j++))
do
temp=${tmp[$j]}
for ((a=$j-$gap; a>=0 & ${temp}>${tmp[$a]} ; a-=$gap))
do
tmp[$a+$gap]=${tmp[$a]}
done
tmp[$a+$gap]=$temp
done
done
}
shellsort
for ((b=0;i<$len;b++))
do
for k in ${!map[@]}
do
if [[ ${tmp[$b]} == ${map[$k]} ]] #函数内定义的变量默认为全局变量
then
printf "${tmp[$b]} ${k}\n"
fi
done
done
#希尔排序(分组插入冒泡排序)上诉代码运行不成功因时间过长
shell 24 nginx日志分析2-统计某个时间段的IP
描述
假设nginx的日志我们存储在nowcoder.txt里,格式如下:
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"现在你需要统计2020年04月23日20-23点的去重IP访问量,你的脚本应该输出
5
题解
内置命令实现
grep "23/Apr/2020:2[0-3]" nowcoder.txt | awk '{a[$1]++}END{print length(a)}'
基础代码实现
declare -A map
while read -a line
do
[[ ${line[3]} =~ 23/Apr/2020:2[0-3] ]] && ((map[${line[0]}]+=1))
done < nowcoder.txt
echo "${#map[@]}"
shell 25 nginx日志分析3-统计访问3次以上的IP
描述
假设nginx的日志我们存储在nowcoder.txt里,格式如下:
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"现在需要你写脚本统计访问3次以上的IP,你的脚本应该输出
6 192.168.1.22
5 192.168.1.21
4 192.168.1.20
题解
内置命令实现
awk '{a[$1]++}END{for (i in a){if (a[i]>3){print a[i]" "i}}}' nowcoder.txt | sort -r
基础代码实现
declare -A map
while read -a line
do
((map[${line[0]}]++))
done < nowcoder.txt
for j in ${!map[@]}
do
[ ${map[$j]} -le 3 ] && unset map[$j]
done
function bubblesort(){
tmp=()
for i in ${!map[@]}
do
tmp[${#tmp[@]}]=${map[$i]}
done
q=${#tmp[@]}
for ((a=0;a<$q-1;a++))
do
for ((b=0;b<${#tmp[@]}-1-a;b++))
do
if [[ ${tmp[$b]} < ${tmp[$b+1]} ]]
then
temp=${tmp[$b+1]}
tmp[$b+1]=${tmp[$b]}
tmp[$b]=${temp}
fi
done
done
}
bubblesort
for ((c=0;c<${#tmp[@]};c++))
do
for x in ${!map[@]}
do
if [[ ${tmp[$c]} == ${map[$x]} ]]
then
printf "${map[$x]} $x\n"
fi
done
done
:<<EOF
function InsertSort(){
tmp=()
for ve in ${map[*]}
do
tmp[${#tmp[*]}]=$ve
done
q=${#tmp[*]}
for ((i=0;i<$q;i++))
do
for ((j=$i+1;j<$q;j++))
do
if [ ${tmp[$i]} -lt ${tmp[$j]} ];then
t=${tmp[$i]}
tmp[$i]=${tmp[$j]}
tmp[$j]=$t
fi
done
done
}
EOF
shell 26 nginx日志分析4-查询某个IP的详细访问情况
描述
假设nginx的日志我们存储在nowcoder.txt里,格式如下:
192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:16:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"现在需要你查询192.168.1.22的详细访问情况,按访问频率降序排序。你的脚本应该输出
4 /1/index.php
2 /3/index.php
题解
内置命令实现
grep "192.168.1.22" nowcoder.txt | awk '{a[$7]++}END{for (i in a){print a[i]" "i}}' | sort -r
基础代码实现
declare -A map
while read -a line
do
[[ ${line[0]} =~ 192.168.1.22 ]] && ((map[${line[6]}]++))
done < nowcoder.txt
function insertsort(){
tmp=()
for i in ${!map[@]}
do
tmp[${#tmp[@]}]=${map[$i]}
done
for ((j=0;j<${#tmp[@]};j++))
do
for ((a=$j+1;a<${#tmp[@]};a++))
do
if [ ${tmp[$j]} -lt ${tmp[$a]} ]
then
t=${tmp[$a]}
tmp[$a]=${tmp[$j]}
tmp[$j]=$t
fi
done
done
}
insertsort
for ((b=0;b<${#tmp[@]};b++))
do
for i in ${!map[@]}
do
if [ ${tmp[$b]} -eq ${map[$i]} ]
then
echo "${map[$i]} $i"
fi
done
done
shell 27 nginx日志分析5-爬虫爬取404的次数
描述
假设nginx的日志我们存储在nowcoder.txt里,格式如下:
192.168.1.20 - - [21/Apr/2020:14:12:49 +0800] "GET /1/index.php HTTP/1.1" 301 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:00:49 +0800] "GET /2/index.php HTTP/1.1" 500 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:21:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:10:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:00:49 +0800] "GET /2/index.php HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:05:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +百度用户服务中心-站长平台)"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:12:49 +0800] "GET /1/index.php HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:00:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:00:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +百度用户服务中心-站长平台)"
192.168.1.22 - - [23/Apr/2020:15:00:49 +0800] "GET /3/index.php HTTP/1.1" 200 490 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +百度用户服务中心-站长平台)"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.php HTTP/1.1" 200 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 300 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 500 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:10:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:59:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"现在需要你统计百度爬虫抓取404的次数,你的脚本应该输出
2
题解
内置命令实现
grep "www.baidu.com/search/spider.html" nowcoder.txt | grep '404' | wc -l
awk '{if($9 ~ /404/ && $15 ~/baidu/){k++}}END{print k}' nowcoder.txt #在awk中使用不需要/不能加= !是可以使用
基础代码实现
while read -a line
do
[[ ${line[8]} =~ 404 ]] && [[ ${line[14]} =~ baidu ]] && ((b++))
done < nowcoder.txt
echo $b
shell 28 nginx 日志分析6-统计每分钟的请求数
描述
假设nginx的日志我们存储在nowcoder.txt里,格式如下:
192.168.1.20 - - [21/Apr/2020:14:12:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [21/Apr/2020:15:00:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [21/Apr/2020:21:21:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.23 - - [21/Apr/2020:22:10:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [22/Apr/2020:15:00:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [22/Apr/2020:15:26:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:08:05:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Baiduspider"
192.168.1.21 - - [23/Apr/2020:09:20:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:10:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:14:12:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:15:00:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:15:00:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Baiduspider"
192.168.1.25 - - [23/Apr/2020:16:15:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.24 - - [23/Apr/2020:20:27:49 +0800] "GET /2/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.25 - - [23/Apr/2020:20:27:49 +0800] "GET /3/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.20 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:20:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.22 - - [23/Apr/2020:22:10:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"
192.168.1.21 - - [23/Apr/2020:23:59:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"现在需要你统计每分钟的请求数,并且按照请求数降序排序。你的脚本应该输出
5 20:27
4 15:00
2 22:10
2 14:12
2 10:27
1 23:59
1 21:21
1 16:15
1 15:26
1 09:20
1 08:05
题解
内置命令实现
awk -F ":" '{a=$2":"$3;b[a]++}END{for(i in b){print b[i]" "i}}' nowcoder.txt | sort -r
基础代码实现
方式1
declare -A map
while read line
do
ar=(${line//:/ })
((map["${ar[4]}:${ar[5]}"]++))
done < nowcoder.txt
function shellsort(){
t=()
for i in ${!map[@]}
do
t[${#t[@]}]=${map[$i]}
done
q=${#t[@]}
for ((gap=$q/2 ; gap > 0 ; gap=$gap/2 ))#控制间距
do
for ((j=$gap ; j < q ; j++))#循环
do
tp=${t[$j]}
for ((a=j-gap ; a>=0 & tp>t[$a] ; a-=gap))#循环判断
do
t[$a+$gap]=t[$a]
done
t[$a+$gap]=$tp
done
done
}
shellsort
for ((a=0;a<$q;a++))
do
for b in ${!map[@]}
do
if [ ${t[$a]} -eq ${map[$b]} ]
then
echo "${t[$a]} $b"
unset map[$b]#避免重复的数字重复输出
fi
done
done #重复的数字顺序不一样 shellsort有问题请换下面bubblesort
:>>EOF
function bubblesort(){
t=()
for i in ${!map[@]}
do
t[${#t[@]}]=${map[$i]}
done
q=${#t[@]}
for ((k=0;k<$q-1;k++))
do
for ((j=0;j<$q-1-$k;j++))
do
if [ ${t[$j+1]} -gt ${t[$j]} ]
then
tp=${t[$j]}
t[$j]=${t[$j+1]}
t[$j+1]=$tp
fi
done
done
}
bubblesort
EOF
方式2
declare -A map
while read line
do
ar=(${line//:/ })
((map["${ar[4]}:${ar[5]}"]++))
done < nowcoder.txt
for i in ${!map[*]}
do
str="${map[${i}]} $i\n$str"
done
printf "$str" | sort -r
#IFS与”//:/ “之间的差别://:/ 是将:替换成空格分隔符;IFS是以:分割;两者执行a="${ar[4]}:${ar[5]}"结果不一样;如果将IFS=’:’定为全局变量其会将新生成字符串又分割一次。
shell 29 netstat练习1-查看各个状态的连接数
描述
假设netstat命令运行的结果我们存储在nowcoder.txt里,格式如下:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7
现在需要你查看系统tcp连接中各个状态的连接数,并且按照连接数降序输出。你的脚本应该输出如下:
ESTABLISHED 22
TIME_WAIT 9
LISTEN 3
题解
内置命令实现
awk '/tcp/{a[$6]++}END{for(i in a){print i" "a[i]}}' nowcoder.txt | sort -nrk 2 #-n大小顺序 -r反序 -k 2 按第二列
基础代码实现
declare -A map
while read -a line
do
[[ ${line[0]} == "tcp" ]] && ((map[${line[5]}]++))
done < nowcoder.txt
function insertsort(){
t=()
for i in ${!map[@]}
do
t[${#t[@]}]=${map[$i]}
done
p=${#t[@]}
for ((j=0;j<$p;j++))
do
for ((a=$j+1;a<$p;a++))
do
if [ ${t[$j]} -lt ${t[$a]} ]
then
tp=${t[$j]}
t[$j]=${t[$a]}
t[$a]=$tp
fi
done
done
}
insertsort
for ((b=0;b<$p;b++))
do
for i in ${!map[@]}
do
if [ ${t[$b]} -eq ${map[$i]} ]
then
echo $i" "${t[$b]}
fi
done
done
shell 30 netstat练习2-查看和3306端口建立的连接
描述
假设netstat命令运行的结果我们存储在nowcoder.txt里,格式如下:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.143:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7现在需要你查看和本机3306端口建立连接并且状态是established的所有IP,按照连接数降序排序。你的脚本应该输出
10 172.16.0.24
9 172.16.34.144
1 172.16.34.143
题解
内置命令实现
awk -F ":" '/ESTABLISHED/{if($3 ~ "3306"){print $2}}' nowcoder.txt | awk '{a[$2]++}END{for(i in a){print a[i]" "i}}' | sort -nr
基础代码实现
declare -A map
while read line
do
t=(${line//:/ })
if [[ ${t[6]} == 3306 && ${t[7]} == ESTABLISHED ]]
then
((map[${t[5]}]++))
fi
done < nowcoder.txt
tp=()
for i in ${map[@]}
do
tp[${#tp[@]}]=$i
done
q=${#tp[@]}
for (( gap=q/2; gap>0; gap=gap/2 ))
do
for (( i=gap; i<q; i++ ))
do
temp=${tp[$i]}
for (( a=i-gap; a>=0&temp>${tp[$a]}; a-=gap )) #默认升序, 如果降序排序改为(( a=i-gap; a>=0&temp>${arr[$a]}; a-=gap ))
do
tp[$a+$gap]=${tp[$a]}
done
tp[$a+$gap]=$temp
done
done
for ((i=0;i<$q;i++))
do
for j in ${!map[@]}
do
if [ ${tp[$i]} -eq ${map[$j]} ]
then
echo ${tp[$i]}" "$j
fi
done
done
shell 31 netstat练习3-输出每个IP的连接数
描述
假设netstat命令运行的结果我们存储在nowcoder.txt里,格式如下:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.143:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7现在需要你输出每个IP的连接数,按照连接数降序排序。你的脚本应该输出
172.16.0.24 10
172.16.34.144 9
100.100.142.4 3
0.0.0.0 3
172.16.34.143 1
172.16.240.74 1
120.55.222.235 1
100.100.54.133 1
100.100.45.106 1
100.100.32.118 1
100.100.30.25 1
100.100.142.5 1
100.100.142.1 1
题解
内置命令实现
awk -F':' '/tcp/{print $2}' nowcoder.txt | awk '{a[$2]++}END{for(i in a){print i" "a[i]}}' | sort -nrk 2
awk '{if($1 == "tcp"){split($5,a,":");res[a[1]]++;}}END{for (i in res){print i" "res[i]}}' nowcoder.txt | sort -nrk2 #split 分离字符串
基础代码实现
declare -A map
while read line
do
a=(${line//:/ })
[[ ${a[0]} == tcp ]] && ((map[${a[5]}]++))
done < nowcoder.txt
for i in ${!map[@]}
do
str=$i" "${map[$i]}"\n"$str
done
echo -e $str | sort -nrk 2
shell 32 netstat练习4-输出和3306端口建立连接总的各个状态数目
描述
假设netstat命令运行的结果我们存储在nowcoder.txt里,格式如下:
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6160 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49822 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49674 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:42316 172.16.34.143:3306 ESTABLISHED
tcp 0 0 172.16.56.200:44076 172.16.240.74:6379 ESTABLISHED
tcp 0 0 172.16.56.200:49656 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58248 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:50108 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41944 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:35548 100.100.32.118:80 TIME_WAIT
tcp 0 0 172.16.56.200:39024 100.100.45.106:443 TIME_WAIT
tcp 0 0 172.16.56.200:41788 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58260 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:41812 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41854 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:58252 100.100.142.4:80 TIME_WAIT
tcp 0 0 172.16.56.200:49586 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41754 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50466 120.55.222.235:80 TIME_WAIT
tcp 0 0 172.16.56.200:38514 100.100.142.5:80 TIME_WAIT
tcp 0 0 172.16.56.200:49832 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:52162 100.100.30.25:80 ESTABLISHED
tcp 0 0 172.16.56.200:50372 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:50306 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49600 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41908 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:60292 100.100.142.1:80 TIME_WAIT
tcp 0 0 172.16.56.200:37650 100.100.54.133:80 TIME_WAIT
tcp 0 0 172.16.56.200:41938 172.16.34.144:3306 ESTABLISHED
tcp 0 0 172.16.56.200:49736 172.16.0.24:3306 ESTABLISHED
tcp 0 0 172.16.56.200:41890 172.16.34.144:3306 ESTABLISHED
udp 0 0 127.0.0.1:323 0.0.0.0:*
udp 0 0 0.0.0.0:45881 0.0.0.0:*
udp 0 0 127.0.0.53:53 0.0.0.0:*
udp 0 0 172.16.56.200:68 0.0.0.0:*
udp6 0 0 ::1:323 :::*
raw6 0 0 :::58 :::* 7现在需要你输出和本机3306端口建立连接的各个状态的数目,按照以下格式输出
TOTAL_IP表示建立连接的ip数目TOTAL_LINK表示建立连接的总数目
TOTAL_IP 3
ESTABLISHED 20
TOTAL_LINK 20
题解
内置命令实现
awk '/tcp/{if($0 ~ /3306/ && $6 == "ESTABLISHED"){a[$5]++;sum++}}END{printf ("TOTAL_IP %d\nESTABLISHED %d\nTOTAL_LINK %d",length(a),sum,sum)}' nowcoder.txt
基础代码实现
declare -A map
while read -a line
do
[[ ${line[4]} =~ 3306 ]] && [[ ${line[0]} == tcp ]] && [[ ${line[5]} =~ ESTABLISHED ]] && ((map[${line[4]}]++)) && ((sum++))
done < nowcoder.txt
printf "TOTAL_IP %d\nESTABLISHED %d\nTOTAL_LINK %d" ${#map[@]} $((sum+3)) $((sum+3)) #结构和思路没问题但是输出总和少3个
shell 33 业务分析-提取值
描述
假设我们的日志nowcoder.txt里,内容如下
12-May-2017 10:02:22.789 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:Apache Tomcat/8.5.15
12-May-2017 10:02:22.813 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:May 5 2017 11:03:04 UTC
12-May-2017 10:02:22.813 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:8.5.15.0
12-May-2017 10:02:22.814 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:Windows, OS Version:10
12-May-2017 10:02:22.814 信息 [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:x86_64现在需要你提取出对应的值,输出内容如下
serverVersion:Apache Tomcat/8.5.15
serverName:8.5.15.0
osName:Windows
osVersion:10
题解
内置命令实现
awk -F "[:,]" '{if($0 ~ /Server version/){print "serverVersion:"$4}}{if($0 ~ /Server number/){print "serverName:"$4}}{if($0 ~ /OS Name/){print "osName:"$4}}{if($0 ~ /OS Version/){print "osVersion:"$6}}' nowcoder.txt
基础代码实现
while read line
do
a=(${line//:/ })
if [[ ${line} =~ "Server version" ]]
then
echo "serverVersion:"${a[9]}" "${a[10]}
fi
if [[ ${line} =~ "Server number" ]]
then
echo "serverName:"${a[9]}
fi
if [[ ${line} =~ "OS Name" ]]
then
IFS=','
b=${a[9]}
echo "osName:"$b
fi
if [[ ${line} =~ "OS Version" ]]
then
echo "osVersion:"${a[12]}
fi
done < nowcoder.txt
shell 34 ps分析-统计VSZ,RZZ各自总和
描述
假设命令运行的结果我们存储在nowcoder.txt里,格式如下:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 37344 4604 ? Ss 2020 2:13 /sbin/init
root 231 0.0 1.5 166576 62740 ? Ss 2020 15:15 /lib/systemd/systemd-journald
root 237 0.0 0.0 0 0 ? S< 2020 2:06 [kworker/0:1H]
root 259 0.0 0.0 45004 3416 ? Ss 2020 0:25 /lib/systemd/systemd-udevd
root 476 0.0 0.0 0 0 ? S< 2020 0:00 [edac-poller]
root 588 0.0 0.0 276244 2072 ? Ssl 2020 9:49 /usr/lib/accountsservice/accounts-daemon
message+ 592 0.0 0.0 42904 3032 ? Ss 2020 0:01 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
root 636 0.0 0.0 65532 3200 ? Ss 2020 1:51 /usr/sbin/sshd -D
daemon 637 0.0 0.0 26044 2076 ? Ss 2020 0:00 /usr/sbin/atd -f
root 639 0.0 0.0 29476 2696 ? Ss 2020 3:29 /usr/sbin/cron -f
root 643 0.0 0.0 20748 1992 ? Ss 2020 0:26 /lib/systemd/systemd-logind
syslog 645 0.0 0.0 260636 3024 ? Ssl 2020 3:17 /usr/sbin/rsyslogd -n
root 686 0.0 0.0 773124 2836 ? Ssl 2020 26:45 /usr/sbin/nscd
root 690 0.0 0.0 19472 252 ? Ss 2020 14:39 /usr/sbin/irqbalance --pid=/var/run/irqbalance.pid
ntp 692 0.0 0.0 98204 776 ? Ss 2020 25:18 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 108:114
uuidd 767 0.0 0.0 28624 192 ? Ss 2020 0:00 /usr/sbin/uuidd --socket-activation
root 793 0.0 0.0 128812 3148 ? Ss 2020 0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 794 0.0 0.2 133376 9120 ? S 2020 630:57 nginx: worker process
www-data 795 0.0 0.2 133208 8968 ? S 2020 633:02 nginx: worker process
www-data 796 0.0 0.2 133216 9120 ? S 2020 634:24 nginx: worker process
www-data 797 0.0 0.2 133228 9148 ? S 2020 632:56 nginx: worker process
web 955 0.0 0.0 36856 2112 ? Ss 2020 0:00 /lib/systemd/systemd --user
web 956 0.0 0.0 67456 1684 ? S 2020 0:00 (sd-pam)
root 1354 0.0 0.0 8172 440 tty1 Ss+ 2020 0:00 /sbin/agetty --noclear tty1 linux
root 1355 0.0 0.0 7988 344 ttyS0 Ss+ 2020 0:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyS0 vt220
root 2513 0.0 0.0 0 0 ? S 13:07 0:00 [kworker/u4:1]
root 2587 0.0 0.0 0 0 ? S 13:13 0:00 [kworker/u4:2]
root 2642 0.0 0.0 0 0 ? S 13:17 0:00 [kworker/1:0]
root 2679 0.0 0.0 0 0 ? S 13:19 0:00 [kworker/u4:0]
root 2735 0.0 0.1 102256 7252 ? Ss 13:24 0:00 sshd: web [priv]
web 2752 0.0 0.0 102256 3452 ? R 13:24 0:00 sshd: web@pts/0
web 2753 0.5 0.1 14716 4708 pts/0 Ss 13:24 0:00 -bash
web 2767 0.0 0.0 29596 1456 pts/0 R+ 13:24 0:00 ps aux
root 10634 0.0 0.0 0 0 ? S Nov16 0:00 [kworker/0:0]
root 16585 0.0 0.0 0 0 ? S< 2020 0:00 [bioset]
root 19526 0.0 0.0 0 0 ? S Nov16 0:00 [kworker/1:1]
root 28460 0.0 0.0 0 0 ? S Nov15 0:03 [kworker/0:2]
root 30685 0.0 0.0 36644 2760 ? Ss 2020 0:00 /lib/systemd/systemd --user
root 30692 0.0 0.0 67224 1664 ? S 2020 0:00 (sd-pam)
root 32689 0.0 0.0 47740 2100 ? Ss 2020 0:00 /usr/local/ilogtail/ilogtail
root 32691 0.2 0.5 256144 23708 ? Sl 2020 1151:31 /usr/local/ilogtail/ilogtail现在需要你统计VSZ,RSS各自的总和(以M兆为统计),输出格式如下
MEM TOTAL
VSZ_SUM:3250.8M,RSS_SUM:179.777M
题解
内置命令实现
awk '{a+=$5;b+=$6}END{printf ("MEM TOTAL\nVSZ_SUM:%0.1fM,RSS_SUM:%0.3fM",a/1024,b/1024)}' nowcoder.txt
基础代码实现
i=1
a=0
b=0
while read -a line
do
if [ $i -gt 1 ]
then
a=$((${line[4]}+$a))
b=$((${line[5]}+$b))
fi
((i++))
done < nowcoder.txt
a=`echo "scale=1; $a/1024" | bc`
b=`echo "scale=3; $b/1024" | bc`
printf "MEM TOTAL\nVSZ_SUM:%.1fM,RSS_SUM:%.3fM\n" $a $b