linux+shell+脚本+过滤NetScreen防火墙日志

一直想学习linux，可是没得时间。前二天，中华财险要求二天现场支持，这二天的时间，看了一些学习资料。看到公司的防火墙日志，试着过滤一下。

防火墙日志如下：

2011-09-30 00:00:20     Local0.Notice   10.2.0.254      ns50: NetScreen device_id=0019022004000299  [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2683 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4048 dst_port=80 src-xlated ip=218.206.244.202 port=4679 dst-xlated ip=119.188.11.3 port=80 session_id=61727 reason=Close - AGE OUT<000>2011-09-30 00:00:20     Local0.Notice   10.2.0.254      ns50: NetScreen device_id=0019022004000299  [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2674 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4045 dst_port=80 src-xlated ip=218.206.244.202 port=15311 dst-xlated ip=119.188.11.3 port=80 session_id=62271 reason=Close - AGE OUT<000>2011-09-30 00:00:20     Local0.Notice   10.2.0.254      ns50: NetScreen device_id=0019022004000299  [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2645 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4044 dst_port=80 src-xlated ip=218.206.244.202 port=14295 dst-xlated ip=119.188.11.3 port=80 session_id=59240 reason=Close - AGE OUT<000>2011-09-30 00:00:20     Local0.Notice   10.2.0.254      ns50: NetScreen device_id=0019022004000299  [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=1485 rcvd=482 src=10.100.1.43 dst=119.188.11.3 src_port=4051 dst_port=80 src-xlated ip=218.206.244.202 port=13926 dst-xlated ip=119.188.11.3 port=80 session_id=54785 reason=Close - AGE OUT<000>2011-09-30 00:00:20     Local0.Notice   10.2.0.254      ns50: NetScreen device_id=0019022004000299  [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2682 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4046 dst_port=80 src-xlated ip=218.206.244.202 port=13692 dst-xlated ip=119.188.11.3 port=80 session_id=60623 reason=Close - AGE OUT<000>2011-09-30 00:00:20     Local0.Notice   10.2.0.254      ns50: NetScreen device_id=0019022004000299  [Root]system-notification-00257(traffic): start_time="2011-09-30 00:01:05" duration=15 policy_id=103 service=http proto=6 src zone=Trust dst zone=Untrust action=Permit sent=2605 rcvd=766 src=10.100.1.43 dst=119.188.11.3 src_port=4043 dst_port=80 src-xlated ip=218.206.244.202 port=13520 dst-xlated ip=119.188.11.3 port=80 session_id=62996 reason=Close - AGE OUT<000>

想获得每条日志的sent 数据，Recv数据，src源地址及dst目的地址，脚本如下：

#!/bin/sh

if [ ! -d /var/tmp ] ; then mkdir /var/tmpfi

if [ -e /var/tmp/sysn ] ; then  rm  /var/tmp/sysnfi

#$1为命令行的每一个参数，这里是防火墙日志的文件路经 

echo " awk { for(i=1;i<=NF;i++) { if( $i~ /sent/ ) print $i,i++,$i,i++,$i,i++,$i } } ' $1 | awk '{ print $1,$3,$5,$7 }'  >/var/tmp/sysn" echo -e "..................................."

#按照模式取出字符串 类似sent=1132 recv=3434 src=10.100.1.32 dst=211.138.24.66 

awk '{ for(i=1;i<=NF;i++) { if( $i~ /sent/ ) print $i,i++,$i,i++,$i,i++,$i } } ' $1 | awk '{ print $1,$3,$5,$7 }'  >/var/tmp/sysn if [ -e /var/tmp/sysnn ] ; then rm  /var/tmp/sysnnfi

 echo " sed 's/=/ /g' /var/tmp/sysn >/var/tmp/sysnn" echo -e "..................................."

#将=换成空格

sed 's/=/ /g' /var/tmp/sysn >/var/tmp/sysnn

if [ -e /var/tmp/sysnnn ] ; then rm /var/tmp/sysnnnfi

 echo "awk '{ sent[$6] += $2;Recv[$6] += $4 } END { for(i in sent) print i,"\t\t", sent[i],"\t\t",Recv[i] }' /var/tmp/sysnn >/var/tmp/sysnnn" echo -e "..................................."

#统计每个地址的sent和recv总数awk '{ sent[$6] += $2;Recv[$6] += $4 } END { for(i in sent) print i,"\t\t", sent[i],"\t\t",Recv[i] }' /var/tmp/sysnn >/var/tmp/sysnnn

if [ -e /var/tmp/sysnnnn ] ; then

 rm  /var/tmp/sysnnnn

fi

#按sent排序  cat /var/tmp/sysnnn | sort -n -r -k 2 | grep '^10\.'  >/var/tmp/sysnnnn 

/bin/echo -e "IP\t\t\tSend bytes(B)\t\tRecv bytes(B)\n====================================================================="

#命令行第二个参数，按recv排序

if [ "$2" = "recv" ] ; then    cat /var/tmp/sysnnnn | sort -n -r -k 3else  cat /var/tmp/sysnnnnfi

if [ -e /var/tmp/sysn ] ; then   rm /var/tmp/sysnfi

if [ -e /var/tmp/sysnn ] ; then rm /var/tmp/sysnnfi

if [ -e /var/tmp/sysnnn ] ; then  rm /var/tmp/sysnnnfi

if [ -e /var/tmp/sysnnnn ] ; then rm /var/tmp/sysnnnnfi

应用如下：

./syslogana  /usr/Syslog2011-09-30.txt    --按sent排序

或./syslogana  /usr/Syslog2011-09-30.txt recv   --按recv排序

[orcle@localhost ~]$ ./syslogana  /usr/Syslog2011-09-30.txt 

awk { for(i=1;i<=NF;i++) { if( ~ /sent/ ) print ,i++,,i++,,i++, } } ' Syslog2011-09-30.txt | awk '{ print Syslog2011-09-30.txt,,, }'  >/var/tmp/sysn................................... 

sed 's/=/ /g' /var/tmp/sysn >/var/tmp/sysnn...................................

awk '{ sent[] += ;Recv[] +=  } END { for(i in sent) print i,tt, sent[i],tt,Recv[i] }' /var/tmp/sysnn >/var/tmp/sysnnn...................................

IP                      Send bytes(B)           Recv bytes(B)=====================================================================10.2.0.195               389190206               3.21879e+0910.2.0.230               133985217               133386378710.2.0.240               86287521                50698167110.100.1.240             69406016                13480948610.2.0.249               56816187                14380941210.2.0.245               40095561                5869195010.2.0.228               36652824                18304863010.2.0.194               27172677                8062195710.2.0.252               23434488                9307896210.100.5.252             20701571                14683126610.2.0.241               18873421                65888402