awk应用

最新推荐文章于 2023-07-04 20:04:09 发布

原创最新推荐文章于 2023-07-04 20:04:09 发布 · 748 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#awk

本文详细介绍Awk命令的基本用法、内置变量、自定义变量、格式化输出、操作符、正则表达式等核心内容，并通过实例展示如何使用Awk进行文本处理。

基本用法：

awk [options] 'program' File ...

    program  :  pattern  {  action statements  }，语句之间用分号分割
    选项：
         -F:指明输入时的分隔符
         -v：自定义变量


         1.print item1，item2，...
               要点：
                 1.逗号分隔符
                 2.输出的item可以是字符串，也可以是数组；当前记录的字段、变量或awk表达式
                 3.如果省略item，相当于print $0
         2.变量
            2.1 内建变量
                    列分隔变量
                                 FS：input field  seperator，默认为空白字符
                                 OFS：output field seperator，默认为空白字符
                    行分隔变量，一般不适用
                                 RS: input field  seperator，输入时的换行符
                                 ORS：output field  seperator，输出时的换行符

                    NF：number of field，字段数量 {print NF}  {print  $NF}

                    NR：number of record，行数
                    FNR：各文件分别计数，行数

                    FILENAME：当前文件名

                    ARGC：命令行参数的个数
                    ARGV：数组，保存的是命令行所给定的各参数

            2.2  自定义变量
                     1. -v var=val
                     变量名区分字符大小写
                     2. 在 statements 中定义

        3、printf命令
             格式化输出：printf FORMAT, item1, item2,...
             (1) FORMAT必须给出，需要用""包裹；
             (2) 不会自动换行，需要显示给出换行控制符，\n
             (3) FORMAT中需要分别为后面的每个item指定一个格式化符号

             格式符：
                 %c    ：显示字符的ASCII码
                 %d，%i：显示十进制整数
                 %e，%E：科学计数法数值显示
                 %f    ：显示为浮点数
                 %g，%G：以科学计数法或浮点形式显示数值
                 %s    : 显示字符串
                 %u    ：无符号显示
                 %%    ：显示%本身
             修饰符：
                 #[.#]:第一个#控制显示的宽度；第二个#表示小数点后的精度
                 %3.1f
                 -：左对齐
                 +：显示数值的符号

        4、操作符
             算术操作符：
                 x+y 相加
                 x-y 相减
                 x*y 相乘
                 x/y 相除
                 x^y 次方
                 x%y 取余
                 -x  取负数
                 +x  转换为数值

             字符串操作符：
             赋值操作符：
                 = +=  -= *= /= %= ^= ++ --
             比较操作符：
                 > >= < <= !=  ==
             模式匹配符：
                 ~ ：是否匹配
                 !~ :是否不匹配
             逻辑操作符：
                 && 并且
                 || 或
                 !  非
              函数调用：
                 function_name(argu1,argu2,  ....)
              条件表达式：
                 selector?if-true-expression:if-false-expression


        5、pattern
             (1)  empty：空模式，匹配每一行
             (2)  /regular expression/  :正则表达式，仅处理能够被此处的模式匹配到的行
             (3)  relational expression  : 关系表达式 表达式结果为真时，才会被处理
             真：数值非零，字符串非空
             (4) line ranges：行范围
                 startline，stopline： /pat1/,/pat2/
                 注意不支持直接给出数字的格式 使用变量NR

             (5) BEGIN/END模式
                 BEGIN{}  :仅在开始处理文件之前执行一次
                 END{}    :仅在文本处理结束之后执行一次

        6、常用的action
             (1) expressions
             (2) control statements  :if while
             (3) compound statements :组合语句
             (4) input statements    :输入语句
             (5) output statements ： 输出语句
        7、控制语句
             if(condition) {statements}
             if(condition) {statements} else{statements}
             while(condition) {statements}
             do {statements} while(condition)
             for(expr1,expr2,expr3) {statements}
             break
             continue
             delete array[index]
             delete array
             exit
             {statements }
            7.1 if-else
                 语法：if(condition) {statements} else{statements}
                 使用场景：对awk取得的整行或某个字段进行条件判断
            7.2 while循环
                 语法：while(condition) {statements}
                 使用场景：对一行内的多个字段进行逐一类似处理，对数组中的各元素进行处理
            7.3 do-while循环
                 语法：do  statements while(condition)
                 意义：至少执行一次循环
            7.4 for循环
                 语法：for(expr1; expr2; expr3) {statements}
                       for(i=1;i<NF;i++) {}
                 特殊用法
                     能够遍历数组中的元素
                     语法：for(var in array) {for-body}
            7.5 switch语句
                语法：switch(expression) {case value1 or /regexp/: staement1;case value2 or /regexp/: staement2;...;default:statement}
            7.6 break和continue 控制字段间的跳转
                break [n]
                continue
            7.7 next  控制行间的跳转
                提前结束对本行的处理，而进入下一行
        8、数组
             关联数组：array[index-expression]
                  (1) 可使用任意字符串,字符串要使用双引号
                  (2) 如果某元素事先不存在，在引用时，awk会自动创建此元素；如果求和时会默认为0
                  若要判断数组中是否存在某元素，要使用"index in array" 格式
                  (3) 若要遍历数组中的每个元素，要使用for循环
                      for(var in array) {for-body}  这里的var为数组中元素的下标值
        9、函数
             9.1  内置函数
                 数值处理
                     rand()：返回0和1之间的一个随机数
                     并且在以后每次都是相同的值
                 字符串处理
                     length([s])：返回指定字符串的长度
                     sub(r,s,[t])：以r表示的模式来查找t所表示的字符串中匹配的内容，并将其第一次出现替换为s所表示的内容
                     gsub(r,s,[t])：以r表示的模式来查找t所表示的字符串中匹配的内容，并将其全部出现替换为s所表示的内容
                     split(s,a[,r])：以r为分隔符切割字符串s，并将切割后的结果保存至a所表示的数组中，下标从1开始

                  ()
                  ()

awk -F ‘:’ ‘pattern { action statements }’ file

使用内建变量 FS、OFS

一个-v指定一个内建变量

# awk -v FS=':' -v OFS=':' '{print $1,$2}' /etc/passwd
root:x
bin:x
daemon:x
adm:x
lp:x
sync:x
shutdown:x
halt:x
mail:x
operator:x

使用内建变量 NF：总列数，$NF：最后一列

# awk -v FS=':' '{print NF}' /etc/passwd
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
7
# awk -v FS=':' '{print $NF}' /etc/passwd
/bin/bash
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/bin/sync
/sbin/shutdown
/sbin/halt
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/sbin/nologin
/bin/bash
/bin/bash
/bin/bash
/bin/bash

NR与FNR的区别

NR在多个文件时是行号累加；FNR在多个文件时，行号是分别处理的

[root@suning ~]# wc -l /etc/passwd /etc/fstab 
  27 /etc/passwd
  13 /etc/fstab
  40 总用量
[root@suning ~]# awk '{print NR,$0}' /etc/passwd /etc/fstab 
1 root:x:0:0:root:/root:/bin/bash
2 bin:x:1:1:bin:/bin:/sbin/nologin
3 daemon:x:2:2:daemon:/sbin:/sbin/nologin
4 adm:x:3:4:adm:/var/adm:/sbin/nologin
5 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
6 sync:x:5:0:sync:/sbin:/bin/sync
7 shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
8 halt:x:7:0:halt:/sbin:/sbin/halt
9 mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
10 operator:x:11:0:operator:/root:/sbin/nologin
11 games:x:12:100:games:/usr/games:/sbin/nologin
12 ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
13 nobody:x:99:99:Nobody:/:/sbin/nologin
14 systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
15 systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
16 dbus:x:81:81:System message bus:/:/sbin/nologin
17 polkitd:x:998:996:User for polkitd:/:/sbin/nologin
18 tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
19 postfix:x:89:89::/var/spool/postfix:/sbin/nologin
20 sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
21 dockerroot:x:997:994:Docker User:/var/lib/docker:/sbin/nologin
22 apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
23 carbon:x:996:993:Carbon cache daemon:/var/lib/carbon:/sbin/nologin
24 test1:x:1000:1000::/home/test1:/bin/bash
25 test2:x:1001:1001::/home/test2:/bin/bash
26 test3:x:1002:1002::/home/test3:/bin/bash
27 test4:x:1003:1003::/home/test4:/bin/bash
28 
29 #
30 # /etc/fstab
31 # Created by anaconda on Thu Apr 27 16:28:57 2017
32 #
33 # Accessible filesystems, by reference, are maintained under '/dev/disk'
34 # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
35 #
36 /dev/mapper/cl-root     /                       ext3    defaults        1 1
37 UUID=7cd893db-e433-4e37-b509-9b8f2cd11094 /boot                   xfs     defaults        0 0
38 /dev/mapper/cl-home     /data                   xfs     defaults        0 0
39 /dev/mapper/cl-usr_local /usr/local              xfs     defaults        0 0
40 /dev/mapper/cl-swap     swap                    swap    defaults        0 0


[root@suning ~]# awk '{print FNR,$0}' /etc/passwd /etc/fstab 
1 root:x:0:0:root:/root:/bin/bash
2 bin:x:1:1:bin:/bin:/sbin/nologin
3 daemon:x:2:2:daemon:/sbin:/sbin/nologin
4 adm:x:3:4:adm:/var/adm:/sbin/nologin
5 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
6 sync:x:5:0:sync:/sbin:/bin/sync
7 shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
8 halt:x:7:0:halt:/sbin:/sbin/halt
9 mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
10 operator:x:11:0:operator:/root:/sbin/nologin
11 games:x:12:100:games:/usr/games:/sbin/nologin
12 ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
13 nobody:x:99:99:Nobody:/:/sbin/nologin
14 systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
15 systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
16 dbus:x:81:81:System message bus:/:/sbin/nologin
17 polkitd:x:998:996:User for polkitd:/:/sbin/nologin
18 tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
19 postfix:x:89:89::/var/spool/postfix:/sbin/nologin
20 sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
21 dockerroot:x:997:994:Docker User:/var/lib/docker:/sbin/nologin
22 apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
23 carbon:x:996:993:Carbon cache daemon:/var/lib/carbon:/sbin/nologin
24 test1:x:1000:1000::/home/test1:/bin/bash
25 test2:x:1001:1001::/home/test2:/bin/bash
26 test3:x:1002:1002::/home/test3:/bin/bash
27 test4:x:1003:1003::/home/test4:/bin/bash
1 
2 #
3 # /etc/fstab
4 # Created by anaconda on Thu Apr 27 16:28:57 2017
5 #
6 # Accessible filesystems, by reference, are maintained under '/dev/disk'
7 # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
8 #
9 /dev/mapper/cl-root     /                       ext3    defaults        1 1
10 UUID=7cd893db-e433-4e37-b509-9b8f2cd11094 /boot                   xfs     defaults        0 0
11 /dev/mapper/cl-home     /data                   xfs     defaults        0 0
12 /dev/mapper/cl-usr_local /usr/local              xfs     defaults        0 0
13 /dev/mapper/cl-swap     swap                    swap    defaults        0 0

FILENAME文件名

[root@suning ~]# awk '{print FILENAME,FNR,$0}' /etc/passwd /etc/fstab 
/etc/passwd 1 root:x:0:0:root:/root:/bin/bash
/etc/passwd 2 bin:x:1:1:bin:/bin:/sbin/nologin
/etc/passwd 3 daemon:x:2:2:daemon:/sbin:/sbin/nologin
/etc/passwd 4 adm:x:3:4:adm:/var/adm:/sbin/nologin
/etc/passwd 5 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
/etc/passwd 6 sync:x:5:0:sync:/sbin:/bin/sync
/etc/passwd 7 shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
/etc/passwd 8 halt:x:7:0:halt:/sbin:/sbin/halt
/etc/passwd 9 mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
/etc/passwd 10 operator:x:11:0:operator:/root:/sbin/nologin
/etc/passwd 11 games:x:12:100:games:/usr/games:/sbin/nologin
/etc/passwd 12 ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
/etc/passwd 13 nobody:x:99:99:Nobody:/:/sbin/nologin
/etc/passwd 14 systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
/etc/passwd 15 systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
/etc/passwd 16 dbus:x:81:81:System message bus:/:/sbin/nologin
/etc/passwd 17 polkitd:x:998:996:User for polkitd:/:/sbin/nologin
/etc/passwd 18 tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
/etc/passwd 19 postfix:x:89:89::/var/spool/postfix:/sbin/nologin
/etc/passwd 20 sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
/etc/passwd 21 dockerroot:x:997:994:Docker User:/var/lib/docker:/sbin/nologin
/etc/passwd 22 apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
/etc/passwd 23 carbon:x:996:993:Carbon cache daemon:/var/lib/carbon:/sbin/nologin
/etc/passwd 24 test1:x:1000:1000::/home/test1:/bin/bash
/etc/passwd 25 test2:x:1001:1001::/home/test2:/bin/bash
/etc/passwd 26 test3:x:1002:1002::/home/test3:/bin/bash
/etc/passwd 27 test4:x:1003:1003::/home/test4:/bin/bash
/etc/fstab 1 
/etc/fstab 2 #
/etc/fstab 3 # /etc/fstab
/etc/fstab 4 # Created by anaconda on Thu Apr 27 16:28:57 2017
/etc/fstab 5 #
/etc/fstab 6 # Accessible filesystems, by reference, are maintained under '/dev/disk'
/etc/fstab 7 # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
/etc/fstab 8 #
/etc/fstab 9 /dev/mapper/cl-root     /                       ext3    defaults        1 1
/etc/fstab 10 UUID=7cd893db-e433-4e37-b509-9b8f2cd11094 /boot                   xfs     defaults        0 0
/etc/fstab 11 /dev/mapper/cl-home     /data                   xfs     defaults        0 0
/etc/fstab 12 /dev/mapper/cl-usr_local /usr/local              xfs     defaults        0 0
/etc/fstab 13 /dev/mapper/cl-swap     swap                    swap    defaults        0 0

ARGC命令行参数个数；ARGV命令行参数，用ARGV[0]表示第一个参数,一般是awk，用ARGV[1]表示第二个参数,一般是第一个文件，

[root@suning ~]# awk 'BEGIN{print ARGC}' /etc/fstab 
2
[root@suning ~]# awk 'BEGIN{print ARGV[0]}' /etc/fstab 
awk
[root@suning ~]# awk 'BEGIN{print ARGV[1]}' /etc/fstab 
/etc/fstab

自定义变量

应用变量时不需要加 $ 符号

在花括号中直接定义

[root@suning ~]# awk 'BEGIN{  hello="i love you " ; print hello }' 
i love you 

使用-v选项

[root@suning ~]# awk -v hello="you love me" 'BEGIN{print hello}'
you love me

printf格式符

[root@suning ~]# awk -v FS=":"   '{printf "UserName: %s, UID: %d, \n" ,$1 ,$3}'   /etc/passwd
UserName: root, UID: 0, 
UserName: bin, UID: 1, 
UserName: daemon, UID: 2, 
UserName: adm, UID: 3, 
UserName: lp, UID: 4, 
UserName: sync, UID: 5, 
UserName: shutdown, UID: 6, 
UserName: halt, UID: 7, 
UserName: mail, UID: 8, 
UserName: operator, UID: 11, 
UserName: games, UID: 12, 
UserName: ftp, UID: 14, 
UserName: nobody, UID: 99, 
UserName: systemd-bus-proxy, UID: 999, 
UserName: systemd-network, UID: 192, 
UserName: dbus, UID: 81, 
UserName: polkitd, UID: 998, 
UserName: tss, UID: 59, 
UserName: postfix, UID: 89, 
UserName: sshd, UID: 74, 
UserName: dockerroot, UID: 997, 
UserName: apache, UID: 48, 
UserName: carbon, UID: 996, 
UserName: test1, UID: 1000, 
UserName: test2, UID: 1001, 
UserName: test3, UID: 1002, 
UserName: test4, UID: 1003,

修饰符左对齐 -

[root@suning ~]# awk -v FS=":"   '{printf "UserName: +%-20s, UID: %-d, \n" ,$1 ,$3}'   /etc/passwd
UserName: +root                , UID: 0, 
UserName: +bin                 , UID: 1, 
UserName: +daemon              , UID: 2, 
UserName: +adm                 , UID: 3, 
UserName: +lp                  , UID: 4, 
UserName: +sync                , UID: 5, 
UserName: +shutdown            , UID: 6, 
UserName: +halt                , UID: 7, 
UserName: +mail                , UID: 8, 
UserName: +operator            , UID: 11, 
UserName: +games               , UID: 12, 
UserName: +ftp                 , UID: 14, 
UserName: +nobody              , UID: 99, 
UserName: +systemd-bus-proxy   , UID: 999, 
UserName: +systemd-network     , UID: 192, 
UserName: +dbus                , UID: 81, 
UserName: +polkitd             , UID: 998, 
UserName: +tss                 , UID: 59, 
UserName: +postfix             , UID: 89, 
UserName: +sshd                , UID: 74, 
UserName: +dockerroot          , UID: 997, 
UserName: +apache              , UID: 48, 
UserName: +carbon              , UID: 996, 
UserName: +test1               , UID: 1000, 
UserName: +test2               , UID: 1001, 
UserName: +test3               , UID: 1002, 
UserName: +test4               , UID: 1003,

修饰符显示数值符号+

[root@suning ~]# cat test.txt 
usea  111
sng   -222
ss    -1111111111111111111

[root@suning ~]# awk -v FS=" "   '{printf "UserName: %-20s, UID: %+d, \n" ,$1 ,$2}'   test.txt 
UserName: usea                , UID: +111, 
UserName: sng                 , UID: -222, 
UserName: ss                  , UID: -1111111111111111168,

修饰符


                 #[.#]:第一个#控制显示的宽度；第二个#表示小数点后的精度
                 %3.1f
                 -：左对齐
                 +：显示数值的符号

[root@suning ~]# cat test.txt 
usea  111
sng   -222
ss    -1111111111111111111
xsh   -333.3333333333333

[root@suning ~]# awk -v FS=" "   '{printf "UserName: %-20s, UID: %+.4f, \n" ,$1 ,$2}'   test.txt 
UserName: usea                , UID: +111.0000, 
UserName: sng                 , UID: -222.0000, 
UserName: ss                  , UID: -1111111111111111168.0000, 
UserName: xsh                 , UID: -333.3333,

操作符（条件表达式）

              条件表达式：
                 selector?if-true-expression:if-false-expression
                 条件为真，执行if-true,否则执行if-false

[root@suning ~]# awk  -v FS=":" ' { $3>=1000?UserType="custom user":UserType="SysUser or Admin"; printf "%-20s,%s \n " ,$1,UserType }' /etc/passwd
root                ,SysUser or Admin 
 bin                 ,SysUser or Admin 
 daemon              ,SysUser or Admin 
 adm                 ,SysUser or Admin 
 lp                  ,SysUser or Admin 
 sync                ,SysUser or Admin 
 shutdown            ,SysUser or Admin 
 halt                ,SysUser or Admin 
 mail                ,SysUser or Admin 
 operator            ,SysUser or Admin 
 games               ,SysUser or Admin 
 ftp                 ,SysUser or Admin 
 nobody              ,SysUser or Admin 
 systemd-bus-proxy   ,SysUser or Admin 
 systemd-network     ,SysUser or Admin 
 dbus                ,SysUser or Admin 
 polkitd             ,SysUser or Admin 
 tss                 ,SysUser or Admin 
 postfix             ,SysUser or Admin 
 sshd                ,SysUser or Admin 
 dockerroot          ,SysUser or Admin 
 apache              ,SysUser or Admin 
 carbon              ,SysUser or Admin 
 test1               ,custom user 
 test2               ,custom user 
 test3               ,custom user 
 test4               ,custom user

正则表达式匹配

匹配以UUID开头的行  /^UUID/
[root@suning ~]# awk -v FS=" " '/^UUID/ {print $0}' /etc/fstab 
UUID=7cd893db-e433-4e37-b509-9b8f2cd11094 /boot                   xfs     defaults        0 0


匹配不以UUID开头的行  !/^UUID/
[root@suning ~]# awk -v FS=" " '!/^UUID/ {print $0}' /etc/fstab 

#
# /etc/fstab
# Created by anaconda on Thu Apr 27 16:28:57 2017
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/cl-root     /                       ext3    defaults        1 1
/dev/mapper/cl-home     /data                   xfs     defaults        0 0
/dev/mapper/cl-usr_local /usr/local              xfs     defaults        0 0
/dev/mapper/cl-swap     swap                    swap    defaults        0 0


最后一个字段以bash结尾   $NF~/bash$/
[root@suning ~]# awk -v FS=":" '$NF~/bash$/   {print $0} ' /etc/passwd
root:x:0:0:root:/root:/bin/bash
test1:x:1000:1000::/home/test1:/bin/bash
test2:x:1001:1001::/home/test2:/bin/bash
test3:x:1002:1002::/home/test3:/bin/bash
test4:x:1003:1003::/home/test4:/bin/bash

最后一个字段不以bash结尾   $NF!~/bash$/
[root@suning ~]# awk -v FS=":" '$NF!~/bash$/   {print $1,$NF} ' /etc/passwd
bin /sbin/nologin
daemon /sbin/nologin
adm /sbin/nologin
lp /sbin/nologin
sync /bin/sync
shutdown /sbin/shutdown
halt /sbin/halt
mail /sbin/nologin
operator /sbin/nologin
games /sbin/nologin
ftp /sbin/nologin
nobody /sbin/nologin
systemd-bus-proxy /sbin/nologin
systemd-network /sbin/nologin
dbus /sbin/nologin
polkitd /sbin/nologin
tss /sbin/nologin
postfix /sbin/nologin
sshd /sbin/nologin
dockerroot /sbin/nologin
apache /sbin/nologin
carbon /sbin/nologin

关系表达式

数值比较   $3>=1000
[root@suning ~]# awk -v FS=":"   '  $3>=1000   {print $0} ' /etc/passwd
test1:x:1000:1000::/home/test1:/bin/bash
test2:x:1001:1001::/home/test2:/bin/bash
test3:x:1002:1002::/home/test3:/bin/bash
test4:x:1003:1003::/home/test4:/bin/bash


字符串比较  $NF=="/bin/bash"
[root@suning ~]# awk -v FS=":" '$NF=="/bin/bash"   {print $1,$NF} ' /etc/passwd
root /bin/bash
test1 /bin/bash
test2 /bin/bash
test3 /bin/bash
test4 /bin/bash

行范围

#在 第一个字段以daemon开头，第五个字段以Apache开头  中查找
[root@suning ~]# awk -v FS=":" '$1~/^daemon/,$5~/^Apache/ {print $0} ' /etc/passwd
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:998:996:User for polkitd:/:/sbin/nologin
tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
dockerroot:x:997:994:Docker User:/var/lib/docker:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin



## 使用变量NR进行  行范围界定
[root@suning ~]# awk -v FS=":" '(NR>=2&&NR<10) {print $0} ' /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin

$5=="daemon" 和$ 5~/daemon/ 的区别

前者是绝对等于，后者是包含

[root@suning ~]# awk -v FS=":" '$5=="daemon" {print $0} ' /etc/passwd
daemon:x:2:2:daemon:/sbin:/sbin/nologin


[root@suning ~]# awk -v FS=":" '$5~/daemon/ {print $0} ' /etc/passwd
daemon:x:2:2:daemon:/sbin:/sbin/nologin
tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
carbon:x:996:993:Carbon cache daemon:/var/lib/carbon:/sbin/nologin

BEGIN和END使用

# awk -v FS=":" 'BEGIN{print "start\n------------"} $5=="daemon" {print $0} END{print "#####\nend"}' /etc/passwd
start
------------
daemon:x:2:2:daemon:/sbin:/sbin/nologin
#####
end

if(condition) {statements} else{statements}

使用场景：对awk取得的整行或某个字段进行条件判断

数值比较
# awk -v FS=":" 'BEGIN{print "start\n------------"}  { if($3>=1000){printf "comman user：%s\n",$1} else{printf "system user: %s\n",$1} } END{print "#####\nend"}' /etc/passwd
start
------------
system user: root
system user: bin
system user: daemon
system user: adm
system user: lp
system user: sync
system user: shutdown
system user: halt
system user: mail
system user: operator
system user: games
system user: ftp
system user: nobody
system user: systemd-bus-proxy
system user: systemd-network
system user: dbus
system user: polkitd
system user: tss
system user: postfix
system user: sshd
system user: dockerroot
system user: apache
system user: carbon
comman user：test1
comman user：test2
comman user：test3
comman user：test4
#####
end



字符比较
[root@suning ~]# awk -v FS=":" 'BEGIN{print "start\n------------"}  { if($NF=="/bin/bash"){printf "comman user：%s\n",$1} else{printf "system user: %s\n",$1} } END{print "#####\nend"}' /etc/passwd
start
------------
comman user：root
system user: bin
system user: daemon
system user: adm
system user: lp
system user: sync
system user: shutdown
system user: halt
system user: mail
system user: operator
system user: games
system user: ftp
system user: nobody
system user: systemd-bus-proxy
system user: systemd-network
system user: dbus
system user: polkitd
system user: tss
system user: postfix
system user: sshd
system user: dockerroot
system user: apache
system user: carbon
comman user：test1
comman user：test2
comman user：test3
comman user：test4
#####
end



## 字段数量比较
[root@suning ~]# awk -v FS=":" 'BEGIN{print "start\n------------"}  { if(NF>5){printf "comman user：%s\n",$1} else{printf "system user: %s\n",$1} } END{print "#####\nend"}' /etc/passwd
start
------------
comman user：root
comman user：bin
comman user：daemon
comman user：adm
comman user：lp
comman user：sync
comman user：shutdown
comman user：halt
comman user：mail
comman user：operator
comman user：games
comman user：ftp
comman user：nobody
comman user：systemd-bus-proxy
comman user：systemd-network
comman user：dbus
comman user：polkitd
comman user：tss
comman user：postfix
comman user：sshd
comman user：dockerroot
comman user：apache
comman user：carbon
comman user：test1
comman user：test2
comman user：test3
comman user：test4
#####
end


查找设备使用率超过20%的
[root@suning ~]# df -h|awk -v FS="%" '/^\/dev/{print $1}'
/dev/mapper/cl-root        20G  2.4G   17G   13
/dev/sda1                 197M  118M   79M   60
/dev/mapper/cl-usr_local   17G   33M   17G    1
/dev/mapper/cl-home        19G   33M   19G    1
[root@suning ~]# df -h|awk -v FS="%" '/^\/dev/{print $1}'|awk -v FS=" " '{ if($NF>20){print $1} }'
/dev/sda1

查找以不定长个数的空格开头的字符串，且含有指定字符的

awk ‘/^[[:space:]]*linux/{print $0}’ /etc/grub2.cfg

[root@suning ~]# awk '/^[[:space:]]*linux/{print $0}' /etc/grub2.cfg 
    linux16 /vmlinuz-3.10.0-514.el7.x86_64 root=/dev/mapper/cl-root ro crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet LANG=zh_CN.UTF-8
    linux16 /vmlinuz-0-rescue-855bf25106634b42b37f13519608dcb6 root=/dev/mapper/cl-root ro crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet

while循环

语法：while(condition) {statements}
使用场景：对一行内的多个字段进行逐一类似处理，对数组中的各元素进行处理

`` 对每行中的每个字段显示字段长度 oot@suning ~]# awk '/^[[:space:]]*linux/{ i=1; while(i<NF){print $i, lengen($i);i++} }' /etc/grub2.cfg awk: cmd. line:1: (FILENAME=/etc/grub2.cfg FNR=99) fatal: functionlengen’ not defined
[root@suning ~]# awk ‘/^[[:space:]]*linux/{ i=1; while(i

while中嵌套if

[root@suning ~]# awk '/^[[:space:]]*linux/{ i=1; while(i<NF){if(length($i)>=4){printf "%-55s %s\n", $i,length($i)};i++} }' /etc/grub2.cfg 
linux16                                                 7
/vmlinuz-3.10.0-514.el7.x86_64                          30
root=/dev/mapper/cl-root                                24
crashkernel=auto                                        16
rd.lvm.lv=cl/root                                       17
rd.lvm.lv=cl/swap                                       17
rhgb                                                    4
quiet                                                   5
linux16                                                 7
/vmlinuz-0-rescue-855bf25106634b42b37f13519608dcb6      50
root=/dev/mapper/cl-root                                24
crashkernel=auto                                        16
rd.lvm.lv=cl/root                                       17
rd.lvm.lv=cl/swap                                       17
rhgb

for循环

[root@suning ~]# awk '/^[[:space:]]*linux/{  for(i=1;i<NF;i++){ if(length($i)>=4){printf "%-55s %s\n", $i,length($i)} }  }' /etc/grub2.cfg 
linux16                                                 7
/vmlinuz-3.10.0-514.el7.x86_64                          30
root=/dev/mapper/cl-root                                24
crashkernel=auto                                        16
rd.lvm.lv=cl/root                                       17
rd.lvm.lv=cl/swap                                       17
rhgb                                                    4
quiet                                                   5
linux16                                                 7
/vmlinuz-0-rescue-855bf25106634b42b37f13519608dcb6      50
root=/dev/mapper/cl-root                                24
crashkernel=auto                                        16
rd.lvm.lv=cl/root                                       17
rd.lvm.lv=cl/swap                                       17
rhgb                                                    4

switch语句

语法：switch(expression) {case value1 or /regexp/: staement1;case value2 or /regexp/: staement2;…;default:statement}

next 提前结束对本行的处理，而进入下一行

[root@suning ~]# awk -v FS=":" '{ if($3%2!=0){next};{print $1,$3}  }' /etc/passwd
root 0
daemon 2
lp 4
shutdown 6
mail 8
games 12
ftp 14
systemd-network 192
polkitd 998
sshd 74
apache 48
carbon 996
test1 1000
test3 1002

array数组

[root@suning ~]# awk 'BEGIN{weekdays["mon"]="monday";weekdays["tue"]="tuesday";print weekdays["mon"]}'
monday
[root@suning ~]# awk 'BEGIN{weekdays["mon"]="monday";weekdays["tue"]="tuesday";print weekdays["tue"]}'
tuesday

数组中使用for循环查看所有数组元素

[root@suning ~]# awk 'BEGIN{weekdays["mon"]="monday";weekdays["tue"]="tuesday";for(i in weekdays){print i,weekdays[i]} }'
tue tuesday
mon monday

awk思维谋定 ^tcp>

[root@suning ~]# netstat -anp|awk '/^tcp/{print $0}'
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1108/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1635/master         
tcp        0      0 192.168.139.137:22      192.168.139.1:50559     ESTABLISHED 3339/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:64639     ESTABLISHED 2484/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:50517     ESTABLISHED 3241/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:64637     ESTABLISHED 2452/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:50620     ESTABLISHED 3445/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:50516     ESTABLISHED 3240/sshd: root@pts 
tcp6       0      0 :::22                   :::*                    LISTEN      1108/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1635/master         
[root@suning ~]# netstat -anp|awk '/^tcp\>/{print $0}'
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1108/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1635/master         
tcp        0      0 192.168.139.137:22      192.168.139.1:50559     ESTABLISHED 3339/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:64639     ESTABLISHED 2484/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:50517     ESTABLISHED 3241/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:64637     ESTABLISHED 2452/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:50620     ESTABLISHED 3445/sshd: root@pts 
tcp        0      0 192.168.139.137:22      192.168.139.1:50516     ESTABLISHED 3240/sshd: root@pts

统计下标的个数

[root@suning ~]# netstat -anp|awk '/^tcp\>/    {state[$6]++ }   END{   for(i in state){print i , state[i] }   }'
LISTEN 2
ESTABLISHED 6

统计每个单词出现的次数

# awk '{ for(i=1;i<=NF;i++){count[$i]++} }END{ for(i in count){print count[i],i} }' /etc/fstab 
1 man
1 and/or
1 maintained
3 xfs
1 Accessible
7 #
1 Thu
1 Apr
1 are
5 defaults
1 blkid(8)
1 /
1 /dev/mapper/cl-root
8 0
1 See
2 1
1 Created
1 on
1 mount(8)
1 ext3
1 anaconda
1 /dev/mapper/cl-home
1 fstab(5),
1 /boot
1 16:28:57
1 findfs(8),
1 /dev/mapper/cl-usr_local
1 2017
1 /dev/mapper/cl-swap
1 '/dev/disk'
2 by
1 /etc/fstab
1 pages
1 /usr/local
1 /data
1 more
1 info
1 27
2 swap
1 filesystems,
1 reference,
1 for
1 under
1 UUID=7cd893db-e433-4e37-b509-9b8f2cd11094

内置函数rand()

[root@suning ~]# awk 'BEGIN{print rand()}'
0.237788
[root@suning ~]# awk 'BEGIN{print rand()}'
0.237788
[root@suning ~]# awk 'BEGIN{print rand()}'
0.237788
[root@suning ~]# awk 'BEGIN{print rand()}'
0.237788
[root@suning ~]# awk 'BEGIN{print rand()}'
0.237788

内置函数sub 不常用

[root@suning ~]# awk -v FS=":" '{print sub(o,O,$1) }' test
1
1
[root@suning ~]# cat test
ooo:ttt
ttt:ooo

内置函数split及数组下标嵌套

[root@suning ~]# netstat -anp|awk '/^tcp\>/  { split($5,ip,":");count[ip[1]]++  }  END{for (i in count){print count[i],i} }'
2 0.0.0.0
4 192.168.139.1