iptables的流量跟踪debug_iptables debug-优快云博客

示例操作系统：centos7

一、计数器跟踪

观察iptables -nvL INPUT返回的第一列pkts是否增长了，可以简单的判断规则被匹配

# 添加一个规则允许ping通本机，如果有ping主机发生，则这条规则被匹配，这条规则计数器为增加
iptables -A INPUT -p icmp -j ACCEPT
#查看
iptables -nvL INPUT
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     icmp -- *      *       0.0.0.0/0            0.0.0.0/0
#他机ping一下服务器，再观察pkts是否增长
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
    1    84 ACCEPT     icmp -- *      *       0.0.0.0/0            0.0.0.0/0
#清空计数器可以更加方便查看
iptables -Z INPUT
iptables -nvL INPUT
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     icmp -- *      *       0.0.0.0/0            0.0.0.0/0

二、使用LOG模块在任意位置打印报文信息

#在/etc/rsyslog.conf添加
kern.=debug /var/log/kern.debug.log
可能需要手动创建文件/var/log/kern.debug.log

重启rsyslog:

touch /var/log/kern.debug.log
systemctl restart rsyslog

之后可以添加如下的iptables规则，当报文到达该规则时，会在/var/log/kern.debug.log中打印出日志：

iptables -t raw -A OUTPUT -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "raw out: "

打印出来的日志如下：

Jun 13 19:58:21 dev-slave-110 kernel: raw prerouting: IN=eth0 OUT= MAC=52:54:15:5d:39:58:02:54:d4:90:3a:57:08:00 SRC=8.8.8.8 DST=10.39.0.110 LEN=84 TOS=0x00 PREC=0x00 TTL=32 ID=0 PROTO=ICMP TYPE=0 CODE=0 ID=13629 SEQ=15

和其它iptables一样，也可以添加IP、端口等条件：

iptables -t raw -A PREROUTING -p icmp -s 8.8.8.8/32 -m limit --limit 500/minute -j LOG --log-level 7 --log-prefix "mangle prerouting: "

全部加上

iptables -t raw -A OUTPUT -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "raw out: "
iptables -t mangle -A OUTPUT -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "mangle out: "
iptables -t nat -A OUTPUT -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "nat out: "
iptables -t filter -A OUTPUT -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "filter out: "
iptables -t mangle -A POSTROUTING -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "mangle post: "
iptables -t nat -A POSTROUTING -m limit --limit 5000/minute -j LOG --log-level 7 --log-prefix "nat post: "

注意这里设置了–limit防止打印出太多的日志，–limit如果设置的太小，可能会恰好丢弃要观察的包的日志

执行命令“ping -c 1 3.3.3.3”，然后查看iptables的日志记录：
cat /var/log/kern.debug.log |grep 3.3.3.3
Mar 31 05:48:29 compile kernel: raw out: IN= OUT=eth0 SRC=10.0.2.15 DST=3.3.3.3 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=29101 DF PROTO=ICMP TYPE=8 CODE=0 ID=4232 SEQ=1
Mar 31 05:48:29 compile kernel: mangle out: IN= OUT=eth0 SRC=10.0.2.15 DST=3.3.3.3 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=29101 DF PROTO=ICMP TYPE=8 CODE=0 ID=4232 SEQ=1
Mar 31 05:48:29 compile kernel: nat out: IN= OUT=eth0 SRC=10.0.2.15 DST=3.3.3.3 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=29101 DF PROTO=ICMP TYPE=8 CODE=0 ID=4232 SEQ=1
Mar 31 05:48:29 compile kernel: filter out: IN= OUT=eth0 SRC=10.0.2.15 DST=3.3.3.3 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=29101 DF PROTO=ICMP TYPE=8 CODE=0 ID=4232 SEQ=1
Mar 31 05:48:29 compile kernel: mangle post: IN= OUT=eth0 SRC=10.0.2.15 DST=3.3.3.3 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=29101 DF PROTO=ICMP TYPE=8 CODE=0 ID=4232 SEQ=1
Mar 31 05:48:29 compile kernel: nat post: IN= OUT=eth0 SRC=10.0.2.15 DST=3.3.3.3 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=29101 DF PROTO=ICMP TYPE=8 CODE=0 ID=4232 SEQ=1

从日志中可以看到，报文依次经过了raw out、mangle out、nat out、filter out、mangle out、nat post。

设置一条规则，在filter阶段丢弃到1.1.1.1的报文:

iptables -t filter -A OUTPUT -d 1.1.1.1 -j DROP
执行命令ping -c 1.1.1.1后，查看日志

$cat /var/log/kern.debug.log |grep 1.1.1.1
Mar 31 06:10:02 compile kernel: raw out: IN= OUT=eth0 SRC=10.0.2.15 DST=1.1.1.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=24260 DF PROTO=ICMP TYPE=8 CODE=0 ID=4426 SEQ=1
Mar 31 06:10:02 compile kernel: mangle out: IN= OUT=eth0 SRC=10.0.2.15 DST=1.1.1.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=24260 DF PROTO=ICMP TYPE=8 CODE=0 ID=4426 SEQ=1
Mar 31 06:10:02 compile kernel: nat out: IN= OUT=eth0 SRC=10.0.2.15 DST=1.1.1.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=24260 DF PROTO=ICMP TYPE=8 CODE=0 ID=4426 SEQ=1
Mar 31 06:10:02 compile kernel: filter out: IN= OUT=eth0 SRC=10.0.2.15 DST=1.1.1.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=24260 DF PROTO=ICMP TYPE=8 CODE=0 ID=4426 SEQ=1

从日志中可以看到，发送到1.1.1.1的报文到达filter表的OUTPUT规则后就终止了。

设置一条规则nat规则:

iptables -t nat -A OUTPUT -d 2.2.2.2 -j DNAT --to-destination 8.8.8.8
｀ping -c 2.2.2.2`之后，可以看到下面的日志:

$cat /var/log/kern.debug.log |grep 2.2.2.2
Mar 31 06:12:38 compile kernel: raw out: IN= OUT=eth0 SRC=10.0.2.15 DST=2.2.2.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=2791 DF PROTO=ICMP TYPE=8 CODE=0 ID=4431 SEQ=1
Mar 31 06:12:38 compile kernel: mangle out: IN= OUT=eth0 SRC=10.0.2.15 DST=2.2.2.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=2791 DF PROTO=ICMP TYPE=8 CODE=0 ID=4431 SEQ=1
Mar 31 06:12:38 compile kernel: nat out: IN= OUT=eth0 SRC=10.0.2.15 DST=2.2.2.2 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=2791 DF PROTO=ICMP TYPE=8 CODE=0 ID=4431 SEQ=1

$ cat /var/log/kern.debug.log |grep 8.8.8.8
Mar 31 06:12:38 compile kernel: filter out: IN= OUT=eth0 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=2791 DF PROTO=ICMP TYPE=8 CODE=0 ID=4431 SEQ=1
Mar 31 06:12:38 compile kernel: mangle post: IN= OUT=eth0 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=2791 DF PROTO=ICMP TYPE=8 CODE=0 ID=4431 SEQ=1
Mar 31 06:12:38 compile kernel: nat post: IN= OUT=eth0 SRC=10.0.2.15 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=2791 DF PROTO=ICMP TYPE=8 CODE=0 ID=4431 SEQ=1
从日志中可以看到，报文经过nat表的OUTPUT规则时进行了DNAT，然后使用新的DST地址经过filter的OUTPUT规则直到发送出。

三、使用TRACE模块对报文进行全程跟踪

TRACE只能在raw表中使用

通过-j TRACE

iptables -t raw -A PREROUTING -p icmp -s 8.8.8.8/32 -j TRACE

在man iptables-extensions中可以找到对TRACE的介绍：

This target marks packets so that the kernel will log every rule which match
the packets as those traverse the tables, chains, rules.

TRACE模块会在符合规则的报文上打上标记，将该报文经过的每一条规则打印出来，很方便的对报文做全程跟踪。TRACE模块只能在raw表中使用，还需要加载内核模块：

modprobe ipt_LOG ip6t_LOG nfnetlink_log
之后可以通过dmesg，或者在/var/log/message中查看到匹配的报文的日志：

Jun 16 17:44:05 dev-slave-110 kernel: TRACE: raw:PREROUTING:rule:2 IN=eth0 OUT= MAC=52:54:15:5d:39:58:02:54:d4:90:3a:57:08:00 SRC=8.8.8.8 DST=10.39.0.110 LEN=84 TOS=0x00 PREC=0x00 TTL=32 ID=0 PROTO=ICMP TYPE=0 CODE=0 ID=4064 SEQ=24
Jun 16 17:44:05 dev-slave-110 kernel: TRACE: raw:cali-PREROUTING:rule:1 IN=eth0 OUT= MAC=52:54:15:5d:39:58:02:54:d4:90:3a:57:08:00 SRC=8.8.8.8 DST=10.39.0.110 LEN=84 TOS=0x00 PREC=0x00 TTL=32 ID=0 PROTO=ICMP TYPE=0 CODE=0 ID=4064 SEQ=24
Jun 16 17:44:05 dev-slave-110 kernel: TRACE: raw:cali-PREROUTING:rule:3 IN=eth0 OUT= MAC=52:54:15:5d:39:58:02:54:d4:90:3a:57:08:00 SRC=8.8.8.8 DST=10.39.0.110 LEN=84 TOS=0x00 PREC=0x00 TTL=32 ID=0 PROTO=ICMP TYPE=0 CODE=0 ID=4064 SEQ=24
Jun 16 17:44:05 dev-slave-110 kernel: TRACE: raw:cali-from-host-endpoint:return:1 IN=eth0 OUT= MAC=52:54:15:5d:39:58:02:54:d4:90:3a:57:08:00 SRC=8.8.8.8 DST=10.39.0.110 LEN=84 TOS=0x00 PREC=0x00 TTL=32 ID=0 PROTO=ICMP TYPE=0 CODE=0 ID=4064 SEQ=24
Jun 16 17:44:05 dev-slave-110 kernel: TRACE: raw:cali-PREROUTING:return:5 IN=eth0 OUT= MAC=52:54:15:5d:39:58:02:54:d4:90:3a:57:08:00 SRC=8.8.8.8 DST=10.39.0.110 LEN=84 TOS=0x00 PREC=0x00 TTL=32 ID=0 PROTO=ICMP TYPE=0 CODE=0 ID=4064 SEQ=24

添加与删除

iptables -t raw -I <OUTPUT|PREROUTING> <filters> -j TRACE
iptables -t raw -D <OUTPUT|PREROUTING> <filters> -j TRACE

# iptables -t raw -A OUTPUT -p icmp -j TRACE
# iptables -t raw -A PREROUTING -p icmp -j TRACE

Trace incoming packets（跟踪入向包）

# iptables -t raw -A PREROUTING -p tcp --destination 1.2.3.4 --dport 443 -j TRACE
# sudo iptables -t raw -L -v -n --line-numbers
Chain PREROUTING (policy ACCEPT 3501 packets, 946K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1      468 28159 TRACE      tcp -- *      *       0.0.0.0/0            1.2.3.4       tcp dpt:443
Chain OUTPUT (policy ACCEPT 885 packets, 695K bytes)
num   pkts bytes target     prot opt in     out     source               destination

Trace to the internal network will look like this.
[...]
Jul 18 18:33:27 cerberus kernel: [68907.892027] TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=00:15:17:c3:a1:aa:00:15:17:c3:fb:07:01:00 SRC=172.69.63.16 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x00 TTL=56 ID=64783 DF PROTO=TCP SPT=62598 DPT=443 SEQ=234589096 ACK=404477568 WINDOW=82 RES=0x00 ACK URGP=0
Jul 18 18:33:27 cerberus kernel: [68907.892093] TRACE: mangle:INPUT:policy:1 IN=eth0 OUT= MAC=00:15:17:c3:a1:aa:00:15:17:c3:fb:07:01:00 SRC=172.69.63.16 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x00 TTL=56 ID=64783 DF PROTO=TCP SPT=62598 DPT=443 SEQ=234589096 ACK=404477568 WINDOW=82 RES=0x00 ACK URGP=0
Jul 18 18:33:27 cerberus kernel: [68907.892113] TRACE: filter:INPUT:rule:6 IN=eth0 OUT= MAC=00:15:17:c3:a1:aa:00:15:17:c3:fb:07:01:00 SRC=172.69.63.16 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x00 TTL=56 ID=64783 DF PROTO=TCP SPT=62598 DPT=443 SEQ=234589096 ACK=404477568 WINDOW=82 RES=0x00 ACK URGP=0
Jul 18 18:33:27 cerberus kernel: [68907.892150] TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=00:15:17:c3:a1:aa:00:15:17:c3:fb:07:01:00 SRC=172.69.63.16 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x00 TTL=56 ID=64784 DF PROTO=TCP SPT=62598 DPT=443 SEQ=234589096 ACK=404477569 WINDOW=82 RES=0x00 ACK RST URGP=0
[...]

Trace outgoing packets（跟踪出向包）

Use raw table and OUTPUT to trace locally generated packets.

$ sudo iptables -t raw -A OUTPUT -p tcp --destination 8.8.8.8 --dport 53 -j TRACE
$ sudo iptables -t raw -A OUTPUT -p udp --destination 8.8.8.8 --dport 53 -j TRACE

Inspect raw table.
$ sudo iptables -t raw -L -v -n --line-numbers
Chain PREROUTING (policy ACCEPT 1281 packets, 422K bytes)
num   pkts bytes target     prot opt in     out     source               destination
Chain OUTPUT (policy ACCEPT 379 packets, 324K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 TRACE      tcp -- *      *       0.0.0.0/0            8.8.8.8              tcp dpt:53
2        0     0 TRACE      udp -- *      *       0.0.0.0/0            8.8.8.8              udp dpt:53

Trace to the external DNS server will look like this.
[...]
Jul 18 18:48:38 cerberus kernel: [69819.286907] TRACE: raw:OUTPUT:policy:3 IN= OUT=eth0 SRC=1.2.3.4 DST=8.8.8.8 LEN=78 TOS=0x00 PREC=0x00 TTL=64 ID=27373 PROTO=UDP SPT=45407 DPT=53 LEN=58 UID=2018 GID=2018
Jul 18 18:48:38 cerberus kernel: [69819.286922] TRACE: nat:OUTPUT:policy:1 IN= OUT=eth0 SRC=1.2.3.4 DST=8.8.8.8 LEN=78 TOS=0x00 PREC=0x00 TTL=64 ID=27373 PROTO=UDP SPT=45407 DPT=53 LEN=58 UID=2018 GID=2018
Jul 18 18:48:38 cerberus kernel: [69819.286929] TRACE: filter:OUTPUT:rule:7 IN= OUT=eth0 SRC=1.2.3.4 DST=8.8.8.8 LEN=78 TOS=0x00 PREC=0x00 TTL=64 ID=27373 PROTO=UDP SPT=45407 DPT=53 LEN=58 UID=2018 GID=2018
Jul 18 18:48:38 cerberus kernel: [69819.286939] TRACE: nat:POSTROUTING:policy:2 IN= OUT=eth0 SRC=1.2.3.4 DST=8.8.8.8 LEN=78 TOS=0x00 PREC=0x00 TTL=64 ID=27373 PROTO=UDP SPT=45407 DPT=53 LEN=58 UID=2018 GID=2018
[...]

iptables自身提供了TRACE功能，一旦设定，当数据包匹配到任意chain上任意table的处理规则时，iptables会在系统日志(/var/log/syslog)中自动输出此时的数据包状态日志。

ebtables的调试设置

我们的重点在iptables，为ebtables只是辅助，帮助我们看清数据包到底是在哪一层被hook进iptables的规则链中进行处理的。因此我们在全图中的每个ebtables的built-in chain上都加上LOG（ebtables目前还不支持TRACE）：

# ebtables -t broute -A BROUTING -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:broute:BROUTING" -j ACCEPT
# ebtables -t nat -A OUTPUT -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:nat:OUTPUT" -j ACCEPT
# ebtables -t nat -A PREROUTING -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:nat:PREROUTING" -j ACCEPT
# ebtables -t filter -A INPUT -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:filter:INPUT" -j ACCEPT
# ebtables -t filter -A FORWARD -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:filter:FORWARD" -j ACCEPT
# ebtables -t filter -A OUTPUT -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:filter:OUTPUT" -j ACCEPT
# ebtables -t nat -A POSTROUTING -p ipv4 --ip-proto 1 --log-level 6 --log-ip --log-prefix "TRACE: eb:nat:POSTROUTING" -j ACCEPT

注意：这里--ip-proto 1 表示仅match icmp packet。

日志解析：

最后一条iptables日志：

Jul 15 22:59:14 centos791 kernel: TRACE: filter:INPUT:rule:7 IN=ens33 OUT= MAC=01:00:5e:00:01:01:00:0c:29:af:56:95:08:00 SRC=192.168.36.5 DST=239.0.1.1 LEN=59 TOS=0x00 PREC=0x00 TTL=1 ID=56429 DF PROTO=UDP SPT=35296 DPT=2223 LEN=39 UID=1000 GID=1000

可以看到数据包被规则 filter:INPUT:rule:7 处理，查看此条规则：

# iptables -t filter -L INPUT -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target     prot opt in     out     source               destination
38058 3475K ACCEPT     all -- *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
3329 200K ACCEPT     all -- lo     *       0.0.0.0/0            0.0.0.0/0
   16 1835 INPUT_direct all -- *      *       0.0.0.0/0            0.0.0.0/0
   16 1835 INPUT_ZONES_SOURCE all -- *      *       0.0.0.0/0            0.0.0.0/0
   16 1835 INPUT_ZONES all -- *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all -- *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
   14 1563 REJECT     all -- *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
可以看到这个数据包被REJECT。