0
点赞
收藏
分享

微信扫一扫

Kubernetes网络flannel之host-gw模式分析

夹胡碰 2022-01-20 阅读 65
网络kubernetes容器

1. Flannel的host-gw模式

Flannel的host-gw模式是一种纯三层的网络互通方案，Pod之间互相访问是通过路由方式实现。host-gw模式下跨节点网络通信需要通过节点上的路由表实现，因此必须要通信双方所在宿主机能够直接路由。这就要求该模式下集群中所有节点必须处于同一个网络内。公有云环境下需要安装对应的插件，如阿里云的CCM。

2. host-gw模式下的组网

在这里插入图片描述

flannel host-gw模式下涉及的主要网络设施有：

  • veth pair
  • cni0网桥
  • 物理网卡
  • 路由表

我们从Pod A出发，去探寻组网方式。首先，进入Pod A的网络namespace中查看Pod A的路由

[root@cn-beijing ~]# ip route
default via 10.10.2.1 dev eth0
10.10.0.0/16 via 10.10.2.1 dev eth0
10.10.2.0/24 dev eth0 proto kernel scope link src 10.10.2.238

Pod A的包都是通过eth0发送给10.10.2.1这个IP。到这里就出现两个问题：

  1. 10.10.2.1这个IP是谁？
  2. eth0是如何连接到10.10.2.1这个设备？

退出Pod A的网络namespace，通过指令查看主机上的设备的IP地址

[root@cn-beijing ~]# ip a
...
5: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 2e:5b:e9:3d:88:51 brd ff:ff:ff:ff:ff:ff
    inet 10.10.2.1/24 brd 10.10.2.255 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::2c5b:e9ff:fe3d:8851/64 scope link
       valid_lft forever preferred_lft forever
...

可以发现10.10.2.1为cni0网桥的地址。那么Pod A是如何连接到cni0网桥上的呢。我们首先查看cni0网桥上存在的接口

[root@cn-beijing ~]# bridge link
2270: vethef2d98bc state UP @docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cni0 state forwarding priority 32 cost 2

可以看到vethef2d98bc设备master到了cni0网桥上。

再次进入Pod A的网络namespace查看Pod A内的eth0网卡的类型

[root@cn-beijing ~]# ip -d link show eth0
3: eth0@if2270: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether 86:41:d0:e3:0c:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 promiscuity 0
    veth addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

发现eth0是一个veth设备的一端，下面通过命令查询eth0对端设备的编号

[root@cn-beijing ~]# ip address show dev eth0
3: eth0@if2270: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 86:41:d0:e3:0c:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.10.2.238/24 brd 10.10.2.255 scope global eth0
       valid_lft forever preferred_lft forever

通过指令结果可以看到eth0的对端veth设备编号为2270(@if2270)。

回到Node A的主网络namespace查找2270编号的网络设备

[root@cn-beijing ~]# ip a
...
2270: vethef2d98bc@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni0 state UP group default
    link/ether 5e:14:e7:9e:ca:fd brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::5c14:e7ff:fe9e:cafd/64 scope link
       valid_lft forever preferred_lft forever
...

可以看到编号为2270的网络设备为vethef2d98bc，并且是挂在在网桥cni0上的（master cni0）。到此，Pod A和10.10.2.1的连接问题已经解决。

host-gw模式下，每个node节点都是一个独立的网段，这个网段配置在cni0网桥上。如图Node A的Pod网段为10.10.2.1/24,cni0网桥的IP为10.10.2.1。
那下面的问题就是要探寻Pod A的网络请求发送到cni0后会发生什么。下面我们就以看一下一个ICMP报文是如何流转的。

3. 图解host-gw模式下包流转

在这里插入图片描述

从上一节的讨论可以看到，ICMP报文通过路由、veth设备发送到了cni0网桥上，网桥收到报文后应该如何处理呢？
为了能够抓到流转方式，我们打开iptables的TRACE功能。从Node A和Node B上执行如下命令

[root@cn-beijing ~]# iptables -t raw -A OUTPUT -p icmp -j TRACE
[root@cn-beijing ~]# iptables -t raw -A PREROUTING -p icmp -j TRACE

上述命令打开TRACE功能后，包的流转信息就会被记录在/var/log/messages文件里。
我们进入Pod A的网络命名空间执行ping（为了方便分析，我们只ping 1次）

[root@cn-beijing ~]# ping 10.10.3.4 -c 1
PING 10.10.3.4 (10.10.3.4) 56(84) bytes of data.
64 bytes from 10.10.3.4: icmp_seq=1 ttl=62 time=0.747 ms

--- 10.10.3.4 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.747/0.747/0.747/0.000 ms

3.1. ICMP REQ流转-发送

可以看到Node A的messages文件里记录了包的流转过程,ICMP的请求流转如下

Jan 19 17:19:47 cn-beijing kernel: TRACE: raw:PREROUTING:policy:2 IN=cni0 OUT= PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: mangle:PREROUTING:policy:1 IN=cni0 OUT= PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:PREROUTING:rule:1 IN=cni0 OUT= PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:KUBE-SERVICES:return:11 IN=cni0 OUT= PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:PREROUTING:policy:3 IN=cni0 OUT= PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: mangle:FORWARD:policy:1 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:1 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:KUBE-FORWARD:return:4 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:2 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:KUBE-SERVICES:return:1 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:3 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:KUBE-EXTERNAL-SERVICES:return:1 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:4 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:DOCKER-USER:return:1 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:5 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:DOCKER-ISOLATION-STAGE-1:return:2 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:10 IN=cni0 OUT=eth0 PHYSIN=vethef2d98bc MAC=2e:5b:e9:3d:88:51:86:41:d0:e3:0c:65:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: mangle:POSTROUTING:policy:1 IN= OUT=eth0 PHYSIN=vethef2d98bc SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:POSTROUTING:rule:1 IN= OUT=eth0 PHYSIN=vethef2d98bc SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:KUBE-POSTROUTING:rule:1 IN= OUT=eth0 PHYSIN=vethef2d98bc SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:POSTROUTING:rule:4 IN= OUT=eth0 PHYSIN=vethef2d98bc SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:POSTROUTING:policy:8 IN= OUT=eth0 PHYSIN=vethef2d98bc SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0

从上述看到从cni0进入的ICMP包经过PREROUTING链之后进入了FORWARD链，报文被转发给了Node A的eth0网卡，从IPTABLES规则看PREROUTING和FORWARD之间经历了路由过程，我们看一下主机路由表

[root@cn-beijing ~]# ip route
default via 192.168.0.13 dev eth0
10.10.0.0/24 via 192.168.0.1 dev eth0
10.10.1.0/24 via 192.168.0.2 dev eth0
10.10.2.0/24 dev cni0 proto kernel scope link src 10.10.2.1
10.10.3.0/24 via 192.168.0.4 dev eth0
10.10.4.0/24 via 192.168.0.5 dev eth0
10.10.5.0/24 via 192.168.0.6 dev eth0
169.254.0.0/16 dev eth0 scope link metric 1002
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
192.168.0.0/28 dev eth0 proto kernel scope link src 192.168.0.3

可以看到路由表上记录

10.10.3.0/24 via 192.168.0.4 dev eth0

发往10.10.3.4的包通过eth0发出，下一跳为192.168.0.4，也就是Node B的IP地址。

3.2. ICMP REQ-接收

可以看到Node B的messages文件里记录了包的流转过程,ICMP的请求流转如下

Jan 19 17:19:47 cn-beijing kernel: TRACE: raw:PREROUTING:policy:2 IN=eth0 OUT= MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: mangle:PREROUTING:policy:1 IN=eth0 OUT= MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:PREROUTING:rule:1 IN=eth0 OUT= MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:KUBE-SERVICES:return:11 IN=eth0 OUT= MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:PREROUTING:policy:3 IN=eth0 OUT= MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=63 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: mangle:FORWARD:policy:1 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:1 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:KUBE-FORWARD:return:4 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:2 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:KUBE-SERVICES:return:1 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:3 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:KUBE-EXTERNAL-SERVICES:return:1 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:4 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:DOCKER-USER:return:1 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:5 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:DOCKER-ISOLATION-STAGE-1:return:2 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: filter:FORWARD:rule:10 IN=eth0 OUT=cni0 MAC=00:16:3e:2e:80:3b:ee:ff:ff:ff:ff:ff:08:00 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: mangle:POSTROUTING:policy:1 IN= OUT=cni0 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:POSTROUTING:rule:1 IN= OUT=cni0 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:KUBE-POSTROUTING:rule:1 IN= OUT=cni0 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:POSTROUTING:rule:4 IN= OUT=cni0 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0 
Jan 19 17:19:47 cn-beijing kernel: TRACE: nat:POSTROUTING:policy:8 IN= OUT=cni0 SRC=10.10.2.238 DST=10.10.3.4 LEN=84 TOS=0x00 PREC=0x00 TTL=62 ID=63186 DF PROTO=ICMP TYPE=8 CODE=0 ID=8704 SEQ=0

从上述看到从cni0进入的ICMP包经过PREROUTING链之后进入了FORWARD链，报文被转发给了cni0网桥，从IPTABLES规则看PREROUTING和FORWARD之间经历了路由过程，我们看一下主机路由表

[root@cn-beijing ~]# ip route
default via 192.168.0.13 dev eth0
10.10.0.0/24 via 192.168.0.1 dev eth0
10.10.1.0/24 via 192.168.0.2 dev eth0
10.10.2.0/24 via 192.168.0.3 dev eth0
10.10.3.0/24 dev cni0 proto kernel scope link src 10.10.3.1
10.10.4.0/24 via 192.168.0.5 dev eth0
10.10.5.0/24 via 192.168.0.6 dev eth0
169.254.0.0/16 dev eth0 scope link metric 1002
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
192.168.0.0/28 dev eth0 proto kernel scope link src 192.168.0.4

可以看到路由表上记录

10.10.3.0/24 dev cni0 proto kernel scope link src 10.10.3.1

发往10.10.3.4的包通过发送给了网桥cni0。

4. 总结

从上面分析可以到host-gw模式下纯粹使用了路由实现了网络的互通，并没有使用到vxlan等虚拟网络技术。

举报

相关推荐

Flannel HOST-GW 跨节点通信

flannelhost-gwkubernetes云计算

Kubernetes网络模型 -flannel +Calico

k8s网络模型flannelCalico云平台云计算私藏项目实操分享

flannel-kubernetes的网络模型

kubernetesk8s那点事儿

Docker 初探Docker的网络模式之host模式

dockercentosf5编程语言

kubernetes使用flannel网络插件服务状态显示CrashLoopBackOff

edn初始化nginx编程语言

kubernetes flannel 查看状态

Pod3d命名空间kubernetes云计算

容器网络之host和none

docker 网络bashdocker命名空间虚拟化云计算

Kubernetes_容器网络_Flannel_02_Flannel跨主机通信Overlay Network三种实现

网络linux运维IPUDPHtml/CSS前端开发

K8S 之 Flannel 网络插件

Flannel运维yyds干货盘点

k8s网络之flannel(vxlan)

网络dockerlinux
0 条评论