0
点赞
收藏
分享

微信扫一扫

wireguard-跨云or vpc网络通讯方案

# 背景:

早期服务器集中于腾讯云,开始是传统网络。后面是自定义的私有网络vpc.当然了vpc中还有容器网络,容器的网络方案使用了默认的Global Router,并没有使用VPC-CNI的容器网络与云主机网络在同一个 VPC 内的方案(腾讯云官方文档还有了Cilium-Overlay 的方案,恩还有个测试环境的k8s集群是kubeadm自建的集群网络插件用的cilum).今年45月份有些新业务又跑在了阿里云上面,集群环境直接用了阿里云的ACK PRO。网络插件使用了Flannel ,也没有使用阿里云的Terway。现在的需求就是要把两个网络打通! # 看了一下网上的各种方案: [1. 腾讯云与阿里云如何建立VPN连接](https://cloud.tencent.com/edu/learning/course-2167-29514)(不过貌似阿里云侧vpn镜像找不到了,故未能跑通) 2.专线?阿里云腾讯云都有专线服务[https://cloud.tencent.com/developer/article/1731806?from=15425](https://cloud.tencent.com/developer/article/1731806?from=15425)。这样的也可以尝试一下? 3.WireGuard[米开朗基杨](https://icloudnative.io/)大佬写过一系列的文章;如:[基于 WireGuard 和 OpenVPN 的混合云基础架构建设](https://www.modb.pro/db/144660) 我也选择了WireGuard的方案..... 以下测试都基于新环境搭建,没有敢在现有的环境去搞...... # 终极实现目标 **三个vpc ** 就先以ABC命名吧 !**A B 网络双向互通**,**c网络可以连接b**,**A网络可以连接C**?初步规划每个集群起码有2台服务器以上,可以ping通ssh目标网络服务器?恩 还可以在每个网络k8s集群中搞一个nginx service,用curl掉一下测试应用的访问?初步就算那么想的! 注:所谓互通都包括容器网络的通信! ## VPC子网规划 还是依赖与这个神奇的工具:[http://www.ab126.com/web/3552.html](http://www.ab126.com/web/3552.html),还是拿10.0.0.0/8去分了。**A网络**腾讯云上海vpc网络设置为10.10.0.0/16, **B网络**阿里云网络**10.20.0.0/16**,**C网络**腾讯云北京VPC**10.40.0.0/21** ### 关于A网络: ![ETWLAhv8RU.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7841db1490769.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78434c2e71059.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 注意:创建子网subnet的时候手残了,没有注意本来wireguard-shanghai3对应上海三区,wireguard-shanghai4对应上海四区的,没有留意,就做成这样的了...... ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842b5b572620.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### 关于B网络 阿里云B网络创建VPC也创建了4个可用区。但是他的定义都是叫什么交换机!,如下: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842020424576.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78432e3471964.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 注意:当然了B网络也选择了上海区域,与腾讯云A网络处于同一地区! ### 关于C网络 觉得A,B网络浪费资源了....就修改了一下子网掩码: 最终网段规划为**10.40.0.0/21** ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845d4db20100.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) subset子网如下: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842e24b18422.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78434e053603.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78431f1994435.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784281576606.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## 关于容器的网络 ### A网络容器网络 A网络容器网络CIDR172.16.0.0/16,Service CIDR172.16.252.0/22(其实就是172.16.0.0/16) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843f20965395.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### B网络容器网络-以及阿里云ack集群的创建 恩 觉得容器网络 /16整段也有点浪费,修改一下就取172.17.0.0/20吧,本来就是演示的这些资源也都够用了,实际生产环境中,如果需要考虑多地域,or多集群,还考虑互通,应该合理划分子网资源! ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844504d62456.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![brGgLgCHnv.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843871855649.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 由于阿里云容器服务需要填写Pod CIDR Service CIDR开始本来想直接分两个网段的但是操作失误删除了集群重建冲突了(开始用的阿里云自己的那操作系统,安装wireguard不顺畅,删除更换系统为centos7了!),就把**172.17.0.0/20**划分了四个子网取**Pod CIDR172.17.8.0/22 Service CIDR172.17.12.0/22** ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78437c9427361.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843f87386195.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### c网络容器网络-以及tke集群的简单创建 以网络c为例:容器网络依然从172.17.0.0/16子网20去分吧!选择了172.17.16.0/20段 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843e4653531.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 创建tke集群可以参考下图: 打开tke控制台,选择区域北京,新建: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844fd8a92461.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 输入集群名称(个人定义了),关键是集群网络选择前面创建的C网络(北京有好几个vpc网络呢),配置容器网络172.17.16.0/20 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844c97b2315.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 比较喜欢TKE容器网络的设置好带能搞在一个段里......阿里云的还要算一下。比较麻烦了,系统选择了ubuntu20.04,也不去用腾讯云自己的TencentOS Server了,怕跟阿里云的系统一样安装wireguard有问题(其实可以参考非标准内核的安装方式,懒得看了)...... 下一步,特意添加了两台cvm且不在同一个subset: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843738823973.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843a7ed11236.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843eb6a31589.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 默认下一步吧选择自己的登陆方式,我这里选择了自己的ssh-key方式! ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784332ae50157.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 可选组件就直接默认了,毕竟我这里主要是演示wireguard组网!下一步: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7846334f65085.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 同意,完成等待创建ing...... ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843e02a64127.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845e7a624414.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) c集群 两台server 10.40.2.6将安装**wireguard,**10.40.3.14后续作测试。 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78430e7434294.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) # wireguard的安装 ## 理一遍关系 注:阿里云ack操作系统智能选择centos7与alios 阿里云系统内核不是标准内核故操作系统我使用的centos7,腾讯云tke集群我用的是ubuntu 20.4系统! 每个个集群都初始保证两台server以上 ### A集群资源: vpc网络资源10.10.0.0/16 容器网络资源:172.16.0.0/16 | 主机名 | ip | subset | 是否安装wireguard | | --- | --- | --- | --- | | VM-4-8-ubuntu | 10.10.4.8 | 上海二区 | 是 | | VM-4-17-ubuntu | 10.10.4.17 | 上海二区 | 否 | | VM-2-4-ubuntu | 10.10.2.4 | 上海一区 | 否 | ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784330d161995.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### B集群资源: **vpc网络资源10.20.0.0/16** ** 容器网络资源:Pod CIDR172.17.8.0/22 Service CIDR172.17.12.0/22(17.17.0.0/20)** | 主机名 | ip | 交换机(subset) | 是否安装wireguard | | --- | --- | --- | --- | | iZuf6fxoj4zcqlpe8jupv2Z | **10.20.4.42** | 上海 可用区L(wireguard-shanghai4) | 是 | | Zuf6fxoj4zcqlpe8jupv3Z | **10.20.4.43** | 上海 可用区L(wireguard-shanghai4) | 否 | ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78432c5c31798.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844d22263437.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### C集群资源: **vpc网络资源10.40.0.0/21** ** 容器网络资源:Pod CIDR172.17.16.0/20 Service CIDR172.17.28.0/22** | 主机名 | ip | subset | 是否安装wireguard | | --- | --- | --- | --- | | VM-2-16-ubuntu | **10.40.2.16 ** | 北京二区 | 是 | | ** **VM-3-16-ubuntu | **10.40.3.16 ** | 北京三区 | 否 | ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78441e6051756.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## 前置条件: **防火墙都是关闭的,代理服务器(安装wireguard节点)开启了ip转发** **保证net.ipv4.ip_forward = 1** ``` sysctl -p|grep ip_forward net.ipv4.ip_forward = 1 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842590e94907.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## A C网络各一节点安装wireguard 初步选择在10.10.4.8这台cvm安装**wireguard** **注:**由于A C都是tke,且系统相同,我的安装演示都是在**10.40.2.16**节点上!主要是开始操作的时候没有截图...... ``` root@VM-2-16-ubuntu:~# uname -a Linux VM-2-6-ubuntu 5.4.0-121-generic #137-Ubuntu SMP Wed Jun 15 13:33:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux ``` 内核5.4 ``` sudo apt upgrade sudo apt install wireguard-dkms wireguard-tools -y sudo mkdir /etc/wireguard/keys cd /etc/wireguard/keys sudo wg genkey > vpn-gw.key sudo wg pubkey < vpn-gw.key > vpn-gw.pub ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844a40919936.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78463b6c73096.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843255997833.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 检查下 wireguard 模块是否已经加载,如果没有加载,试下执行 modprobe wireguard加载模块 ``` root@VM-2-16-ubuntu:/etc/wireguard/keys# lsmod |grep wire root@VM-2-16-ubuntu:/etc/wireguard/keys# modprobe wireguard && lsmod | grep wireguard wireguard 212992 0 ip6_udp_tunnel 16384 1 wireguard udp_tunnel 16384 1 wireguard ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843a68115440.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 注意:一般推荐是升级一下内核到linux-kernel 5.6以上,5.6kernel已经内置wireguard的mod,but鉴于集群的稳定性,我没有升级内核 ## B网络一节点安装wireguard B网络服务器系统为centos7故yum操作安装wireguard: ``` yum install epel-release https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm yum install yum-plugin-elrepo yum install kmod-wireguard wireguard-tools #检查下 wireguard 模块是否已经加载,如果没有加载,试下执行 modprobe wireguard加载模块 modprobe wireguard modprobe wireguard && lsmod | grep wireguard ``` 注:关于内核是否升级看个人意愿。或者找一个额外机器升级....为了维护集群稳定性,未能升级内核 # A B网络互通 基本就是实现下图这样B网络中容器and vpc网络通过cvm 10.20.4.42的公网ip 到A网络10.10.4.8的公网Ip 路由访问A网络中资源,反之A网络容器and vpc网络资源通过10.10.4.8的公网ip到B网络的10.20.4.42的公网ip路由访问B网络中资源(自己的理解,可能语言表达有误) ![](https://s2.51cto.com/images/blog/202208/24122228_6305a78443f6c54849.jpg?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 注意:10.30.0.1/24 10.10.30.2/24为wareguard网络,避免与各个网络地址冲突! ## 名词解释: - Interface Address:这个是开启wg之后本机的隧道IP ListenPort:这个就设置51820了忘了看了哪个文章的了 PrivateKey:本机的私钥/etc/wireguard/keys/vpn-gw.key - Peer Public Key: 服务端(就是要连接的网络的有公网ip,安装wireguard的节点)的公钥即需要连接的对方10.20.4.42的/etc/wireguard/keys/vpn-gw.pub Allow IPs: 需要走隧道的IP段, 一般包含隧道自身IP段, 以及要借道隧道的IP段, 但是这里设置并不会产生对应的route Endpoint: 服务端的IP和端口 Persis.Keepalive: 25 保持活动的时间间隔 ## A网络10.10.4.8节点配置 10.10.4.8节点wg0.conf配置 cat /etc/wireguard/ wg0.conf ``` [Interface] Address = 10.30.0.1/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxxx AllowedIPs = 10.30.0.0/24,10.20.0.0/16,172.17.0.0/20 [Peer] # Client1: aliyun-shanghai PublicKey = xxxxxxxxxxxxxxxxxxxxxxxx Endpoint = xxxxxxxx:51820 AllowedIPs = 10.30.0.2/24,10.20.0.0/16,172.17.0.0/20 PersistentKeepalive = 25 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784271fc18217.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## B网络10.20.4.42节点配置 10.20.4.42节点wg0.conf配置 cat /etc/wireguard/ wg0.conf ``` [Interface] Address = 10.30.0.2/24 ListenPort = 51820 PrivateKey = xxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE [Peer] # Client1: MacOS-Desktop PublicKey = xxxxxxx Endpoint = xxxxx:51820 AllowedIPs = 10.30.0.2/24,10.10.0.0/16,172.16.0.0/16 PersistentKeepalive = 25 [Peer] PublicKey = xxxxxxxxx AllowedIPs = 10.30.0.1/16,10.10.0.0/16,172.16.0.0/16 ``` ## 启动服务: 注: 10.20.4.42 10.10.4.8都启动服务,关闭服务down只是演示,保持服务up启动! ``` cd /etc/wireguard wg-quick up wg0 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78440ed569854.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 同理关闭服务就是: ``` wg-quick down wg0 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842b27c99335.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## 测试安装wireguard节点之间的互相通信: 10.10.4.8 ping 10.20.4.42 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842347d72650.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 10.20.4.42 ping 10.10.4.8 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844147867505.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## vpc内自定义路由 如何ping 通vpc内其他cvm等网络资源呢? 登陆腾讯云后台私有网络控制台找到对应vpc路由表,添加到B网络的路由(关于172.17.0.0/16网段并没有与B完全匹配.....C网络的容器网络我也会路由到这里就先这样写了) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78455f9e75379.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 同理,阿里云后台,打开路由表 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843b81f52465.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 将下一条指定到服务器i-uf6fxoj4zcqlpe8jupv2即10.20.4.42节点 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844384d39046.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## 测试一下其他节点的互通: 从A网络开始开始: **10.10.4.17 ping 10.20.4.42 10.20.4.43** ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78433b1123851.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 换一个subset: **10.10.2.4 ping 10.20.4.42 10.20.4.43** ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842bdd382793.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 可以在A集群10.10.4.8 tcpdump看一下icmp包的抓包: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7842e6e156480.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 反之B网络测试: 10.20.4.43 ping 10.10.4.8 10.10.4.17 10.10.2.4 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78443f3637325.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844a06874477.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844e4fb9349.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### AB 容器网络and vpc的互通: A网络run 一个nginx deployment 并开启service: 没有开启内网访问,开启一下吧,否则都没有办法用kubectl,将生成文件内容copy到A集群任一节点用户家目录下./kube/config文件中(当然了可以不开启,直接控制台操作) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843155f70223.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 可以kubectl run or 控制台创建一个nginx镜像就好 主要是创建一个service: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845474f95297.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) #### B网络服务器对A网络容器网络的连通性测试 - B网络服务器pingA网络 nginx pod service ip ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7846472787277.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) - B 网络服务器curl nginx pod service ip ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844f8b675136.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845c42a34856.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) #### B 网络集群创建pod 访问A 网络vpc and容器网络中应用: ``` kubectl run php --image=richarvey/nginx-php-fpm ``` 注意:可以随便找个带ping curl的镜像,这个镜像我是用习惯了 ``` [root@iZuf6fxoj4zcqlpe8jupv2Z wireguard]# kubectl exec -it php bash ``` - B网络集群中pod ping A网络vpc cvm地址 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844f9dd25493.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78448ce281026.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) - B集网络集群pod ping A网络集群 中nginx pod service ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78445b0278096.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784401cd66337.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) - B集网络集群pod curl A网络集群 中nginx pod service ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78451f256086.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) #### A网络服务器对B网络容器连通性测试 B网络创建nginx deployment svc ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844218c5184.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) - 同理A 网络cvm ping B网络容器集群nginx pod svc ip ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78465d4d48276.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784540f221653.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) - 同理curl 测试 curl就拿一台cvm测试了。。。。。 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844e9f953540.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843d8f384585.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) #### A 网络集群创建pod 访问B网络vpc and容器网络中应用: ``` kubectl run php --image=richarvey/nginx-php-fpm ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844d7f536219.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ``` root@VM-4-17-ubuntu:~/.kube# kubectl exec -it php bash bash-5.1# ping 10.20.4.42 bash-5.1# ping 10.20.4.43 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78442d9058950.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78470b808433.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ping 容器网络 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78468c257472.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845084714163.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844730147302.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78444bbb87716.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) # C连接B连通性测试 ## 配置类: ![](https://s2.51cto.com/images/blog/202208/24122228_6305a78441a6b36898.jpg?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 由图中关系可见需要在10.20.4.42节点允许10.30.0.3/24的通信 ### B集群10.20.4.42节点wg0.conf增加10.30.0.3相关设置 ``` [Peer] PublicKey = xxxxxxxxxxxxxxxxx AllowedIPs = 10.30.0.3/24 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844631395066.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ``` [root@iZuf6fxoj4zcqlpe8jupv2Z wireguard]# wg-quick down wg0 [root@iZuf6fxoj4zcqlpe8jupv2Z wireguard]# wg-quick up wg0 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78446ef570203.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### C集群10.40.2.16节点: ``` root@VM-2-16-ubuntu:/etc/wireguard# pwd /etc/wireguard root@VM-2-16-ubuntu:/etc/wireguard# cat wg0.conf [Interface] Address = 10.30.0.3/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE [Peer] # Client1: aliyun-shanghai PublicKey = xxxxxxxxxxxxxxxxxxxxxxxxxx Endpoint = xxxxxxxxxxxxxx:51820 AllowedIPs = 10.30.0.2/24,10.20.0.0/16,172.17.0.0/20 PersistentKeepalive = 25 ``` ``` root@VM-2-16-ubuntu:/etc/wireguard# wg-quick up wg0 ``` ## C集群cvm ping B集群vpc and nginx pod svc ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845ba3154747.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844984013310.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## C集群创建pod ping B集群网络 and 容器网络资源 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845714f3262.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845638248722.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## C 集群其他节点ping B集群网络 and 容器网络资源 en 登陆另外一节点10.40.3.16 ping B集群节点不通? ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78443ae872037.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844394374396.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 恩 依然是**路由**的问题,找到vpc 路由表,添加路由 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784567fb82044.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784796f942678.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) curl就稍微看一眼了 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784672fc50549.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## B集群ping C集群网络 and 容器网络资源 测试一下就好了....不通就是对的 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844ca0f13068.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) # A连接C连通性测试 ![](https://s2.51cto.com/images/blog/202208/24122228_6305a7845f87e99360.jpg?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## 配置类 ### C集群10.40.2.16节点 ``` root@VM-2-16-ubuntu:/etc/wireguard# cat wg0.conf [Interface] Address = 10.30.0.3/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE [Peer] # Client1: aliyun-shanghai PublicKey = xxxxxxxxxxxxxxxxxxxxxxxxxxx Endpoint = xxxxxxxxxxxxx:51820 AllowedIPs = 10.30.0.2/24,10.20.0.0/16,172.17.0.0/20 PersistentKeepalive = 25 [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxxxx AllowedIPs = 10.30.0.1/24 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78475d464957.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) **wg-quick重启wg0服务** ``` root@VM-2-16-ubuntu:/etc/wireguard# wg-quick down wg0 root@VM-2-16-ubuntu:/etc/wireguard# wg-quick up wg0 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843d46760829.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ### A集群10.0.4.8节点: ``` root@VM-4-8-ubuntu:/etc/wireguard# cat wg0.conf [Interface] Address = 10.30.0.1/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxxxx AllowedIPs = 10.30.0.0/24,10.20.0.0/16,172.17.0.0/20 [Peer] # Client1: aliyun-shanghai PublicKey = xxxxxxxxxxxxxxxxxxxxxxxxx Endpoint = xxxxxxxxxxxxx:51820 AllowedIPs = 10.30.0.2/24,10.20.0.0/16,172.17.0.0/20 PersistentKeepalive = 25 [Peer] # Client1: wireguard-beijing PublicKey = xxxxxxxxxxxxxxxxxx Endpoint = xxxxxxxxxxxxxxxxx:51820 AllowedIPs = 10.30.0.3/24,10.40.0.0/21,172.17.16.0/20 PersistentKeepalive = 25 ``` ``` root@VM-4-8-ubuntu:/etc/wireguard# wg-quick down wg0 root@VM-4-8-ubuntu:/etc/wireguard# wg-quick up wg0 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78455ccb98512.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## 强调一下 防火墙 udp:51820 前面没有说特意拿C网络来说一下。防火墙要开通UDP:51820(或者着针对自己的ip地址),前面AB集群我都默认打开了。不开通的时候是ping 不通的,打开tcp:51820依然不通,打开udp:51820就可以ping 通了! ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7843c39727924.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78440ae653907.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## A集群CVM10.10.4.8(wireguard)节点ping C集群网络 and 容器网络资源 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784501e977290.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## A集群进入容器pingC集群网络 and 容器网络资源 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844987482682.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## A集群其他CVM节点(安装wireguard节点外节点)ping C集群网络资源 A网络其他cvm ping C网络一定记得手动添加路由: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844404d83673.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7844881242914.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) ## C集群 ping A集群 草草测试一下吧吧...... ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784630b056607.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) # 问题来了: 上面 A B CB AC的测试再作一遍发现....AB不通了: ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a7845e4c483403.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 看一眼AB 节点的配置文件 A集群 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a78440f1563221.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) B集群 ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784560ae13611.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) 初步怀疑两个集群配置文件里面有冲突了?过程就是一次次改尝试 ## 最终配置文件如下: A集群10.0.4.8节点 ``` [Interface] Address = 10.30.0.1/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE #[Peer] #PublicKey =xxxxxxxxxxxxxxx #AllowedIPs = 10.30.0.2/24 [Peer] # Client1: aliyun-shanghai PublicKey =xxxxxxxxxxxxxxxxxxxxxx Endpoint = xxxxxx:51820 AllowedIPs = 10.30.0.2/32,10.20.0.0/16,172.17.0.0/20 PersistentKeepalive = 25 [Peer] # Client1: wireguard-beijing PublicKey = xxxxxxxxxxxxxxx Endpoint = xxxxxxxx:51820 AllowedIPs = 10.30.0.3/32,10.40.0.0/21,172.17.16.0/20 PersistentKeepalive = 25 ``` ![image.png](https://s2.51cto.com/images/blog/202208/24122228_6305a784342d963981.png?x-oss-process=image/watermark,size_14,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_30,g_se,x_10,y_10,shadow_20,type_ZmFuZ3poZW5naGVpdGk=) B集群10.20.4.42节点 ``` [Interface] Address = 10.30.0.2/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE #[Peer] #PublicKey = xxxxxxxxxxxxxxxxxxxxxxx #AllowedIPs = 10.30.0.1/24 [Peer] # Client1: MacOS-Desktop PublicKey = xxxxxxxxxxxxxxxxxxxxxxxx Endpoint = xxxxxxxxxxxxxxxxxxx:51820 AllowedIPs = 10.30.0.1/32,10.10.0.0/16,172.16.0.0/16 PersistentKeepalive = 25 [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxx AllowedIPs = 10.30.0.3/24 ``` C集群10.40.2.16节点 ``` [Interface] Address = 10.30.0.3/24 ListenPort = 51820 PrivateKey = xxxxxxxxxxxxxxx SaveConfig = false MTU = 1420 # Internet Gateway config: nat wg0 out to the internet on ens5 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE [Peer] # Client1: aliyun-shanghai PublicKey = xxxxxxxxxxxxxxxx Endpoint = xxxxxxx:51820 AllowedIPs = 10.30.0.2/32,10.20.0.0/16,172.17.0.0/20 PersistentKeepalive = 25 [Peer] PublicKey = xxxxxxxxxxxxxxxxxxxx AllowedIPs = 10.30.0.1/32 ``` ``` wg-quick down wg0 wg-quick up wg0 ``` 重复一遍各种Ping curl测试通过! # 总结: 1. wireguard跨网 组网还是很方便的 1. 两个网络组网还好,三个组网要注意下子网掩码或者各种的冲突,不知道是否会覆盖冲突...... 1. 网络基础还是薄弱,没有搞太明白....但是好歹三个网络的测试搞通了 1. 安全组防火墙的放行 1. 新增自定义路由策略的添加

举报

相关推荐

0 条评论