云原生微服务第四章之使用kubeasz部署二进制k8s集群

一、机器环境

二、所有机器初始化

注：所有机器（10）都执行一篇，部署机初始化完后再重启

systemctl stop firewalld
systemctl disable firewalld
yum -y install ntpdate
setenforce 0
sed -i 's/enforcing/disabled/' /etc/selinux/config
yum -y remove docker-ce docker-ce-cli docker-scaqn-plugin containerd
echo "* * * * *  root ntpdate time.windows.com"  >> /etc/crontab

三、部署机部署前的准备工作

yum -y install ansible git sshpass
cat > /root/key.sh <<eof
#!/bin/bash

IP="
10.0.7.18
10.0.7.19
10.0.7.20
10.0.7.21
10.0.7.22
10.0.7.23
10.0.7.24
10.0.7.25
10.0.7.26
"

for NODE in ${IP}
do
        sshpass -p 你的服务器密码 ssh-copy-id ${NODE} -o StrictHostKeyChecking=no
done
eof

sh /root/key.sh

四、部署kubeasz

export release=3.3.1
wget https://github.com/easzlab/kubeasz/releases/download/${release}/ezdown
chmod +x ./ezdown
cd /etc/kubeasz
./ezdown -D

五、初始化集群并配置

./ezctl new k8s-cluster1 
 
 
 vim clusters/k8s-cluster1/hosts
 # 要修改的地方
 [etcd]
10.0.7.18
10.0.7.19
10.0.7.20
# master node(s)
[kube_master]
10.0.7.21
10.0.7.22
# work node(s)
[kube_node]
10.0.7.24
10.0.7.25

# K8S Service CIDR, not overlap with node(host) networking
SERVICE_CIDR="10.100.0.0/16"

# Cluster CIDR (Pod CIDR), not overlap with node(host) networking
CLUSTER_CIDR="10.200.0.0/16"

# NodePort Range
NODE_PORT_RANGE="30000-62767"

# Binaries Directory
bin_dir="/usr/local/bin"


vim clusters/k8s-cluster1/config.yml 
############################
# role:kube-master
############################
# k8s 集群 master 节点证书配置，可以添加多个ip和域名（比如增加公网ip和域名）
MASTER_CERT_HOSTS:
  - "10.0.7.6"    # 填写VIP
  
# node节点最大pod 数
MAX_PODS: 500 

############################
# role:cluster-addon
############################
下面的插件安装全部改为no
..

六、K8s集群服务器初始化

vim playbooks/01.prepare.yml    
# [optional] to synchronize system time of nodes with 'chrony' 
# 注释 ex_lob chrony
- hosts:
  - kube_master
  - kube_node
  - etcd
 # - ex_lb
 # - chrony

# 服务器初始化
./ezctl setup k8s-cluster 01

七、部署ETCD

# 部署ETCD
./ezctl setup k8s-cluster 02

# 验证etcd
 NODE_IPS="10.0.7.18 10.0.7.19 10.0.7.20"
for ip in ${NODE_IPS}; do  /usr/local/bin/etcdctl  --endpoints=https://${ip}:2379 --cacert=/etc/kubernetes/ssl/ca.pem --cert=/etc/kubernetes/ssl/etcd.pem --key=/etc/kubernetes/ssl/etcd-key.pem endpoint health;done

# 检查ETCD数据库性能
NODE_IPS="10.0.7.18 10.0.7.19 10.0.7.20"
for ip in ${NODE_IPS}; do  /usr/local/bin/etcdctl --write-out=table --endpoints=https://${ip}:2379 --cacert=/etc/kubernetes/ssl/ca.pem --cert=/etc/kubernetes/ssl/etcd-peer.pem --key=/etc/kubernetes/ssl/etcd-peer-key.pem check perf;done

八、配置部署Containerd

vim roles/containerd/templates/config.toml.j2 

如果你的harbor没有安装https，加以下两行，即可以拉harbor的公开仓库
         [plugins."io.containerd.grpc.v1.cri".registry.mirrors."10.0.7.12"]
          endpoint = ["http://10.0.7.12"]
      
如果你的harbor安装了证书，还需要在140行下面加多以下两行跳过证书验证
        [plugins."io.containerd.grpc.v1.cri".registry.configs."10.0.7.12".tls]
          insecure_skip_verify = true      
      
如果你还需要push镜像需要再加多以下两行
        [plugins."io.containerd.grpc.v1.cri".registry.configs."10.0.7.12".auth]
          username = "admin"
          password = "Lava28691"
          
# 部署Containerd          
./ezctl setup k8s-cluster 03

# 扩展
# containerd 配置文件亦可这样生成
containerd config default > /root/a.toml

九、部署K8s-Master

./ezctl setup k8s-cluster 04

十、部署K8s-Node

./ezctl setup k8s-cluster 05

十一、部署Calico网络

./ezctl setup k8s-cluster 06

十二、验证集群以及网络

# 部署机操作
kubectl get cs  # 查看集群状态
kubectl get node -o wide # 查看node节点
kubectl get pods -A # 查看刚安装的Calico

# master 或 node 节点操作，查看calico状态
calicoctl node status

十三、运行一个容器测试

kubectl create ns yhttest
kubectl run test1 --image=centos:7.9.2009 sleep 1000000000000000000000000 -n yhttest
kubectl get pods -n yhttest -o wide
kubectl exec -it test1 sh -n yhttest
ping 163.177.151.110

十四、销毁集群

./ezctl destroy