K8S搭建Redis 7.4高可用集群-CFANZ编程社区

网上看到很多资料搭建redis集群的时候都存在redis节点出现IP变化时，需要重新更新redis的IP，不过redis7.0之后就可以使用POD的名称进行集群的关联，极大减少维护工作量

主要步骤:

1.共享动态存储（这里使用NFS服务，需要提前搭建）

2.服务搭建

3.集群创建以及测试

1.创建共享动态存储

（1）共享存储的RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

(2)搭建deploy部署

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: nfs-client-provisioner （这里需要修改）
          # image: quay.mirrors.ustc.edu.cn/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 192.168.0.1
            - name: NFS_PATH
              value: /managed-nfs-storage
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.0.1
            path: /managed-nfs-storage

(3)class创建

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

2.redis服务的搭建

（1）配置文件

kind: ConfigMap
apiVersion: v1
metadata:
  name: redis-cluster-configmap # configmap的名字，加上下面的demo-redis就是这个configmap在k8s集群中的唯一标识
  namespace: default
data:
  redis.conf: |
    bind 0.0.0.0
    port 6379
    daemonize no
    appendonly yes
    protected-mode no
    cluster-enabled yes
    cluster-config-file /var/lib/redis/nodes.conf
    cluster-node-timeout 5000
    dir /var/lib/redis
    cluster-announce-bus-port 16379

2.服务

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
  namespace: default
spec:
  serviceName: redis-cluster-headless
  replicas: 6
  selector:
    matchLabels:
      app: redis
      appCluster: redis-cluster
  template:
    metadata:
      labels:
        app: redis
        appCluster: redis-cluster
    spec:
      imagePullSecrets:
      - name: aliyun-registry-secret
      terminationGracePeriodSeconds: 20
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - redis
              topologyKey: kubernetes.io/hostname
      containers:
      - name: redis
        image: registry.cn-shanghai.aliyuncs.com/qlxp/redis:7.4
        env:
          - name: TZ
            value: Asia/Shanghai
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_SERVICE_NAME
            value: "redis-cluster-headless"
        command:
          - "redis-server"                  #redis启动命令
        args:
          - "/etc/redis/redis.conf"         #redis-server后面跟的参数,换行代表空格
          - "--protected-mode"              #允许外网访问
          - "no"
          - "--cluster-announce-ip"
          - "$(POD_NAME).$(POD_SERVICE_NAME)"
        resources:
          requests:                         # 每个pod请求的资源
            cpu: 64m                      # m代表千分之，这里申请2个逻辑核
            memory: 128M                     # 内存申请4G大小
          limits:                           # 资源限制
            cpu: 2000m
            memory: 4Gi
        readinessProbe:
          failureThreshold: 2
          tcpSocket:
            port: 6379
          initialDelaySeconds: 10
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        livenessProbe:
          failureThreshold: 2
          tcpSocket:
            port: 6379
          initialDelaySeconds: 16
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        ports:
            - name: redis
              containerPort: 6379
              protocol: "TCP"
            - name: cluster
              containerPort: 16379
              protocol: "TCP"
        volumeMounts:
          - name: redis-conf              # 把下面创建的redis.conf配置文件挂载到容器的/etc/redis目录下
            mountPath: /etc/redis
          - name: redis-data              # 把叫做redis-data的volume挂载到容器的/var/lib/redis目录
            mountPath: /var/lib/redis
      volumes:
      - name: redis-conf                  # 创建一个名为redis-conf的volumes
        configMap:
          name: redis-cluster-configmap
          items:
            - key: redis.conf
              path: redis.conf
  volumeClaimTemplates:                   # pod使用哪个pvc，这里是通过StorageClass自动创建pvc并对应上pv
  - metadata:
      name: redis-data                    # pvc创建一个volumes叫做redis-data
    spec:
      accessModes:
      - ReadWriteOnce
      storageClassName: managed-nfs-storage
      resources:
        requests:
          storage: 5Gi

3.redis svc的创建

apiVersion: v1
kind: Service
metadata:
  name: redis-cluster-headless
  namespace: default
  labels:
    app: redis
spec:
  ports:
  - name: redis-port
    port: 6379
  - name: redis-cluster-port
    port: 16379
  clusterIP: None
  selector:
    app: redis
    appCluster: redis-cluster

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  namespace: default
  labels:
    app: redis
spec:
  ports:
  - name: redis-port
    protocol: TCP
    port: 6379
    targetPort: 6379
  selector:
    app: redis
    appCluster: redis-cluster

3.集群的创建

kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 redis-cluster-0.redis-cluster-headless:6379 redis-cluster-1.redis-cluster-headless:6379 redis-cluster-2.redis-cluster-headless:6379 redis-cluster-3.redis-cluster-headless:6379 redis-cluster-4.redis-cluster-headless:6379 redis-cluster-5.redis-cluster-headless:6379