0
点赞
收藏
分享

微信扫一扫

k8s学习之使用ingress暴露集群内部的pod服务

编程练习生J 2022-02-14 阅读 56

使用ingress暴露kubernetes集群内部的pod服务

Kubernetes暴露服务的方式目前只有三种:LoadBlancer Service、ExternalName、NodePort Service、Ingress,

四层负载均衡调度器Service可以以标签的形式选定一组带有指定标签的Pod,并监控和自动负载他们的Pod IP,通过NodePort暴露。当客户端访问kubernetes集群内部的应用时,数据包的走向如下面流程所示:client—>node ip:port—>service ip:port—>pod ip:port

采用NodePort方式暴露服务面临的问题是,服务一旦多起来,NodePort在每个节点上开启的端口会及其庞大,而且难以维护;而使用 DaemonSet 在每个Node上监听 80的nginx反向代理又有域名分配及动态更新问题,此时Ingress出现了。

ngress 包含两大组件:Ingress Controller 和 Ingress

Ingress Controller:可以理解为控制器,它通过不断的跟 Kubernetes API 交互,实时获取后端 Service、Pod 的变化,比如新增、删除等,然后结合 Ingress 定义的规则生成配置,然后动态更新上边的七层负载均衡调度器 Nginx 或者trafik负载均衡器,并刷新使配置生效,来达到服务自动发现的作用。

nginx:需要手动加载配置文件
traefik:定期自动加载配置文件,不需要手动干预,在微服务中几乎都会使用这种调度器

Ingress:定义规则,通过它定义某个域名的请求过来之后转发到集群中指定的 Service。它可以通过 Yaml 文件定义,更新。可以给一个或多个 Service 定义一个或多个 Ingress 规则。简单的理解就是你原来需要改Nginx配置,然后配置各种域名对应哪个 Service

使用七层负载均衡调度器的步骤

(1)部署ingress controller,traefik或者nginx
(2)创建service,用来分组pod
(3)创建pod应用,可以通过控制器创建pod
(4)创建ingress http,测试通过http访问k8s内部pod
(5)创建ingress https,测试通过https访问k8s内部pod

使用七层负载均衡调度器ingress controller时,当客户端访问kubernetes集群内部的应用时,数据包走向如下面流程所示:

client—>Nodeip:port----->IngressController—>service—>pod

如何创建Ingress资源

Ingress资源是基于HTTP虚拟主机或URL的转发规则,它在资源配置清单中的spec字段中嵌套了rules、backend和tls等字段进行定义。如下定义一个Ingress资源转发规则:
将发往tomcat.lucky.com的请求,代理给一个名字为myapp的Service资源。

apiVersion: extensions/v1beta1    
kind: Ingress    
metadata:      
  name: ingress-myapp   
  namespace: default     
spec:     
  rules:   
  - host: tomcat.lucky.com   
    http:
      paths:       
      - path:       
        backend:    
          serviceName: myapp
          servicePort: 80

Ingress中的spec字段是Ingress资源的核心组成部分,主要包含以下3个字段:

**rules:**用于定义当前Ingress资源的转发规则列表;由rules定义规则,或没有匹配到规则,所有的流量会转发到由backend定义的默认后端。

**backend:**默认的后端用于服务那些没有匹配到任何规则的请求;定义Ingress资源时,必须要定义backend或rules两者之一,该字段用于让负载均衡器指定一个全局默认的后端。

**tls:**TLS配置,目前仅支持通过默认端口443提供服务,如果要配置指定的列表成员指向不同的主机,则需要通过SNI TLS扩展机制来支持该功能。

backend对象的定义由2个必要的字段组成:serviceName和servicePort,分别用于指定流量转发的后端目标Service资源名称和端口。

rules对象由一系列的配置的Ingress资源的host规则组成,这些host规则用于将一个主机上的某个URL映射到相关后端Service对象,其定义格式如下:

spec:
  rules:
  - hosts: <string>
    http:
      paths:
      - path:
        backend:
          serviceName: <string>
          servicePort: <string>

spec.rules.host属性值,目前暂不支持使用IP地址定义,也不支持IP:Port的格式,该字段留空,代表着通配所有主机名。

tls对象由2个内嵌的字段组成,仅在定义TLS主机的转发规则上使用。

**hosts:**包含 于 使用 的 TLS 证书 之内 的 主机 名称 字符串 列表, 因此, 此处 使用 的 主机 名 必须 匹配 tlsSecret 中的 名称。

secretName: 用于 引用 SSL 会话 的 secret 对象 名称, 在 基于 SNI 实现 多 主机 路 由 的 场景 中, 此 字段 为 可选。

Ingress资源类型

1、单Service资源型Ingress
2、基于URL路径进行流量转发
3、基于主机名称的虚拟主机
4、TLS类型的Ingress资源

单Service资源型Ingress

暴露单个服务的方法有多种,如NodePort、LoadBanlancer等等,当然也可以使用Ingress来进行暴露单个服务,只需要为Ingress指定default backend即可,

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
spec:
  backend:
    serviceName: my-svc
    servicePort: 80

Ingress控制器会为其分配一个IP地址接入请求流量,并将其转发至后端my-svc

Ingress Nginx部署

1、安装部署ingress controller Pod

创建命名空间

[root@master ingress]# cat namespace.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
[root@master ingress]# kubectl apply -f namespace.yaml
namespace/ingress-nginx created
[root@master ingress]# kubectl get namespace
NAME                   STATUS   AGE
ingress-nginx          Active   11s

创建

[root@master test]# cat rbac.yaml
---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
        - events
    verbs:
        - create
        - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx
[root@master test]# cat with-rbac.yaml
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            capabilities:
                drop:
                - ALL
                add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
[root@master test]# cat configmap.yaml
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
[root@master test]# cat default-backend.yaml
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissible as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: whychoice/defaultbackend:1.4
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---

apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: ingress-nginx
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend
[root@master test]# cat tcp-services-configmap.yaml
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
[root@master test]# cat udp-services-configmap.yaml
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
[root@master test]# kubectl apply -f .
[root@master test]# kubectl get all -n ingress-nginx
NAME         READY   STATUS    RESTARTS   AGE
pod/default-http-backend-78d75577fd-g5lfs  1/1   Running   0      29s
pod/nginx-ingress-controller-7c7d57b55d-g85w6   1/1   Running 0   29s

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/default-http-backend   ClusterIP   10.98.163.106   <none>        80/TCP    29s

NAME           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/default-http-backend       1/1     1   1          29s
deployment.apps/nginx-ingress-controller   1/1     1   1          29s

NAME        DESIRED   CURRENT   READY   AGE
replicaset.apps/default-http-backend-78d75577fd  1   1  1   29s
replicaset.apps/nginx-ingress-controller-7c7d57b55d  1  1  1  29s

2.部署后端服务

1)部署后端服务

cat deploy-demo.yaml

[root@master ingress]# cat deploy-demo.yaml
#创建service为myapp
apiVersion: v1
kind: Service
metadata:
  name: myapp
  namespace: default
spec:
  selector:
    app: myapp
    release: canary
  ports:
  - name: http
    targetPort: 80
    port: 80
---
#创建后端服务的pod
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-backend-pod
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
      release: canary
  template:
    metadata:
      labels:
        app: myapp
        release: canary
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v2
        ports:
        - name: http
          containerPort: 80
[root@master ingress]# kubectl apply -f deploy-demo.yaml
[root@master ingress]# kubectl get pod
NAME                                READY   STATUS    RESTARTS   AGE
myapp-backend-pod-559ff5c66-5s4mk   1/1     Running   0          51s
myapp-backend-pod-559ff5c66-g9zkc   1/1     Running   0          51s
myapp-backend-pod-559ff5c66-lxpjz   1/1     Running   0          51s

2)创建ingress-controller的service

[root@master ingress]# cat service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app: ingress-nginx
[root@master ingress]# ss -antup |grep 30443
[root@master ingress]# ss -antup |grep 30443
[root@master ingress]# kubectl apply -f service-nodeport.yaml
service/ingress-nginx created
[root@master ingress]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.98.163.106   <none>        80/TCP                       2h
ingress-nginx          NodePort    10.101.32.19    <none>        80:30080/TCP,443:30443/TCP   3s

http://192.168.1.12:30080/ 显示 default backend - 404

[root@master ingress]# cat ingress-myapp.yaml
apiVersion: extensions/v1beta1          #api版本
kind: Ingress           #清单类型
metadata:                       #元数据
  name: ingress-myapp    #ingress的名称
  namespace: default     #所属名称空间
  annotations:           #注解信息
    kubernetes.io/ingress.class: "nginx"
spec:      #规格
  rules:   #定义后端转发的规则
  - host: tomcat.lucky.com    #通过域名进行转发
    http:
      paths:
      - path:       #配置访问路径,如果通过url进行转发,需要修改;空默认为访问的路径为"/"
        backend:    #配置后端服务
          serviceName: myapp
          servicePort: 80

[root@master ingress]# kubectl apply -f ingress-myapp.yaml

C:\Windows\System32\drivers\etc\hosts写入

192.168.1.12 tomcat.lucky.com

http://tomcat.lucky.com:30080/ 显示如下

Hello MyApp | Version: v2 | Pod Name

进入nginx-ingress-control查看,可以找到自动加入的域名

server_name tomcat.lucky.com

[root@master ingress]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
default-http-backend-78d75577fd-g5lfs       1/1     Running   0       
nginx-ingress-controller-7c7d57b55d-g85w6   1/1     Running   0       
[root@master ingress]# kubectl exec -it -n ingress-nginx  nginx-ingress-controller-7c7d57b55d-g85w6 -- /bin/bash
www-data@nginx-ingress-controller-7c7d57b55d-g85w6:/etc/nginx$ ls
fastcgi.conf            koi-utf             modsecurity         owasp-modsecurity-crs  uwsgi_params.default	modules
fastcgi.conf.default    koi-win  scgi_params            win-utf
fastcgi_params          lua                 nginx.conf          scgi_params.default   fastcgi_params.default  mime.types          nginx.conf.default  template     geoip                   mime.types.default  opentracing.json    uwsgi_params
www-data@nginx-ingress-controller-7c7d57b55d-g85w6:/etc/nginx$ cat nginx.conf        server {
                server_name tomcat.lucky.com ;

                listen 80;

                listen [::]:80;

                set $proxy_upstream_name "-";

                location / {

5.部署ingress-测试代理tomcat服务

1)部署tomcat服务

[root@master ingress]# cat tomcat-demo.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat
  namespace: default
spec:
  selector:
    app: tomcat
    release: canary
  ports:
  - name: http
    targetPort: 8080
    port: 8080
  - name: ajp
    targetPort: 8009
    port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat
      release: canary
  template:
    metadata:
      labels:
        app: tomcat
        release: canary
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5-jre8-alpine
        #此镜像在dockerhub上进行下载,需要查看版本是否有变化,hub.docker.com
        ports:
        - name: http
          containerPort: 8080
          name: ajp
          containerPort: 8009
[root@master ingress]# kubectl apply -f tomcat-demo.yaml
[root@master ingress]# cat ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.lucky6.com    #主机域名
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@master ingress]# kubectl apply -f ingress-tomcat.yaml

C:\Windows\System32\drivers\etc\hosts写入

192.168.1.12 tomcat.lucky6.com

http://tomcat.lucky6.com:30080/ 显示

Home Documentation Configuration Examples Wiki Mailing Lists
Find HelpApache Tomcat/8.5.41的首页

总结

①下载Ingress-controller相关的YAML文件,给Ingress-controller创建独立的名称空间;

②部署后端的服务,如myapp,并通过service进行暴露;

③部署Ingress-controller的service,以实现接入集群外部流量;

④部署Ingress,进行定义规则,使Ingress-controller和后端服务的Pod组进行关联。

构建TLS站点

(1)准备证书,在k8s的master节点操作

[root@master ingress]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
.........................+++
.....+++
e is 65537 (0x10001)
[root@master ingress]# ls
tls.key
[root@master ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=tomcat.lucky.com

(2)生成secret,在k8s的master节点操作

[root@master ingress]# kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
[root@master ingress]# kubectl get secret
NAME                    TYPE                             DATA   AGE
default-token-86dcd     kubernetes.io/service-account-token 3  3d
tomcat-ingress-secret   kubernetes.io/tls                  2    18s

(3)查看tomcat-ingress-secret详细信息

[root@master ingress]# kubectl describe secret tomcat-ingress-secret
Name:         tomcat-ingress-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1294 bytes
tls.key:  1675 bytes

(5)创建ingress

[root@master ingress]# cat ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-tls
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - tomcat.lucky.com
    secretName: tomcat-ingress-secret
  rules:
  - host: tomcat.lucky.com
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@master ingress]# kubectl apply -f ingress-tomcat-tls.yaml
ingress.extensions/ingress-tomcat-tls created

http://tomcat.lucky.com:30443/ 400 Bad Request

https://tomcat.lucky.com:30443/ Hello MyApp | Version: v2

举报

相关推荐

0 条评论