0
点赞
收藏
分享

微信扫一扫

k8s学习之使用ingress暴露集群内部的pod服务

编程练习生J 2022-02-14 阅读 57
kubernetesnginx运维

使用ingress暴露kubernetes集群内部的pod服务

Kubernetes暴露服务的方式目前只有三种：LoadBlancer Service、ExternalName、NodePort Service、Ingress，

四层负载均衡调度器Service可以以标签的形式选定一组带有指定标签的Pod，并监控和自动负载他们的Pod IP，通过NodePort暴露。当客户端访问kubernetes集群内部的应用时，数据包的走向如下面流程所示：client—>node ip:port—>service ip:port—>pod ip:port。

采用NodePort方式暴露服务面临的问题是，服务一旦多起来，NodePort在每个节点上开启的端口会及其庞大，而且难以维护；而使用 DaemonSet 在每个Node上监听 80的nginx反向代理又有域名分配及动态更新问题，此时Ingress出现了。

ngress 包含两大组件：Ingress Controller 和 Ingress

Ingress Controller：可以理解为控制器，它通过不断的跟 Kubernetes API 交互，实时获取后端 Service、Pod 的变化，比如新增、删除等，然后结合 Ingress 定义的规则生成配置，然后动态更新上边的七层负载均衡调度器 Nginx 或者trafik负载均衡器，并刷新使配置生效，来达到服务自动发现的作用。

nginx：需要手动加载配置文件
traefik：定期自动加载配置文件，不需要手动干预，在微服务中几乎都会使用这种调度器

Ingress：定义规则，通过它定义某个域名的请求过来之后转发到集群中指定的 Service。它可以通过 Yaml 文件定义，更新。可以给一个或多个 Service 定义一个或多个 Ingress 规则。简单的理解就是你原来需要改Nginx配置，然后配置各种域名对应哪个 Service

使用七层负载均衡调度器的步骤

（1）部署ingress controller，traefik或者nginx
（2）创建service，用来分组pod
（3）创建pod应用，可以通过控制器创建pod
（4）创建ingress http，测试通过http访问k8s内部pod
（5）创建ingress https，测试通过https访问k8s内部pod

使用七层负载均衡调度器ingress controller时，当客户端访问kubernetes集群内部的应用时，数据包走向如下面流程所示：

client—>Nodeip:port----->IngressController—>service—>pod

如何创建Ingress资源

Ingress资源是基于HTTP虚拟主机或URL的转发规则，它在资源配置清单中的spec字段中嵌套了rules、backend和tls等字段进行定义。如下定义一个Ingress资源转发规则：
将发往tomcat.lucky.com的请求，代理给一个名字为myapp的Service资源。

apiVersion: extensions/v1beta1    
kind: Ingress    
metadata:      
  name: ingress-myapp   
  namespace: default     
spec:     
  rules:   
  - host: tomcat.lucky.com   
    http:
      paths:       
      - path:       
        backend:    
          serviceName: myapp
          servicePort: 80

Ingress中的spec字段是Ingress资源的核心组成部分，主要包含以下3个字段：

**rules：**用于定义当前Ingress资源的转发规则列表；由rules定义规则，或没有匹配到规则，所有的流量会转发到由backend定义的默认后端。

**backend：**默认的后端用于服务那些没有匹配到任何规则的请求；定义Ingress资源时，必须要定义backend或rules两者之一，该字段用于让负载均衡器指定一个全局默认的后端。

**tls：**TLS配置，目前仅支持通过默认端口443提供服务，如果要配置指定的列表成员指向不同的主机，则需要通过SNI TLS扩展机制来支持该功能。

backend对象的定义由2个必要的字段组成：serviceName和servicePort，分别用于指定流量转发的后端目标Service资源名称和端口。

rules对象由一系列的配置的Ingress资源的host规则组成，这些host规则用于将一个主机上的某个URL映射到相关后端Service对象，其定义格式如下：

spec:
  rules:
  - hosts: <string>
    http:
      paths:
      - path:
        backend:
          serviceName: <string>
          servicePort: <string>

spec.rules.host属性值，目前暂不支持使用IP地址定义，也不支持IP:Port的格式，该字段留空，代表着通配所有主机名。

tls对象由2个内嵌的字段组成，仅在定义TLS主机的转发规则上使用。

**hosts：**包含 于 使用 的 TLS 证书 之内 的 主机 名称 字符串 列表， 因此， 此处 使用 的 主机 名 必须 匹配 tlsSecret 中的 名称。

secretName： 用于 引用 SSL 会话 的 secret 对象 名称， 在 基于 SNI 实现 多 主机 路 由 的 场景 中， 此 字段 为 可选。

Ingress资源类型

1、单Service资源型Ingress
2、基于URL路径进行流量转发
3、基于主机名称的虚拟主机
4、TLS类型的Ingress资源

单Service资源型Ingress

暴露单个服务的方法有多种，如NodePort、LoadBanlancer等等，当然也可以使用Ingress来进行暴露单个服务，只需要为Ingress指定default backend即可，

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
spec:
  backend:
    serviceName: my-svc
    servicePort: 80

Ingress控制器会为其分配一个IP地址接入请求流量，并将其转发至后端my-svc

Ingress Nginx部署

1、安装部署ingress controller Pod

创建命名空间

[root@master ingress]# cat namespace.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
[root@master ingress]# kubectl apply -f namespace.yaml
namespace/ingress-nginx created
[root@master ingress]# kubectl get namespace
NAME                   STATUS   AGE
ingress-nginx          Active   11s

创建

[root@master test]# cat rbac.yaml
---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
        - events
    verbs:
        - create
        - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

[root@master test]# cat with-rbac.yaml
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-nginx
  template:
    metadata:
      labels:
        app: ingress-nginx
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            capabilities:
                drop:
                - ALL
                add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
          - name: http
            containerPort: 80
          - name: https
            containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1

[root@master test]# cat configmap.yaml
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app: ingress-nginx

[root@master test]# cat default-backend.yaml
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissible as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: whychoice/defaultbackend:1.4
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---

apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: ingress-nginx
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend

[root@master test]# cat tcp-services-configmap.yaml
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx

[root@master test]# cat udp-services-configmap.yaml
---

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx

[root@master test]# kubectl apply -f .

[root@master test]# kubectl get all -n ingress-nginx
NAME         READY   STATUS    RESTARTS   AGE
pod/default-http-backend-78d75577fd-g5lfs  1/1   Running   0      29s
pod/nginx-ingress-controller-7c7d57b55d-g85w6   1/1   Running 0   29s

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/default-http-backend   ClusterIP   10.98.163.106   <none>        80/TCP    29s

NAME           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/default-http-backend       1/1     1   1          29s
deployment.apps/nginx-ingress-controller   1/1     1   1          29s

NAME        DESIRED   CURRENT   READY   AGE
replicaset.apps/default-http-backend-78d75577fd  1   1  1   29s
replicaset.apps/nginx-ingress-controller-7c7d57b55d  1  1  1  29s

2.部署后端服务

1）部署后端服务

cat deploy-demo.yaml

[root@master ingress]# cat deploy-demo.yaml
#创建service为myapp
apiVersion: v1
kind: Service
metadata:
  name: myapp
  namespace: default
spec:
  selector:
    app: myapp
    release: canary
  ports:
  - name: http
    targetPort: 80
    port: 80
---
#创建后端服务的pod
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-backend-pod
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
      release: canary
  template:
    metadata:
      labels:
        app: myapp
        release: canary
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v2
        ports:
        - name: http
          containerPort: 80

[root@master ingress]# kubectl apply -f deploy-demo.yaml
[root@master ingress]# kubectl get pod
NAME                                READY   STATUS    RESTARTS   AGE
myapp-backend-pod-559ff5c66-5s4mk   1/1     Running   0          51s
myapp-backend-pod-559ff5c66-g9zkc   1/1     Running   0          51s
myapp-backend-pod-559ff5c66-lxpjz   1/1     Running   0          51s

2）创建ingress-controller的service

[root@master ingress]# cat service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app: ingress-nginx
[root@master ingress]# ss -antup |grep 30443
[root@master ingress]# ss -antup |grep 30443
[root@master ingress]# kubectl apply -f service-nodeport.yaml
service/ingress-nginx created
[root@master ingress]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.98.163.106   <none>        80/TCP                       2h
ingress-nginx          NodePort    10.101.32.19    <none>        80:30080/TCP,443:30443/TCP   3s

http://192.168.1.12:30080/ 显示 default backend - 404

[root@master ingress]# cat ingress-myapp.yaml
apiVersion: extensions/v1beta1          #api版本
kind: Ingress           #清单类型
metadata:                       #元数据
  name: ingress-myapp    #ingress的名称
  namespace: default     #所属名称空间
  annotations:           #注解信息
    kubernetes.io/ingress.class: "nginx"
spec:      #规格
  rules:   #定义后端转发的规则
  - host: tomcat.lucky.com    #通过域名进行转发
    http:
      paths:
      - path:       #配置访问路径，如果通过url进行转发，需要修改；空默认为访问的路径为"/"
        backend:    #配置后端服务
          serviceName: myapp
          servicePort: 80

[root@master ingress]# kubectl apply -f ingress-myapp.yaml

C:\Windows\System32\drivers\etc\hosts写入

192.168.1.12 tomcat.lucky.com

http://tomcat.lucky.com:30080/ 显示如下

Hello MyApp | Version: v2 | Pod Name

进入nginx-ingress-control查看，可以找到自动加入的域名

server_name tomcat.lucky.com

[root@master ingress]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
default-http-backend-78d75577fd-g5lfs       1/1     Running   0       
nginx-ingress-controller-7c7d57b55d-g85w6   1/1     Running   0       
[root@master ingress]# kubectl exec -it -n ingress-nginx  nginx-ingress-controller-7c7d57b55d-g85w6 -- /bin/bash
www-data@nginx-ingress-controller-7c7d57b55d-g85w6:/etc/nginx$ ls
fastcgi.conf            koi-utf             modsecurity         owasp-modsecurity-crs  uwsgi_params.default	modules
fastcgi.conf.default    koi-win  scgi_params            win-utf
fastcgi_params          lua                 nginx.conf          scgi_params.default   fastcgi_params.default  mime.types          nginx.conf.default  template     geoip                   mime.types.default  opentracing.json    uwsgi_params
www-data@nginx-ingress-controller-7c7d57b55d-g85w6:/etc/nginx$ cat nginx.conf        server {
                server_name tomcat.lucky.com ;

                listen 80;

                listen [::]:80;

                set $proxy_upstream_name "-";

                location / {

5.部署ingress-测试代理tomcat服务

1）部署tomcat服务

[root@master ingress]# cat tomcat-demo.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat
  namespace: default
spec:
  selector:
    app: tomcat
    release: canary
  ports:
  - name: http
    targetPort: 8080
    port: 8080
  - name: ajp
    targetPort: 8009
    port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat
      release: canary
  template:
    metadata:
      labels:
        app: tomcat
        release: canary
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5-jre8-alpine
        #此镜像在dockerhub上进行下载，需要查看版本是否有变化，hub.docker.com
        ports:
        - name: http
          containerPort: 8080
          name: ajp
          containerPort: 8009
[root@master ingress]# kubectl apply -f tomcat-demo.yaml

[root@master ingress]# cat ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.lucky6.com    #主机域名
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@master ingress]# kubectl apply -f ingress-tomcat.yaml

C:\Windows\System32\drivers\etc\hosts写入

192.168.1.12 tomcat.lucky6.com

http://tomcat.lucky6.com:30080/ 显示

Home Documentation Configuration Examples Wiki Mailing Lists
Find HelpApache Tomcat/8.5.41的首页

总结

①下载Ingress-controller相关的YAML文件，给Ingress-controller创建独立的名称空间；

②部署后端的服务，如myapp，并通过service进行暴露；

③部署Ingress-controller的service，以实现接入集群外部流量；

④部署Ingress，进行定义规则，使Ingress-controller和后端服务的Pod组进行关联。

构建TLS站点

（1）准备证书，在k8s的master节点操作

[root@master ingress]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
.........................+++
.....+++
e is 65537 (0x10001)
[root@master ingress]# ls
tls.key
[root@master ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=tomcat.lucky.com

（2）生成secret，在k8s的master节点操作

[root@master ingress]# kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
[root@master ingress]# kubectl get secret
NAME                    TYPE                             DATA   AGE
default-token-86dcd     kubernetes.io/service-account-token 3  3d
tomcat-ingress-secret   kubernetes.io/tls                  2    18s

（3）查看tomcat-ingress-secret详细信息

[root@master ingress]# kubectl describe secret tomcat-ingress-secret
Name:         tomcat-ingress-secret
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  kubernetes.io/tls

Data
====
tls.crt:  1294 bytes
tls.key:  1675 bytes

（5）创建ingress

[root@master ingress]# cat ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-tls
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - tomcat.lucky.com
    secretName: tomcat-ingress-secret
  rules:
  - host: tomcat.lucky.com
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@master ingress]# kubectl apply -f ingress-tomcat-tls.yaml
ingress.extensions/ingress-tomcat-tls created

http://tomcat.lucky.com:30443/ 400 Bad Request

https://tomcat.lucky.com:30443/ Hello MyApp | Version: v2

举报

相关推荐

使用k8s Ingress暴露gRPC服务

ingressk8s云原生nginxNginxPython后端开发

k8s 暴露pod

vue.js前端javascript学习html5

k8s集群使用ingress转发grafana服务

listwindowsjava

k8s 使用ingress-nginx访问集群内部应用

pythonpycharmide

【Kubernetes】k8s集群对外服务之Ingress

kubernetes分布式容器

K8S：K8S对外服务之Ingress

kubernetes容器云原生运维

K8s之Pod学习

kubernetesdocker容器

K8S之Ingress

kubernetesnginx负载均衡工作原理云计算

k8s之Ingress

kubernetes

【k8s】pod集群调度

kubernetes容器云原生
0 条评论