0
点赞
收藏
分享

微信扫一扫

Azure Kubernetes Service 和 Argo CD

Brose 2022-08-18 阅读 54
gitdockeraksargocdgitlabkubernetes云计算

概述

最近一个Azure的项目需要先做一系列Proof of Concept （PoC）可行性的演示。下面简单的记录一下做过的准备。

  1. 首先是通过Gitlab和Terraform来搭建一个Azure的服务器的环境
    https://blog.51cto.com/beanxyz/5547469
  2. 通过Gitlab来创建容器，并能上传到Azure Container Registry里面
    https://blog.51cto.com/beanxyz/5551081
    https://blog.51cto.com/beanxyz/5565612
  3. 打算创建Azure Kubernetes Service，并通过Argo CD来部署第二步创建的容器镜像。

下面看看第三步是如何实现的。

Terraform 创建AKS

关于service principal和相关的环境变量我在第一步已经搭建好了，因此这里直接就不记录了。下面是相关的tf 代码。微软的官方网站提供的演示代码使用的是azurerm 2.x的版本，我这里改成了最新的3.x的版本，因此addon_profile的语法有所不同

main.tf

# Generate random resource group name
resource "random_pet" "rg-name" {
  prefix    = var.resource_group_name_prefix
}

resource "azurerm_resource_group" "k8s" {
  name      = random_pet.rg-name.id
  location  = var.resource_group_location
}

resource "random_id" "log_analytics_workspace_name_suffix" {
  byte_length = 8
}

resource "azurerm_log_analytics_workspace" "test" {
  # The WorkSpace name has to be unique across the whole of azure, not just the current subscription/tenant.
  name                = "${var.log_analytics_workspace_name}-${random_id.log_analytics_workspace_name_suffix.dec}"
  location            = var.log_analytics_workspace_location
  resource_group_name = azurerm_resource_group.k8s.name
  sku                 = var.log_analytics_workspace_sku
}

resource "azurerm_log_analytics_solution" "test" {
  solution_name         = "ContainerInsights"
  location              = azurerm_log_analytics_workspace.test.location
  resource_group_name   = azurerm_resource_group.k8s.name
  workspace_resource_id = azurerm_log_analytics_workspace.test.id
  workspace_name        = azurerm_log_analytics_workspace.test.name

  plan {
    publisher = "Microsoft"
    product   = "OMSGallery/ContainerInsights"
  }
}

resource "azurerm_kubernetes_cluster" "k8s" {
  name                = var.cluster_name
  location            = azurerm_resource_group.k8s.location
  resource_group_name = azurerm_resource_group.k8s.name
  dns_prefix          = var.dns_prefix

#  linux_profile {
#    admin_username = "ubuntu"
#
#
#    ssh_key {
#      key_data = file(var.ssh_public_key)
#    }
#  }

  default_node_pool {
    name            = "agentpool"
    node_count      = var.agent_count
    vm_size         = "Standard_D2_v2"
  }
  identity {
    type = "SystemAssigned"
  }

#  service_principal {
#    client_id     = var.aks_service_principal_app_id
#    client_secret = var.aks_service_principal_client_secret
#  }

  # addon_profile {
  #   oms_agent {
  #     enabled                    = true
  #     log_analytics_workspace_id = azurerm_log_analytics_workspace.test.id
  #   }
  #   http_application_routing {
  #     enabled = true
  #   }
  # }
  http_application_routing_enabled = true

  oms_agent {

       log_analytics_workspace_id = azurerm_log_analytics_workspace.test.id
  }
  network_profile {
    load_balancer_sku = "standard"
    network_plugin = "kubenet"
  }

  tags = {
    Environment = "Development"
  }
}

data "azurerm_container_registry" "acr" {
  name                = "gitlabazuredemo"
  resource_group_name = "rg1"
}

# add the role to the identity the kubernetes cluster was assigned
resource "azurerm_role_assignment" "kubweb_to_acr" {
  scope                = data.azurerm_container_registry.acr.id
  role_definition_name = "AcrPull"
  principal_id         = azurerm_kubernetes_cluster.k8s.kubelet_identity[0].object_id
}

provider.tf

terraform {

  required_version = ">=0.12"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~>3.0"
    }
  }
  backend "azurerm" {
    resource_group_name  = "rg1"
    storage_account_name = "gitlabstoragetest"
    container_name       = "terraform-state-file"
    key                  = "test.terraform.tfstate"
  }
}

provider "azurerm" {
  features {}
}

variables.tf

variable "resource_group_name_prefix" {
  default       = "rg"
  description   = "Prefix of the resource group name that's combined with a random ID so name is unique in your Azure subscription."
}

variable "resource_group_location" {
  default       = "eastasia"
  description   = "Location of the resource group."
}

variable "agent_count" {
  default = 1
}

#variable "ssh_public_key" {
#  default = "~/.ssh/id_rsa.pub"
#}

variable "dns_prefix" {
  default = "k8stest"
}

variable cluster_name {
  default = "k8stest"
}

variable resource_group_name {
  default = "azure-k8stest"
}

variable location {
  default = "East Asia"
}

variable log_analytics_workspace_name {
  default = "testLogAnalyticsWorkspaceName"
}

# refer https://azure.microsoft.com/global-infrastructure/services/?products=monitor for log analytics available regions
variable log_analytics_workspace_location {
  default = "eastasia"
}

# refer https://azure.microsoft.com/pricing/details/monitor/ for log analytics pricing
variable log_analytics_workspace_sku {
  default = "PerGB2018"
}

output.tf

output "resource_group_name" {
  value = azurerm_resource_group.k8s.name
}

在gitlab里面我的CI的架构仍然是使用子目录的形式，如下所示

.gitlab-ci.yml
.sub-gitlab-ci.yml
angularjs_project 
    - .gitlab-ci.yml
    -  Dockerfile
    -  other files..
dotnet_project
    - .gitlab-ci.yml
    - Dockerfile
    - other files..    
aks_project
    - .gitlab-ci.yml
    - terraform files
    - test folder

最外面的 .gitlab-ci.yml

stages:
  - child_pipeline

aks:
  stage: child_pipeline
  variables:
    CHILD_PIPELINE_EXECUTION_CONTEXT: "aks_project"
  trigger:
    include: .sub-gitlab-ci.yml
    strategy: depend

dotnet:
  stage: child_pipeline
  when: manual
  variables:
    CHILD_PIPELINE_EXECUTION_CONTEXT: "dotnet_project"
  trigger:
    include: .sub-gitlab-ci.yml
    strategy: depend

angularjs:
  stage: child_pipeline
  when: manual
  variables:
    CHILD_PIPELINE_EXECUTION_CONTEXT: "angularjs_project"
  trigger:
    include: .sub-gitlab-ci.yml
    strategy: depend

最外面的.sub-gitlab-ci.yml,这里我使用了before_script,他会在每个stage里面的job的script执行之前执行下面的代码

before_script:
    - "echo Running child pipeline in subdirectory: $CHILD_PIPELINE_EXECUTION_CONTEXT"
    - cd $CHILD_PIPELINE_EXECUTION_CONTEXT
    - apk add terraform --repository=https://dl-cdn.alpinelinux.org/alpine/edge/community
    - rm -rf .terraform
    - terraform --version
    - terraform init

include: $CHILD_PIPELINE_EXECUTION_CONTEXT/.gitlab-ci.yml

最后是aks_project目录里面的.gitlab-ci.yml
.gitlab-ci.yml CI的文件

image:
  name: mcr.microsoft.com/azure-cli
  entrypoint:
    - '/usr/bin/env'
    - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

stages:
  - validate
  - plan
  - apply

validate:
  stage: validate
  script:
    - terraform validate

plan:
  stage: plan
  script:
    - terraform plan 
  dependencies:
    - validate

apply:
  stage: apply
  script:
    - terraform apply -auto-approve
  dependencies:
    - plan

执行相关的代码 会成功创建AKS

image.png

user@VirtualBox:~/devops/gitlab-2-azure/aks_project$ kubectl get node
NAME                                STATUS   ROLES   AGE   VERSION
aks-agentpool-11728208-vmss000000   Ready    agent   21h   v1.22.11

配置使用Argo CD

安装 argo cd

执行下面的命令，即可安装argo cd

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl get all -n argocd

为了访问argo cd的web界面，最简单的方式是可以直接通过service来暴露端口，然后访问对应的外网IP即可

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

如果不想给外界暴露端口，也可以从本地端口转发的形式来访问 http://localhost:8080

kubectl port-forward svc/argocd-server -n argocd 8080:443

默认的用户名是 admin, 密码可以通过下面的命令获取

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

测试manifest文件

简单的说，他会从acr上面pull我之前上传的镜像，创建deployment，service和ingress。 我在之前创建aks的时候，已经自动创建了ingress controller的addon，和绑定了acr，因此这里可以直接使用。如果我的docker registry是保存在第三方的，那么我还需要创建secret来保存docker的用户密码，然后挂载在pod下面。但是因为secret本身保存的是base4格式的明文，出于安全原因所以不可以直接存放在git repo中，那有需要额外的解决方案来保存相关的key和value。从演示的角度，这种azure的全家桶解决方案是最简单的。

deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-dotnet
spec:
  selector:
    matchLabels:
      name: hello-dotnet
  template:
    metadata:
      labels:
        name: hello-dotnet
    spec:
      containers:
        - name: test
          image: gitlabazuredemo.azurecr.io/dotnet_demo:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 80

ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-dotnet
  annotations:
    kubernetes.io/ingress.class: addon-http-application-routing
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
               service:
                  name: hello-dotnet
                  port: 
                    number: 80

svc.yml

apiVersion: v1
kind: Service
metadata:
  name: hello-dotnet
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 80
  selector:
    name: hello-dotnet

Argo cd配置 manifest文件

首先先要配置一下repository，我这里用ssh 连接我的gitlab repo
image.png

然后添加app

image.png
image.png

刷新一下

image.png

最后验证一下


user@VirtualBox:~/devops/gitlab-2-azure/aks_project$ kubectl get all 

NAME                               READY   STATUS    RESTARTS   AGE

pod/hello-dotnet-cfd9bd58c-8gqpc   1/1     Running   0          20h

NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE

service/hello-dotnet   ClusterIP   10.0.217.138   <none>        80/TCP    20h

service/kubernetes     ClusterIP   10.0.0.1       <none>        443/TCP   22h

NAME                           READY   UP-TO-DATE   AVAILABLE   AGE

deployment.apps/hello-dotnet   1/1     1            1           20h

NAME                                     DESIRED   CURRENT   READY   AGE

replicaset.apps/hello-dotnet-cfd9bd58c   1         1         1       20h

image.png

举报

相关推荐

【 云原生 | kubernetes 】- Argo CD 持续交付

git应用程序githubkubernetes云计算yyds干货盘点

Argo CD 基础

应用程序APIServerHtml/CSS前端开发

Tekton 结合 Argo CD

dockerIT职场

Kubernetes声明性GitOps持续交付工具-Argo CD

CICD运维

Argo CD系列视频配套图文版-Argo CD初步认识

javapythonkubernetesgitdocker系统/运维

Argo CD 实践教程 02

gitkubernetesdocker原力计划

Argo项目实战示例：Argo Workflows（when分支，循环，递归等）； Argo CD； Argo Events； Argo Rollouts

kubernetesjava容器新版本多任务编程语言

Argo CD安装配置

argocdk8s运维yyds干货盘点

【Azure 应用服务】基于Azure的CI/CD工具链部署App Service

gitlinuxgithub编程语言

【 云原生 | kubernetes 】- Argo CD Gitlab身份验证及SSO单点登录

身份验证Argo CDSSOkubernetes云计算私藏项目实操分享
0 条评论