0
点赞
收藏
分享

微信扫一扫

Loki实现Harbor日志的高效管理

半夜放水 2022-07-12 阅读 64
云原生云计算运维Harborkubernetesyyds干货盘点


前言

在Harbor的

使用Loki实现Harbor日志的管理，在Grafana页面查询Harbor的日志。


环境：

已通过Helm在Kubernetes集群中安装Harbor，Harbor服务部署在Harbor命名空间中。

Helm部署Harbor的文档请阅读：

Helm部署Harbor,实现高可用的镜像仓库（超详细分享）~后附踩坑记录

​​https://blog.51cto.com/lidabai/5195706​​

[root@master1 harbor]# kubectl -n harbor  get  pods
NAME                                    READY   STATUS    RESTARTS   AGE
harbor-chartmuseum-68f68d88ff-nq97s     1/1     Running   0          2m52s
harbor-core-7565bfb9cc-w5k29            1/1     Running   0          2m52s
harbor-database-0                       1/1     Running   0          2m52s
harbor-jobservice-6747b486df-vm9zs      1/1     Running   0          2m52s
harbor-nginx-956fff988-r8dpx            1/1     Running   0          2m52s
harbor-notary-server-776f85f9c6-4nlsn   1/1     Running   0          2m52s
harbor-notary-signer-7f895d59d6-k5957   1/1     Running   0          2m52s
harbor-portal-5766b784c7-sndmz          1/1     Running   0          2m52s
harbor-redis-0                          1/1     Running   0          2m52s
harbor-registry-6b88cfb465-fcxvn        2/2     Running   0          2m52s
harbor-trivy-0                          1/1     Running   0          2m52s
[root@master1 harbor]# kubectl -n harbor  get  svc
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE
harbor                 NodePort    10.98.71.137     <none>        80:30002/TCP,4443:30004/TCP   3m5s
harbor-chartmuseum     ClusterIP   10.102.194.148   <none>        80/TCP                        3m5s
harbor-core            ClusterIP   10.100.87.174    <none>        80/TCP                        3m5s
harbor-database        ClusterIP   10.100.79.72     <none>        5432/TCP                      3m5s
harbor-jobservice      ClusterIP   10.111.33.230    <none>        80/TCP                        3m5s
harbor-notary-server   ClusterIP   10.97.144.222    <none>        4443/TCP                      3m5s
harbor-notary-signer   ClusterIP   10.110.131.62    <none>        7899/TCP                      3m5s
harbor-portal          ClusterIP   10.105.213.145   <none>        80/TCP                        3m5s
harbor-redis           ClusterIP   10.110.18.210    <none>        6379/TCP                      3m5s
harbor-registry        ClusterIP   10.110.129.16    <none>        5000/TCP,8080/TCP             3m5s
harbor-trivy           ClusterIP   10.109.0.155     <none>        8080/TCP                      3m5s

Loki实现Harbor日志的高效管理_云原生

Loki简介

Loki是 Grafana Labs 团队最新的开源项目，是一个水平可扩展，高可用性，多租户的日志聚合系统。它的设计非常经济高效且易于操作，因为它不会为日志内容编制索引，而是为每个日志流编制一组标签。

 

与其他日志系统不同，Loki 是围绕仅索引有关日志的元数据的想法构建的：标签（就像 Prometheus 标签一样）。然后，日志数据本身会被压缩并以块的形式存储在对象存储（例如 S3 或 GCS）中，甚至本地存储在文件系统中。小索引和高度压缩的块简化了操作并显着降低了 Loki 的成本。


使用Helm部署Loki日志平台


环境准备

1)创建命名空间

[root@lidabai-master1 ~]#  kubectl create namespace loki

2）安装Helm

    如果你已经安装好helm了，可以忽略本步骤。

[root@lidabai-master1 ~]# wget  https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[root@lidabai-master1 ~]#  tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[root@lidabai-master1 ~]#  cp linux-amd64/helm  /usr/local/bin/
[root@lidabai-master1 ~]#  helm versio
version.BuildInfo{Version:"v3.7.2", GitCommit:"663a896f4a815053445eec4153677ddc24a0a361", 
GitTreeState:"clean", GoVersion:"go1.16.10"}

3）添加Chart仓库

[root@lidabai-master1 ~]#  helm repo add grafana ​​https://grafana.github.io/helm-charts
[root@lidabai-master1 ~]#  helm repo list
NAME      URL
grafana     https://grafana.github.io/helm-charts
[root@lidabai-master1 ~]#  helm repo update


4)搜索chart

[root@master1 ~]# helm search repo grafana
NAME                                  CHART VERSION  APP VERSION  DESCRIPTION                                       
grafana/grafana                       6.32.1         9.0.2        The leading tool for querying and visualizing t...
grafana/grafana-agent-operator        0.2.2          0.25.1       A Helm chart for Grafana Agent Operator           
grafana/enterprise-logs               2.2.2          v1.4.1       Grafana Enterprise Logs                           
grafana/enterprise-logs-simple        1.2.0          v1.4.0       DEPRECATED Grafana Enterprise Logs (Simple Scal...
grafana/enterprise-metrics            1.9.0          v1.7.0       DEPRECATED Grafana Enterprise Metrics             
grafana/fluent-bit                    2.3.1          v2.1.0       Uses fluent-bit Loki go plugin for gathering lo...
grafana/loki                          2.12.2         v2.5.0       Loki: like Prometheus, but for logs.              
grafana/loki-canary                   0.8.1          2.5.0        Helm chart for Grafana Loki Canary                
grafana/loki-distributed              0.50.0         2.5.0        Helm chart for Grafana Loki in microservices mode 
grafana/loki-simple-scalable          1.6.1          2.5.0        Helm chart for Grafana Loki in simple, scalable...
grafana/loki-stack                    2.6.5          v2.4.2       Loki: like Prometheus, but for logs.              
grafana/mimir-distributed             2.1.0          2.1.0        Grafana Mimir                                     
grafana/mimir-openshift-experimental  2.1.0          2.0.0        Grafana Mimir on OpenShift Experiment             
grafana/oncall                        1.0.2          v1.0.3       Developer-friendly incident response with brill...
grafana/promtail                      6.0.2          2.5.0        Promtail is an agent which ships the contents o...
grafana/rollout-operator              0.1.2          v0.1.1       Grafana rollout-operator                          
grafana/tempo                         0.15.4         1.4.1        Grafana Tempo Single Binary Mode                  
grafana/tempo-distributed             0.20.3         1.4.1        Grafana Tempo in MicroService mode                
grafana/tempo-vulture                 0.2.0          1.3.0        Grafana Tempo Vulture - A tool to monitor Tempo...

Loki实现Harbor日志的高效管理_Harbor_02

 grafana/loki-stack: 将多个进程封装到一个Pod内，未做数据持久化，适用于测试环境,单体模式。

 grafana/loki-canary: 金丝雀更新模式；

 grafana/loki-distributed:  微服务模式，适合生产较大规模场景；

 grafana/loki-simple-scalable: 读写分离模式，简单可扩展；


下载并安装Loki Chart

  • 下载解压chart

[root@master1 ~]# helm pull  grafana/loki-stack  --untar
[root@master1 ~]# cd loki-stack/
[root@master1 loki-stack]# ls
charts  Chart.yaml  README.md  requirements.lock  requirements.yaml  templates  values.yaml

--untar：下载并解压Chart包；

  • 修改values.yaml配置

[root@master1 loki-stack]# vim values.yaml 
grafana:
  enabled: true    #启用Grafana组件
  sidecar:
    datasources:
      enabled: true
      maxLines: 1000
  image:
    tag: 8.3.5


  • 安装

[root@master1 loki-stack]# helm install loki-stack . -n loki
NAME: loki-stack
LAST DEPLOYED: Thu Jul  7 13:31:02 2022
NAMESPACE: loki
STATUS: deployed
REVISION: 1
NOTES:
The Loki stack has been deployed to your cluster.
Loki can now be added as a datasource in Grafana.
See http://docs.grafana.org/features/datasources/loki/ for more detail.
[root@master1 loki-stack]# helm -n loki ls

Loki实现Harbor日志的高效管理_Harbor_03


查看验证服务

[root@master1 loki-stack]# kubectl -n loki  get  pods
NAME                                  READY   STATUS    RESTARTS   AGE
loki-stack-0                          1/1     Running   0          74s
loki-stack-grafana-69966bcf94-lp4z5   2/2     Running   0          74s
loki-stack-promtail-ckpsh             1/1     Running   0          74s
loki-stack-promtail-g7kft             1/1     Running   0          74s
loki-stack-promtail-gf86b             1/1     Running   0          74s
loki-stack-promtail-jnznc             1/1     Running   0          74s
nfs-provisioner-56fd47fc9c-lktrv      1/1     Running   3          23h
[root@master1 loki-stack]# kubectl -n loki  get  svc
NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
loki-stack              ClusterIP   10.102.81.250   <none>        3100/TCP   76s
loki-stack-grafana      ClusterIP   10.103.62.31    <none>        80/TCP     76s
loki-stack-headless     ClusterIP   None            <none>        3100/TCP   76s
loki-stack-memberlist   ClusterIP   None            <none>        7946/TCP   76s

Pod都部署成功！

  • 修改Grafana服务访问方式为NodePort

[root@master1 loki-stack]# kubectl -n loki  edit  svc loki-stack-grafana
spec:
  type: NodePort

Loki实现Harbor日志的高效管理_云计算_04


Grafana配置

登录Grafana​

[root@master1 loki-stack]# kubectl -n loki  get  svc loki-stack-grafana 
NAME                 TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
loki-stack-grafana   NodePort   10.103.62.31   <none>        80:32555/TCP   4m9s

查看Grafana密码

$ kubectl -n loki get secret loki-stack-grafana  -o jsonpath="{.data.admin-password}" | base64  --decode ; echo
fPPZ19cAnP0J7MYPiU4vkWIcMDlbxlgGJYe4jKYT

Loki实现Harbor日志的高效管理_运维_05

浏览器登录Grafana

  • 登录地址： k8s节点IP:NodePort
  • 用户名：admin
  • 密码： 刚才查看到的secret解析串

Loki实现Harbor日志的高效管理_云计算_06


Grafana查看Loki日志

Loki实现Harbor日志的高效管理_云原生_07

然后在输入： {namespace="harbor"} 即可查看Harbor命名空间的所有日志。

Loki实现Harbor日志的高效管理_运维_08



举报

相关推荐

使用 OpenTelemetry 和 Loki 实现高效的应用日志采集和分析

云原生Kubernetes云平台云计算

【日志】Loki

数据公众号日志系统虚拟化云计算

基于Grafana Loki的日志报警架构实现

数据源docker数据架构后端开发

loki-日志

loki日志kubernetes

Nginx loki监控日志

nginxlinuxHTTP办公效率

Nginx loki监控日志的学习

nginxlinuxHTTPJavaScript前端开发

windows实现Grafana+Loki+loki4j轻量级日志系统，告别沉重的ELK

python自动化数据库

Loki配置采集haproxy日志

运维大数据

轻量化日志系统 Loki

Windowsapachelinux办公效率

云原生日志Loki

前端vscodevue
0 条评论