0
点赞
收藏
分享

微信扫一扫

安装ELK-Docker方式

跟着Damon写代码 2022-08-02 阅读 95
ELKelasticsearchlogstashkibanadocker虚拟化云计算


目录

​​ELK介绍​​

​​安装步骤​​

​​docker环境准备​​

​​先启动docker服务​​

​​拉取镜像​​

​​启动镜像​​

​​再次重新启动​​

​​测试​​

ELK介绍

“ELK”是三个开源项目的首字母缩写，这三个项目分别是：Elasticsearch、Logstash 和 Kibana。Elasticsearch 是一个搜索和分析引擎。Logstash 是服务器端数据处理管道，能够同时从多个来源采集数据，转换数据，然后将数据发送到诸如 Elasticsearch 等“存储库”中。Kibana 则可以让用户在 Elasticsearch 中使用图形和图表对数据进行可视化。

安装步骤

docker环境准备

这个比较简单，这里就不详细介绍，如果大家感兴趣的，可以翻阅我之前的文章，里面应该有相关的介绍文章。

我自己有一台服务器，是专门用于做研究测试的，已经有现成docker环境，所以直接用就可以了。

先查看一下自己服务器里的docker版本信息。

[root@localhost ~]# docker --version
Docker version 19.03.12, build 48a66213fe

先启动docker服务

systemctl start docker

拉取镜像

docker pull sebp/elk

然后漫长等待。。。

安装ELK-Docker方式_elasticsearch

确实很慢，所以决定睡一觉。。。

睡醒起来一看，终于拉取完。。。

安装ELK-Docker方式_ELK_02

启动镜像

docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name elkdemo sebp/elk

但出现报错情况：

Last login: Mon Mar 15 23:14:26 2021 from 10.18.0.13
[root@localhost ~]# docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name elkdemo sebp/elk
 * Starting periodic command scheduler cron                              [ OK ] 
 * Starting Elasticsearch Server                                         [ OK ] 
waiting for Elasticsearch to be up (1/30)
waiting for Elasticsearch to be up (2/30)
waiting for Elasticsearch to be up (3/30)
waiting for Elasticsearch to be up (4/30)
waiting for Elasticsearch to be up (5/30)
waiting for Elasticsearch to be up (6/30)
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
ERROR: Elasticsearch did not exit normally - check the logs at /var/log/elasticsearch/elasticsearch.log
waiting for Elasticsearch to be up (7/30)
waiting for Elasticsearch to be up (8/30)
waiting for Elasticsearch to be up (9/30)
waiting for Elasticsearch to be up (10/30)
waiting for Elasticsearch to be up (11/30)
waiting for Elasticsearch to be up (12/30)
waiting for Elasticsearch to be up (13/30)
waiting for Elasticsearch to be up (14/30)
waiting for Elasticsearch to be up (15/30)
waiting for Elasticsearch to be up (16/30)
waiting for Elasticsearch to be up (17/30)
waiting for Elasticsearch to be up (18/30)
waiting for Elasticsearch to be up (19/30)
waiting for Elasticsearch to be up (20/30)
waiting for Elasticsearch to be up (21/30)
waiting for Elasticsearch to be up (22/30)
waiting for Elasticsearch to be up (23/30)
waiting for Elasticsearch to be up (24/30)
waiting for Elasticsearch to be up (25/30)
waiting for Elasticsearch to be up (26/30)
waiting for Elasticsearch to be up (27/30)
waiting for Elasticsearch to be up (28/30)
waiting for Elasticsearch to be up (29/30)
waiting for Elasticsearch to be up (30/30)
Couldn't start Elasticsearch. Exiting.
Elasticsearch log follows below.
[2021-03-16T01:47:54,076][INFO ][o.e.n.Node               ] [elk] version[7.11.1], pid[208], build[default/tar/ff17057114c2199c9c1bbecc727003a907c0db7a/2021-02-15T13:44:09.394032Z], OS[Linux/3.10.0-862.el7.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2021-03-16T01:47:54,078][INFO ][o.e.n.Node               ] [elk] JVM home [/opt/elasticsearch/jdk], using bundled JDK [true]
[2021-03-16T01:47:54,078][INFO ][o.e.n.Node               ] [elk] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-7079508974677256554, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms4918m, -Xmx4918m, -XX:MaxDirectMemorySize=2579496960, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.path.home=/opt/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true]
[2021-03-16T01:47:55,523][INFO ][o.e.p.PluginsService     ] [elk] loaded module [aggs-matrix-stats]
[2021-03-16T01:47:55,523][INFO ][o.e.p.PluginsService     ] [elk] loaded module [analysis-common]
[2021-03-16T01:47:55,523][INFO ][o.e.p.PluginsService     ] [elk] loaded module [constant-keyword]
[2021-03-16T01:47:55,523][INFO ][o.e.p.PluginsService     ] [elk] loaded module [flattened]
[2021-03-16T01:47:55,523][INFO ][o.e.p.PluginsService     ] [elk] loaded module [frozen-indices]
[2021-03-16T01:47:55,523][INFO ][o.e.p.PluginsService     ] [elk] loaded module [ingest-common]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [ingest-geoip]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [ingest-user-agent]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [kibana]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [lang-expression]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [lang-mustache]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [lang-painless]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [mapper-extras]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [mapper-version]
[2021-03-16T01:47:55,524][INFO ][o.e.p.PluginsService     ] [elk] loaded module [parent-join]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [percolator]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [rank-eval]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [reindex]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [repositories-metering-api]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [repository-url]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [search-business-rules]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [searchable-snapshots]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [spatial]
[2021-03-16T01:47:55,525][INFO ][o.e.p.PluginsService     ] [elk] loaded module [transform]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [transport-netty4]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [unsigned-long]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [vectors]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [wildcard]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-aggregate-metric]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-analytics]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-async]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-async-search]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-autoscaling]
[2021-03-16T01:47:55,526][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-ccr]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-core]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-data-streams]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-deprecation]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-enrich]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-eql]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-fleet]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-graph]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-identity-provider]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-ilm]
[2021-03-16T01:47:55,527][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-ingest]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-logstash]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-ml]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-monitoring]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-ql]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-rollup]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-runtime-fields]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-security]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-sql]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-stack]
[2021-03-16T01:47:55,528][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-voting-only-node]
[2021-03-16T01:47:55,529][INFO ][o.e.p.PluginsService     ] [elk] loaded module [x-pack-watcher]
[2021-03-16T01:47:55,529][INFO ][o.e.p.PluginsService     ] [elk] no plugins loaded
[2021-03-16T01:47:55,561][INFO ][o.e.e.NodeEnvironment    ] [elk] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/mapper/vg01-lv_root)]], net usable_space [289.6gb], net total_space [299.3gb], types [xfs]
[2021-03-16T01:47:55,562][INFO ][o.e.e.NodeEnvironment    ] [elk] heap size [4.8gb], compressed ordinary object pointers [true]
[2021-03-16T01:47:55,586][INFO ][o.e.n.Node               ] [elk] node name [elk], node ID [gR6iR2QgT727xonm7n8PpA], cluster name [elasticsearch], roles [transform, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]
[2021-03-16T01:47:58,099][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [elk] [controller/237] [Main.cc@117] controller (64 bit): Version 7.11.1 (Build b7aec245e3d54f) Copyright (c) 2021 Elasticsearch BV
[2021-03-16T01:47:59,149][INFO ][o.e.t.NettyAllocator     ] [elk] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2021-03-16T01:47:59,197][INFO ][o.e.d.DiscoveryModule    ] [elk] using discovery type [zen] and seed hosts providers [settings]
[2021-03-16T01:47:59,498][INFO ][o.e.g.DanglingIndicesState] [elk] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2021-03-16T01:47:59,787][INFO ][o.e.n.Node               ] [elk] initialized
[2021-03-16T01:47:59,788][INFO ][o.e.n.Node               ] [elk] starting ...
[2021-03-16T01:47:59,953][INFO ][o.e.x.s.c.PersistentCache] [elk] persistent cache index loaded
[2021-03-16T01:48:00,027][INFO ][o.e.t.TransportService   ] [elk] publish_address {172.17.0.2:9300}, bound_addresses {0.0.0.0:9300}
[2021-03-16T01:48:00,240][INFO ][o.e.b.BootstrapChecks    ] [elk] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2021-03-16T01:48:00,242][ERROR][o.e.b.Bootstrap          ] [elk] node validation exception
[1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2021-03-16T01:48:00,244][INFO ][o.e.n.Node               ] [elk] stopping ...
[2021-03-16T01:48:00,250][INFO ][o.e.n.Node               ] [elk] stopped
[2021-03-16T01:48:00,250][INFO ][o.e.n.Node               ] [elk] closing ...
[2021-03-16T01:48:00,257][INFO ][o.e.n.Node               ] [elk] closed

从日志信息来看，应该是Elasticsearch的内存太小，需要我们重新设置一下。

执行以下命令：

vi /etc/sysctl.conf

在配置文件里添加以下配置信息：

vm.max_map_count=262144
sysctl -p

安装ELK-Docker方式_kibana_03

其中“sysctl -p”必须加上，否则会没效。

再次重新启动

在启动之前，先把之前的容器进程信息干掉。

可执行以下命令，先获取对应的容器id信息，然后再删除。

docker ps -a

结果：

[root@localhost ~]# docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS                               NAMES
c56b10216542        sebp/elk            "/usr/local/bin/star…"   59 minutes ago      Exited (1) 59 minutes ago                                       elkdemo
13bc33de0dee        mysql:8.0           "docker-entrypoint.s…"   5 months ago        Exited (0) 7 weeks ago                                          mysql
63084f7ad6fb        tomcat              "catalina.sh run"        6 months ago        Exited (130) 6 months ago                                       tomcat
ada355786110        mysql               "docker-entrypoint.s…"   6 months ago        Exited (137) 5 months ago                                       mysql1
543f9a143110        mysql               "docker-entrypoint.s…"   6 months ago        Exited (1) 6 months ago                                         mysql-docker2
4240d8d8183c        mysql               "docker-entrypoint.s…"   6 months ago        Created                     0.0.0.0:3306->3306/tcp, 33060/tcp   mysql-docker1
d471b2e8dc3c        mysql               "docker-entrypoint.s…"   6 months ago        Exited (137) 6 months ago                                       mysql-docker
377a1391fdc8        hello-world         "/hello"                 6 months ago        Exited (0) 6 months ago                                         blissful_merkle

因为我自己的服务器已经运行了其它docker容器，所以我们只需找到我们之前执行命令的容器名字以及对应的容器ID。

即，c56b10216542：

安装ELK-Docker方式_ELK_04

删除这个容器：

docker rm c56b10216542

最后重新启动。

docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name elkdemo sebp/elk

启动后，没发现报错，我们就先登录界面看看。

​​http://10.18.0.57:5601/​​

出现以下界面：

安装ELK-Docker方式_logstash_05

感觉像成功了。

测试

安装好后，这个是一个“裸”的ELK，没有任何日志数据，无法体验报表功能。我们可以添加一些样例数据去体验一下。

在首页（http://10.18.0.57:5601/app/home#/），选择“Add data”：

安装ELK-Docker方式_logstash_06

选择“Sample data”：

安装ELK-Docker方式_logstash_07

我们先随意挑一个样例数据看看效果：

安装ELK-Docker方式_docker_08

添加成功后，这里的状态信息会变成这样：

安装ELK-Docker方式_logstash_09

我们选择“Dashboard”方式看看：

安装ELK-Docker方式_ELK_10

证明数据已成功导入到我们的ELK平台：

安装ELK-Docker方式_elasticsearch_11

举报

相关推荐

docker 安装 ELK

dockerubuntulinux运维

ELK docker 安装logstash

bcmysql字段JavaScript前端开发

安装ELK（Docker Compose）

elasticsearchdocker配置文件运维yyds干货盘点

Docker 安装ELK7.7.1

docker运维

docker-compose安装ELK

ELKFDocker

Docker 安装 ELK，EFK代替

elasticsearchdockerElasticOpenStack云计算

elk+ filebeat环境部署(docker 方式)

nginxdockerhtml虚拟化云计算

docker安装ELK 收集springboot日志

dockerelk编程语言

Docker——ELK

centos运维笔记dockerIT必备技能ELKDocker

ELK - docker

运维
0 条评论