文章目录
- 1. 什么是 Beats
- 2. Metricbeat
- 2.1 Metricbeat 组成
- 2.2 Module
- 2.3 metricbeat实战
- 2.4 Metricsets
- 2.5 Metricbeat Event
- 3. Packetbeat
- 3.1 packetbeat实战
1. 什么是 Beats
● Light weight data shippers
○ 以搜集数据为主
○ 支持与 Logstash 或 ES 集成
● 全品类 / 轻量级 / 开箱即用 / 可插拔
/ 可扩展 / 可视化
2. Metricbeat
● 用来定期搜集操作系统，软件的指标数据
○ Metric v.s Logs
■ Metric – 可聚合的数据，定期搜集
■ Logs 文本数据，随机搜集
● 指标存储在 Elasticsearch 中，可以通过 Kibana 进行实时的数据分析
下载地址：
https://www.elastic.co/cn/downloads/beats/metricbeat
2.1 Metricbeat 组成
● Module
○ 搜集的指标对象，例如不同的操作系统，不同的数据库，不同的应用系统
● Metricset
○ 一个 Module可以有多个 metricset
○ 具体的指标集合。以减少调用次数为原则进行划分
■ 不同的 metricset 可以设置不同的抓取时长
2.2 Module
● Metricbeat 提供了大量的开箱即用的 Module
○ https://www.elastic.co/guide/en/beats/metricbeat/7.1/index.html ● 通过执行 metricbeat module list 查看
● 通过执行 metricbeat moudle enable module_name 定制
2.3 metricbeat实战
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat modules list
Enabled:
system
Disabled:
aerospike
apache
aws
beat
beat-xpack
ceph
cockroachdb
consul
coredns
couchbase
couchdb
docker
dropwizard
elasticsearch
elasticsearch-xpack
envoyproxy
etcd
golang
graphite
haproxy
http
jolokia
kafka
kibana
kibana-xpack
kubernetes
kvm
logstash
logstash-xpack
memcached
mongodb
mssql
munin
mysql
nats
nginx
oracle
php_fpm
postgresql
prometheus
rabbitmq
redis
traefik
uwsgi
vsphere
windows
zookeeper
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat modules enable mysql
Enabled mysql
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat modules enable kibana
Enabled kibana
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat modules enable elasticsearch
Enabled elasticsearch
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat modules list
Enabled:
elasticsearch
kibana
mysql--
system
运行metricbeats
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
打开这个面板。发现没有数据，因为metricbeats没有运行
配置mysql面板
[root@master metricbeat-7.3.1-linux-x86_64]# vim modules.d/mysql.yml
username: root
password: 123456
运行metricbeats
[root@master metricbeat-7.3.1-linux-x86_64]# ./metricbeat运行一个测试app对mysql进行读写插入等操作
# 安装mysql
create database db_example
use db_example;
show tables;
select * from user
curl localhost:8080/demo/add -d name=Mike -d email=mike@xyz.com -d tags=Elasticsearch,IntelliJ
curl localhost:8080/demo/add -d name=Jack -d email=jack@xyz.com -d tags=Mysql,IntelliJ
curl localhost:8080/demo/add -d name=Bob -d email=bob@xyz.com -d tags=Mysql,IntelliJ
curl 'localhost:8080/demo/all'
效果图
2.4 Metricsets
● 每个 Module 都有自己的 metricsets，以 System Module 为例
○ core
○ CPU
○ disk IO
○ filesystem
○ load
○ memory
2.5 Metricbeat Event
3. Packetbeat
下载地址：https://www.elastic.co/cn/downloads/beats/packetbeat ● Packetbeat - 实时网络数据分析，监控应用服务器之间的网络流量
○ 常见抓包工具 - Tcpdump /wireshark
○ 常见抓包配置 - Pcap 基于 libpcap，跨平台 / Af_packet 仅支持 Linux，基于内存映射嗅探，高性
能
● Packetbeat 支持的协议
○ ICMP / DHCP / DNS / HTTP / Cassandra / Mysql / PostgresSQL / Redis / MongoDB / Memcache /
TLS
● Network flows：抓取记录网络流量数据，不涉及协议解析
3.1 packetbeat实战
#确保mysql监控打开，一般默认都是开的
[root@master packetbeat-7.11.2-linux-x86_64]# vim packetbeat.yml
ports: [80, 8080, 8000, 5000, 8002]
- type: memcache
# Configure the ports where to listen for memcache traffic. You can disable
# the Memcache protocol by commenting out the list of ports.
ports: [11211]
- type: mysql
# Configure the ports where to listen for MySQL traffic. You can disable
# the MySQL protocol by commenting out the list of ports.
ports: [3306,3307]
- type: pgsql
# Configure the ports where to listen for Pgsql traffic. You can disable
# the Pgsql protocol by commenting out the list of ports.
ports: [5432]
- type: redis
# Configure the ports where to listen for Redis traffic. You can disable
# the Redis protocol by commenting out the list of ports.
ports: [6379]
- type: thrift
# Configure the ports where to listen for Thrift-RPC traffic. You can disable
# the Thrift-RPC protocol by commenting out the list of ports.
ports: [9090]
- type: mongodb
# Configure the ports where to listen for MongoDB traffic. You can disable
# the MongoDB protocol by commenting out the list of ports.
ports: [27017]
- type: nfs
# Configure the ports where to listen for NFS traffic. You can disable
# the NFS protocol by commenting out the list of ports.
ports: [2049]
- type: tls
# Configure the ports where to listen for TLS traffic. You can disable
# the TLS protocol by commenting out the list of ports.
ports:
- 443 # HTTPS
- 993 # IMAPS
- 995 # POP3S
- 5223 # XMPP over SSL
- 8443
- 8883 # Secure MQTT
- 9243 # Elasticsearch
- type: sip
# Configure the ports where to listen for SIP traffic. You can disable the SIP protocol by commenting out the list of ports.
ports: [5060]
[root@master packetbeat-7.11.2-linux-x86_64]# ./packetbeat setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
[root@master packetbeat-7.11.2-linux-x86_64]# ./packetbeat
查看一些监控数据
