0
点赞
收藏
分享

微信扫一扫

部署logstash收集daemonset filebeat发过去的kafka日志

实现:部署如下图红框 位置logstash

架构:


主机:10.0.7.56

部署logstash

yum -y install java-1.8.0-openjdk
yum -y install https://mirror.tuna.tsinghua.edu.cn/elasticstack/7.x/yum/7.9.0/logstash-7.9.0.rpm
systemctl start logstash
systemctl enable logstash

创建配置文件

从kafka接收数据,发送到es

vim /etc/logstash/conf.d/daemonset-filebeat-kafka-to-es.conf
input {
kafka {
bootstrap_servers => "10.0.7.53:9092,10.0.7.54:9092,10.0.7.55:9092"
topics => ["daemonset-pod-console-log"]
codec => json {
charset => "UTF-8"
}
}
}

output {
if [fields][log_topic] == "daemonset-pod-console-log" {
elasticsearch {
hosts => ["10.0.7.46:9200","10.0.7.47:9200","10.0.7.48:9200"]
index => "daemonset-pod-console-log-%{+YYYY.MM.dd}"
}}
}

启动并查看日志

systemctl restart logstash
tail -f /var/log/logstash/logstash-plain.log

举报

相关推荐

0 条评论