elk之[logstash-input-file]插件使用详解

https://www.elastic.co/guide/en/logstash/current/index.html    官方文档

一、安装配置kibana

5.1 下载解压缩

 [admin@node21 elk]$ wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-linux-x86_64.tar.gz

[admin@node21 elk]$ tar -xzf kibana-6.2.4-linux-x86_64.tar.gz

5.2 修改config/kibana.yml

[admin@node21 elk]$ vi kibana-6.2.4-linux-x86_64/config/kibana.yml 
#server.host: "localhost"
server.host: "192.168.100.21"   #设置自己机器的IP
#elasticsearch.url: "http://localhost:9200"
elasticsearch.url: "http://192.168.100.21:9200"

5.3 启动Kibana

进入kibana/bin/目录

[admin@node21 bin]$ ./kibana &

页面访问：192.168.100.21:5601

1、logstash 收集多个系统日志及换行设置  注意空格

[root@tes datas]# cat /opt/datas/logstash-test-if.conf
input {
  file {
    path => "/opt/datas/test.txt"
    type => "system"
    start_position => "beginning"
    sincedb_path => "/dev/null"
     }
 
  file {
    path => "/var/log/elk/my-application.log"
    type => "system-message"
    start_position => "beginning"
    codec => multiline {
      pattern => "^\["
      negate => true
      what => "previous"
      }
    }
}
output {
  if [type] == "system" {
    elasticsearch {
      hosts => ["100.16.3.108:9200"]
      index => "system-%{+YYYY.MM.dd}"
        }
    } 
  if [type] == "system-message" {
    elasticsearch {
      hosts => ["100.16.3.108:9200"]
      index => "system-message%{+YYYY.MM.dd}"
        }
    }
}、logstash处理日志追加    （打开源文件，然后手动追加，会翻倍复制原文本，如果在文本外面用echo >> 追加就不会）
[root@test ~]# cat /opt/datas/file.conf 
input {
  file {
    path => "/root/test.txt"
    type => "test"
    start_position => "end"
    sincedb_path => "/dev/null" 
     }}
output {
  if [type] == "test" {
    elasticsearch {
      hosts => ["192.168.33.118:9200"]
      index => "test-%{+YYYY.MM.dd}"
        }
    }}