ELK on Docker
Nothing really crazy going on here, this is very similiar to the other post I have on monitoring.
This time I used a github repo to setup the stack.
Docker ELK
I used the docker elk stack from deviantony on github (see link below). I modified the stack and removed from the docker-compose.yml everything with logstash.
I had a heck of a time and still do with Logstash, so I dumped it for syslog-ng and elkstack.
So the above statement was a complete lie…
I failed to realize that logstash was managing the movement of files from /var/log/remotelogs/* to the https://elastisearch:9200 instance.
Below is a logstash config I am currently using to read logs from /var/log/remotelogs/*:
Note that a bindmount is created from the host at /var/log/remotelogs/ to the container at /var/log/remotelogs.
input {
file {
path => "/var/log/remotelogs/*/*"
start_position => "beginning"
type => "syslog"
}
beats {
port => 5044
}
tcp {
port => 5000
}
}
filter {
grok {
match => { 'message' => '%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}' }
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
ecs_compatibility => disabled
}
}
I finally got the filter to work for syslog after some time.. it took me a while to figure out, but I finally found it online, tinkered with it and it worked.
Permissions
I also managed to break permissions between the three containers logstash, elasticsearch and kibana.
I had it working between kibana and elasticsearch, but just couldnt get logstash configured with elasticsearch.