Jack Moore

Email: jack(at)jmoore53.com
Project Updates

Monitoring Server

13 Nov 2020 » system configuration, sysadmin, homelab, server build, monitoring

ELK on Docker

Nothing really crazy going on here, this is very similiar to the other post I have on monitoring.

This time I used a github repo to setup the stack.

Docker ELK

I used the docker elk stack from deviantony on github (see link below). I modified the stack and removed from the docker-compose.yml everything with logstash.

I had a heck of a time and still do with Logstash, so I dumped it for syslog-ng and elkstack.

So the above statement was a complete lieā€¦

I failed to realize that logstash was managing the movement of files from /var/log/remotelogs/* to the https://elastisearch:9200 instance.

Below is a logstash config I am currently using to read logs from /var/log/remotelogs/*:

Note that a bindmount is created from the host at /var/log/remotelogs/ to the container at /var/log/remotelogs.

input {
    file {
        path => "/var/log/remotelogs/*/*"
        start_position => "beginning"
        type => "syslog"
    beats {
        port => 5044

    tcp {
        port => 5000

filter {
  grok {
    match => { 'message' => '%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}' }

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
        ecs_compatibility => disabled

I finally got the filter to work for syslog after some time.. it took me a while to figure out, but I finally found it online, tinkered with it and it worked.


I also managed to break permissions between the three containers logstash, elasticsearch and kibana.

I had it working between kibana and elasticsearch, but just couldnt get logstash configured with elasticsearch.

© Jack Moore