Jack Moore

Email: jack(at)jmoore53.com
Project Updates

POC Homelab Networking

01 Nov 2020 » docker, networking, sysadmin, homelab

ELK Stack

Logging!

Elasticsearch/Kibana

docker run -d -p 9200:9200 -p 9300:9300 -it -h elasticsearch --name elasticsearch -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.9.3
docker run -d -p 5601:5601 -h kibana --link elasticsearch:elasticsearch --name kibana docker.elastic.co/kibana/kibana:7.9.3
docker run -h logstash --name logstash --link elasticsearch:elasticsearch -it --rm -v "$PWD":/config-dir docker.elastic.co/logstash/logstash:7.9.3 -f /config-dir/logstash.conf

Filebeat

Filebeat is a lightweight shipper for forwarding and centralizing log data. Installed as an agent on your servers, Filebeat monitors the log files or locations that you specify, collects log events, and forwards them either to Elasticsearch or Logstash for indexing.

The different Beats are used as lightweight agents installed on the different servers in your infrastructure for shipping logs or metrics (see diagram below). These can be log files (Filebeat), network metrics (Packetbeat), server metrics (Metricbeat), or any other type of data that can be collected by the growing number of Beats being developed by both Elastic and the community.

Right now I don’t need Logstash because I need something lightweight.

Be sure to install syslog to monitor system logs.

Filebeat Setup

Commands to Run:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.9.3-linux-x86_64.tar.gz
tar xzvf filebeat-7.9.3-linux-x86_64.tar.gz

Filebeat.yml setup

filebeat.inputs:
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*
  document_type: syslog

- type: docker
  containers.ids:
    - '*'

output.elasticsearch:
  hosts: ["localhost:9200"]

setup.kibana:
  host: "mykibanahost:5601" 
  username: "my_kibana_user"  
  password: "{pwd}"

After configuring filebeat.yml:

./filebeat modules list
./filebeat modules enable system nginx docker
./filebeat setup -e

Specify a user who is authorized to publish events:

sudo chown root filebeat.yml 
sudo chown root modules.d/system.yml 
sudo ./filebeat -e

Running filebeat setup -e may throw errors about permissions and ownership of modules. As a quick hacky fix in this POC I just chown’d everything to root to get it workign. (oops.)

Filebeat as a Service

By default filebeat just runs as a script on the server. This is filebeat defined as a service.

[Unit]
Description=Filebeat Monitoring Service
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/home/jack/Documents/elk-test/filebeat-7.9.3-linux-x86_64/filebeat -c /home/jack/Documents/elk-test/filebeat-7.9.3-linux-x86_64/filebeat.yml

[Install]
WantedBy=multi-user.target

Configure Filebeat with Logstash

Logstash’s main feature is its ability to Transform logs for input/output.

Here’s an exmaple config for Filebeat to Logstash with logstash running at port 5044:

filebeat.inputs:
- input_type: log
  paths:
    - /var/log/httpd/access.log

document_type: apache-access
fields_under_root: true

output.logstash:
  hosts: ["127.0.0.1:5044"]
input {
  beats {
    port => 5044
  }
 }

filter {
  grok {
    match => { "message" => "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:time}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)" }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
 }

output {
  elasticsearch { hosts => ["localhost:9200"] }
}

Setting up Authentication with Elasticsearch/Kibana

© Jack Moore