Jack Moore

Email: jack(at)jmoore53.com
Project Updates

Moving away from Netlifly

26 Dec 2021 » web, nginx, html, configuration, logging

I’m more than a bit upset to now find out analytics for the netlifly platform costs $9/month. I was pretty close to pulling the trigger but decided against it as I don’t want to deal with the hassle of cancelling.

This means I will be moving the site to a Digital Ocean Droplet and deploy via NGINX/Jekyll and using automatic deploys via what will most likely be easy to understand and deploy infrastructure (Gitlab CI/CD & something server side cron job and bash to pick up the build every night for new posts).

Nginx

# Installation of nginx on the host
apt update && apt install nginx

And NGINX just picks up /var/log/www/html where I am generating the static site.

Build the Site Every Night

I should have done this entirely via bash, however I didn’t. I completed this via a dockerfile and some bash scripts because I didn’t want to deal with Ruby on the host.

The Dockerfile looks like the following:

FROM ruby:2.3-alpine

RUN apk update && apk add \
    git \
    build-base

WORKDIR /app

RUN git clone https://gitlab.com/jmoore53/jmoore53.com.git

WORKDIR /app/jmoore53.com

RUN bundle install

ENTRYPOINT bundle exec jekyll build -d /public

The cronjob which runs on a 0 4 * * * schedule (everyday at 4 am) looks like the following:

#!/bin/bash

docker image rm -f jmoore53-blog
docker build -t jmoore53-blog - < /opt/blog/Dockerfile
docker run --rm -v /var/www/html:/public jmoore53-blog
systemctl reload nginx

goaccess html reports

Goaccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly.

cat access.log | docker run --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format COMBINED - > report.html

To get even fancier with all the logs that have been compressed (use the -f force flag on the zcat to cat the normal .log files):

zcat -f access.log.* | docker run --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format COMBINED - > report.html

Taking it one more step further with grep (replacing public ip address with the ip address of your house/how you access the site):

zcat -f access.log.* | grep -v "my.public.ip.address" | docker run --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format COMBINED - > report.html

Then I used nginx docker image to deploy the report.html. This is what I used to run the webserver:

docker run --rm -v /opt/jekyll-site/public:/var/www/html -p 8080:80 nginx

Then in /opt/jekyll-site/public I added the report.html which was then renamed to index.html.

One Step Further, CRON

I wanted to not have to run this process manually, one might say I wanted to “automate” it. So here’s the hourly go:

Cron entry to run hourly:


0 * * * * /opt/jekyll-site/stats.sh

The script at /opt/jekyll-site/stats.sh

#!/bin/bash

# rsync down data from web server
rsync -avP user@jmoore53.com:/var/log/nginx/access.* /opt/jekyll-site/data/

# zcat data from log files into the index file
zcat -f /opt/jekyll-site/data/access.log.* | docker run --rm -i -e LANG=$LANG allinurl/goaccess -a -o html --log-format COMBINED - > /opt/jekyll-site/public/index.html

Now I am able to hit 10.0.5.10:8080 in my web browser and it will show the updated report from the latest hour.

© Jack Moore - This site was last built Fri 30 Aug 2024 12:31:24 PM EDT