From c84e37ec3fe6da4054ee92e948c172692f5a25d3 Mon Sep 17 00:00:00 2001 From: Tobias Date: Sun, 4 Oct 2026 22:34:30 +0000 Subject: [PATCH] Add README.md --- README.md | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..a85e1d2 --- /dev/null +++ b/README.md @@ -0,0 +1,78 @@ +## Overview + +For my personal self-hosted infrastructure I found the typical Prometheus and Grafana set up to be too bloated, involving too much network traffic and overhead, for what I actually want to monitor + +Consequently I decided to build my own lightweight tooling with Python, Linux commands, SSH, Cron, JSON, HTML/CSS and Javascript + +It is capable of sending offline/healing alerts via email, as well as scraping relevant metrics, (CPU, RAM, Disk Space etc), to display on a centralized webpage + +The metrics page, as well as the generated JSON files and Python script is hosted on my [Apache-Intranet LXC Container](https://git.tobiastime.xyz/Tobias/Apache-LXC-Intranet) + +## High Level Architecture Overview + +``` +Cron starts scraper.py every 15 minutes + ↓ + For each monitored server + ↓ + Attempt to reach via ICMP + / \ + SUCCESS FAILURE + ↓ ↓ +Check previous state Check previous state + ↓ ↓ + Previously offline? Previously online? + / \ / \ + YES NO YES NO + ↓ ↓ ↓ ↓ +Send "back Continue Send Continue +online" email "offline" email + \ / \ / + └───────┬───────────────┴──────────┘ + ↓ + If server is pingable + ↓ + Connect via dedicated SSH key + as restricted user + ↓ + Run collector.py to scrape metrics + ↓ + Collector generates JSON + ↓ + Copy JSON back via SFTP + ↓ +Store in appropriate Apache server directory + ↓ + index.html reads JSON + ↓ + Metrics displayed in web portal + ↓ +Save current online/offline server state to JSON +``` + +## Monitoring Failover + +I have a secondary Debian LXC container on a separate hypervisor host to provide monitoring redundancy + +It is scheduled via Cron to attempt to ping Apache-Intranet every 5 minutes + +If Apache-Intranet, (the primary), is unreachable it takes over monitoring/alerting duties, otherwise it lies dormant + +Its purpose is to provide email alerts to me informing that we are currently in the midst of an outage and which servers are offline + +``` +┌──────────────────────────────────────────┐ +│ Secondary Monitoring LXC │ +│ Runs ping.py every 5 minutes which │ +│ checks Apache-Intranet's health via ICMP │ +└──────────────────────────────────────────┘ + │ + Is primary reachable? + / \ + YES NO + ↓ ↓ + Do nothing Take over + ↓ + Begin monitoring servers' reachability + and sending alerts +``` \ No newline at end of file