“THANOS” — Monitoring with Prometheus and Grafana

Baseline Monitoring Setup with Prometheus and Grafana

Thanos

Why Integrate Prometheus with Thanos?

  • Storing historical data in a reliable and cost-efficient way
  • Accessing all metrics, whether they are new or old using a single-query API
  • Merging replicated data collected via Prometheus high-availability (HA) setups

Thanos Components Architecture Overview

Image from Thanos Github
Block Diagram of Prometheus with Thanos

Deployment Overview

prometheus0:
image: prom/prometheus:v2.9.2
container_name: prometheus0
user: root
volumes:
- thanos0:/data
- ./data/prom0/prometheus.yml:/etc/prometheus/prometheus.yml
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/data/prom0"
- "--web.enable-lifecycle"
- "--storage.tsdb.min-block-duration=2h"
- "--storage.tsdb.max-block-duration=2h"
- "--web.listen-address=0.0.0.0:9090"
networks:
- thanos
sidecar0:
image: thanosio/thanos:v0.10.0
container_name: thanos-sidecar0
command:
- "sidecar"
- "--debug.name=sidecar-0"
- "--grpc-address=0.0.0.0:10901"
- "--http-address=0.0.0.0:10902"
- "--prometheus.url=http://prometheus0:9090"
- "--tsdb.path=/data/prom0"
- "--objstore.config-file=/bucket.yml"
volumes:
- thanos0:/data
- ./data/bucket.yml:/bucket.yml
depends_on:
- prometheus0
networks:
- thanos
query0:
image: thanosio/thanos:v0.10.0
container_name: thanos-query0
command:
- "query"
- "--grpc-address=0.0.0.0:10903"
- "--http-address=0.0.0.0:10904"
- "--query.replica-label=prometheus"
- "--store=sidecar0:10901"
- "--store=store:10905"
ports:
- 10904:10904
depends_on:
- sidecar0
- store
networks:
- thanos
store:
image: thanosio/thanos:v0.10.0
container_name: thanos-store
restart: always
command:
- "store"
- "--grpc-address=0.0.0.0:10905"
- "--http-address=0.0.0.0:10906"
- "--data-dir=/data/store"
- "--objstore.config-file=/bucket.yml"
volumes:
- store:/data
- ./data/bucket.yml:/bucket.yml
networks:
- thanos
compactor:
image: thanosio/thanos:v0.10.0
container_name: compactor
command:
- "compact"
- "--wait"
- "--data-dir=/tmp/thanos-compact"
- "--objstore.config-file=/bucket.yml"
- "--http-address=0.0.0.0:10902"
volumes:
- compact:/tmp
- ./data/bucket.yml:/bucket.yml
depends_on:
- sidecar0
- store
networks:
- thanos
node-exporter:
image: prom/node-exporter:v0.18.1
container_name: node-exporter
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- '--path.procfs=/host/proc'
- '--path.rootfs=/rootfs'
- '--path.sysfs=/host/sys'
- '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
ports:
- 9100:9100
networks:
- thanos
grafana:
image: grafana/grafana:6.5.2
container_name: grafana
environment:
- GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}
- GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}
- GF_USERS_ALLOW_SIGN_UP=false
restart: unless-stopped
ports:
- 3000:3000
networks:
- thanos
networks:
thanos: {}
volumes:
thanos0: {}
store: {}
compact: {}
- DevOps
|---docker-compose.yml
|---data
|---bucket.yml
|---prom0
|---prometheus.yml
type: FILESYSTEM
config:
directory: “/data/prom0”
global:
external_labels:
prometheus: prom-0
scrape_configs:
- job_name: prometheus
scrape_interval: 5s
static_configs:
- targets:
- "localhost:9090"
- job_name: 'nodeexporter'
scrape_interval: 5s
static_configs:
- targets:
- "node-exporter:9100"
- job_name: thanos-sidecar
scrape_interval: 5s
static_configs:
- targets:
- "sidecar0:10902"
- job_name: thanos-store
scrape_interval: 5s
static_configs:
- targets:
- "store:10906"
- job_name: thanos-query
scrape_interval: 5s
static_configs:
- targets:
- "query0:10904"

Let’s Execute it and See the Results

$ docker-compose up -d
Image from Docker Desktop application
Thanos Query UI
Grafaana Data Source Add Option
Adding Thanos Query URL to data source and Save it.
Grafana Query and Dashboard

In A Nutshell

Image Taken from Unsplash

--

--

--

Python, DevOps, Cryptography, Infra-Structure Automation. https://syedsaadahmed.com/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

What an awesome community!

How to make money with Python

A Day In My (New) Programmer Life

Which Hosting Should I choose to pass Core Web vitals? Shared or VPS

Reduce Cost and Increase Productivity with Value Added IT Services from buzinessware — {link} -

Popular web applications built with Python

Using GDELT for Community Landscaping

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Syed Saad Ahmed

Syed Saad Ahmed

Python, DevOps, Cryptography, Infra-Structure Automation. https://syedsaadahmed.com/

More from Medium

Run containers securely with gVisor on EKS

Deploying Kubernetes (K3S) to an ARM based VM on Oracle with ArgoCD, Cert Manager, Gitlabs CI and…

Run Argo CD using operator on Kind

Automate Certificate Lifecycle Management for CockroachDB using HashiCorp Vault