Docker, Prometheus and Grafana, Three in one !
Monitoring Server Metrics via Prometheus and Docker
For monitoring a server memory, CPU, disk and other stats, we are here using Docker, Prometheus and Grafana in our whole stack. In particular, we have used prometheus an open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach. for keeping the stack in separate containers and spaces we have kept every service in a separate docker container. Furthermore for scraping the metrics we have used a node-exporter which was provided by the community.
Pre-requisites, requirements & Compiling the Stack
One must have docker and docker-compose intalled in his local system for deploying this stack easily.
Installing Docker
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -sudo add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable”sudo apt-get updatesudo apt-get install -y docker-cesudo systemctl status docker
Installing docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/1.18.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-composesudo chmod +x /usr/local/bin/docker-composedocker-compose — version
Cloning the Repository
git clone https://github.com/syedsaadahmed/EMBL-EBI-DevOps-SRE/tree/master/Operations_SectionRunning the monitoring stack
sudo docker-compose up -dIt will deploy whole stack, no need of any manual or human intervention.
Checking the Grafana User Interface
https://localhost:3000/Credentials are (username:admin & Password:admin), you can change it in docker-compose.yml file as desired.
Why Docker, Prometheus and Grafana
Following the strategy to use the concept of Infrastructure as a code, i have used the stack of docker, prometheus and grafana. Prometheus is an easy to use open-source monitoring system. it works very well in a distributed, cloud-native environment. It is very fast and easy to deploy, Moreover it also has compatible integrations with visualization tools like Grafana etc. Grafana provides bulit-in dashboards that have pre-compiled ways to display a huge number of metrics. moreover you can also create your custom dashboards in grafana, many options are there like heatmap,graphs,status-map,alert-list and bubble chart etc.
Yes, there are other options available to solve this question, such as we can use NewRelic, Sensu and other such tools for achieving this task. But nowadays many huge organizations and companies are adopting prometheus as their baseline for monitoring their infrastructure and applications. There is a lot to explore and achieve via using prometheus.
Service Discovery and Other Such Options
Prometheus provides a File-based Service Discovery mechanism, Easiest is to utilize that method, one must have to maintain a target.json file containing information of all the node-exporters. furthermore to automate the whole process we can use ansible playbook or some other cookbook on Prometheus server to edit that target.json file locally and same playbook to install node-exporter service on our servers/nodes.
In the next part, we will surely discuss about seeing all of these three tools separately one by one.
For now, Here is the URL for complete project;
https://github.com/syedsaadahmed/EMBL-EBI-DevOps-SRE/tree/master/Operations_Section
