Grafana

DevOps / Monitoring / Monitoring Tools
CEO at Scrayos UG (haftungsbeschr盲nkt)

Grafana is used in combination with Prometheus to display the gathered stats and to monitor our physical servers aswell as their virtual applications. While Grafana also allows to configure automated alerts and rules, we decided to use Prometheus Alertmanager, as it is offers advanced features for silences (muting of alerts for a specific time) and also allows more fine-grained rules and notifications for each alert.

READ MORE
1 upvote30.9K views
CEO at Scrayos UG (haftungsbeschr盲nkt)

We primarily use Prometheus to gather metrics and statistics to display them in Grafana. Aside from that we poll Prometheus for our orchestration-solution "JCOverseer" to determine, which host is least occupied at the moment.

While there are existing orchestration softwares/suites like Kubernetes, that we also plan to adopt in the future, we're of the opinion that those solutions do not fit our special environment within minecraft and our own solution will outperform them in the limited scope that it needs to cover.

READ MORE
1 upvote56K views
Team Lead at DBS
Needs advice
on
Prometheus
and
Metricbeat

Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.

Is it OK to use Metricbeat for Linux server or can we use Prometheus?

What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?

Regards, Sunil.

READ MORE
2 upvotes164.8K views
Replies (2)
Recommends
Prometheus

If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.

Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.

READ MORE
3 upvotes94K views
Needs advice
on
Sysdig
New Relic
and
Datadog

We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.

We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.

We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.

You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?

READ MORE
6 upvotes603.5K views
Replies (3)
Recommends
Datadog

Can't say anything to Sysdig. I clearly prefer Datadog as

  • they provide plenty of easy to "switch-on" plugins for various technologies (incl. most of AWS)
  • easy to code (python) agent plugins / api for own metrics
  • brillant dashboarding / alarms with many customization options
  • pricing is OK, there are cheaper options for specific use cases but if you want superior dashboarding / alarms I haven't seen a good competitor (despite your own Prometheus / Grafana / Kibana dog food)

IMHO NewRelic is "promising since years" ;) good ideas but bad integration between their products. Their Dashboard query language is really nice but lacks critical functions like multiple data sets or advanced calculations. Needless to say you get all of that with Datadog.

Need help setting up a monitoring / logging / alarm infrastructure? Send me a message!

READ MORE
9 upvotes2 comments107.1K views
Medeti Vamsi Krishna
Medeti Vamsi Krishna
June 30th 2020 at 11:52AM

Thanks for the reply, I am working on DataDog trail version now. I am able to see my containers/pods/VMs metrics in the DataDog.

I am trying to do the jmx integration with autodiscovery now. But I am not able to see the jvm metrics in DataDog. Can you please help on this?

Here is my deployment yaml:

`

apiVersion: apps/v1

kind: Deployment

metadata:

name: myapp

namespace: datadog

annotations:

ad.datadoghq.com/myapp.check_names: >-

'["myapp"]'

ad.datadoghq.com/myapp.init_configs: >-

'[{"is_jmx": true, "collect_default_metrics": true}]'

ad.datadoghq.com/tomcat.instances: >-

'[{"host": "%%host%%","port":"5000"}]'

labels:

app: myapp

spec:

selector:

matchLabels:

app: myapp

template:

metadata:

labels:

app: myapp

spec:

containers:

- name: myapp

image: nexus.nslhub.com/sample-java-app:2.0

imagePullPolicy: Always

ports:

- containerPort: 8080

name: http

- containerPort: 5000

name: jmx

imagePullSecrets:

- name: myappsecret

nodeSelector:

kubernetes.io/hostname: ip-10-5-7-173.ap-south-1.compute.internal

`

Reply
Jens G眉nther
Jens G眉nther
June 30th 2020 at 11:57AM

Would like to help, but there could be hundreds of reasons why the incoming and outgoing jmx ports are not accessible from the agent.

Reply
Recommends
Instana

Hi Medeti,

you are right. Building based on your stack something with open source is heavy lifting. A lot of people I know start with such a set-up, but quickly run into frustration as they need to dedicated their best people to build a monitoring which is doing the job in a professional way.

As you are microservice focussed and are looking for 'low implementation and maintenance effort', you might want to have a look at INSTANA, which was built with modern tool stacks in mind. https://www.instana.com/apm-for-microservices/

We have a public sand-box available if you just want to have a look at the product once and of course also a free-trial: https://www.instana.com/getting-started-with-apm/

Let me know if you need anything on top.

READ MORE
7 upvotes107.1K views
View all (3)
Senior SRE at African Bank
Needs advice
on
Prometheus
Graphite
and
Grafana

Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:

  • Must be able to get custom data from AS400,
  • Able to display automation test results,
  • System monitoring / Nginx API,
  • Able to get data from 3rd parties DB.

Grafana is almost solving all the problems, except AS400 and no database to get automation test results.

READ MORE
4 upvotes304.5K views
Replies (1)
Technical Specialist, Software Engineering at AT&T

You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.

READ MORE
3 upvotes1 comment108K views
Susmita Meher
Susmita Meher
October 3rd 2020 at 8:01PM

Thank you for the suggestions.

However, I managed to write libraries for Prometheus using NodeJs for the adhoc quesries.

Reply
Software Engineering Manager at PayIt

Grafana and Prometheus together, running on Kubernetes , is a powerful combination. These tools are cloud-native and offer a large community and easy integrations. At PayIt we're using exporting Java application metrics using a Dropwizard metrics exporter, and our Node.js services now use the prom-client npm library to serve metrics.

READ MORE
With multiplying microservices running on Kubernetes, PayIt turned to Grafana and Prometheus for observability at cloud native scale | Grafana Labs (grafana.com)
5 upvotes14.2K views