Prometheus

Prometheus

DevOps / Monitoring / Monitoring Tools
Needs advice
on
Azure MonitorAzure Monitor
and
PrometheusPrometheus

Can I get metrics available through Prometheus into Azure Monitor, specifically into log analytics? (VM'S). I am running a couple of VM's inside Azure portal and I have my own private besu nodes running on them. I have my metrics set up inside the Prometheus but I was hoping to hook it up securely to Grafana but I tried everything and I can't. So the next thing is to see if can I get the metrics available through Prometheus into azure monitor, specifically into log analytics. The aim is to get the sync status, and the highest block number on each node, into log analytics so we can see what each is doing. That way we know, on a quick look, the status of each node and by extension, the condition of the private chain. What worries me is that although I have alerts if blocks stop being created or nodes lose peers we cannot see it quickly.

Prometheus is one option to give us those stats. If we can get data from Prometheus into log analytics that would solve the problem.

Can anyone help me with how I can go about it or any links? All I am seeing is for containers but I want for my VMs.

READ MORE
4 upvotes·50.7K views
Replies (2)
Avatar of kstkrv
Systems Engineer ·

Hi Darragh,

You can use Function Apps to scrape metrics and send them to a Log Analytics Data Collector API. Still, from my experience, I can say that it's hard to query data in Prometheus format with KQL. Instead, I'd add the Application Insights instrumentation to the application or proceed with building the Prometheus setup.

READ MORE
2 upvotes·3.8K views

Hi Darragh, I'm part of the User Success team at Grafana. I'd love to hear more about what you're working on, understand your use case and see if I can provide more resources for you. Care to chat?

1 upvote·4.1K views
Needs advice
on
NagiosNagios
and
PrometheusPrometheus

I am new to DevOps and looking for training in DevOps. Some institutes are offering Nagios while some Prometheus in their syllabus. Please suggest which one is being used in the industry and which one should I learn.

3 upvotes·242.1K views
Needs advice
on
PrometheusPrometheus
and
ThanosThanos

Hi All, We have Thanos sidecar and Prometheus set up in GCP and a Prometheus server in AWS. we want to push all the metrics of GCP to AWS. We will be creating a VPN link-up between them. But then how the GCP metrics would be connected with Prometheus/ Grafana.? I mean which IP to use it for this.?

READ MORE
3 upvotes·45.6K views
Replies (1)
Recommends
on
Prometheus
Prometheus
Thanos
Thanos

Hi Suraj,

If :

  1. Thanos is installed on AWS
  2. The Prometheus on GCP is configured with a Thanos sidecar forwarding metrics to the AWS Thanos.
  3. The Prometheus on AWS is configured with a Thanos sidecar forwarding metrics to the AWS Thanos.

Then :

Your Grafana should then point to the Thanos Querier IP:port on AWS. You will then be able to view the GCP metrics as well as the AWS metrics.

Trust this helps. Send me an email if you need more help in this area.

Scott Fulton scott.fulton@opscruise.com

READ MORE
5 upvotes·466 views
Avatar of menzenski
Senior Software Engineering Manager at PayIt·

Grafana and Prometheus together, running on Kubernetes , is a powerful combination. These tools are cloud-native and offer a large community and easy integrations. At PayIt we're using exporting Java application metrics using a Dropwizard metrics exporter, and our Node.js services now use the prom-client npm library to serve metrics.

READ MORE
With multiplying microservices running on Kubernetes, PayIt turned to Grafana and Prometheus for observability at cloud native scale | Grafana Labs (grafana.com)
16 upvotes·1.1M views
Needs advice
on
DatadogDatadogNew RelicNew Relic
and
SysdigSysdig

We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.

We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.

We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.

You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?

READ MORE
7 upvotes·1.5M views
Replies (3)
Recommends
on
Datadog
Datadog

Can't say anything to Sysdig. I clearly prefer Datadog as

  • they provide plenty of easy to "switch-on" plugins for various technologies (incl. most of AWS)
  • easy to code (python) agent plugins / api for own metrics
  • brillant dashboarding / alarms with many customization options
  • pricing is OK, there are cheaper options for specific use cases but if you want superior dashboarding / alarms I haven't seen a good competitor (despite your own Prometheus / Grafana / Kibana dog food)

IMHO NewRelic is "promising since years" ;) good ideas but bad integration between their products. Their Dashboard query language is really nice but lacks critical functions like multiple data sets or advanced calculations. Needless to say you get all of that with Datadog.

Need help setting up a monitoring / logging / alarm infrastructure? Send me a message!

READ MORE
10 upvotes·2 comments·421.2K views
Medeti Vamsi Krishna
Medeti Vamsi Krishna
·
June 30th 2020 at 11:52AM

Thanks for the reply, I am working on DataDog trail version now. I am able to see my containers/pods/VMs metrics in the DataDog.

I am trying to do the jmx integration with autodiscovery now. But I am not able to see the jvm metrics in DataDog. Can you please help on this?

Here is my deployment yaml:

`

apiVersion: apps/v1

kind: Deployment

metadata:

name: myapp

namespace: datadog

annotations:

ad.datadoghq.com/myapp.check_names: >-

'["myapp"]'

ad.datadoghq.com/myapp.init_configs: >-

'[{"is_jmx": true, "collect_default_metrics": true}]'

ad.datadoghq.com/tomcat.instances: >-

'[{"host": "%%host%%","port":"5000"}]'

labels:

app: myapp

spec:

selector:

matchLabels:

app: myapp

template:

metadata:

labels:

app: myapp

spec:

containers:

- name: myapp

image: nexus.nslhub.com/sample-java-app:2.0

imagePullPolicy: Always

ports:

- containerPort: 8080

name: http

- containerPort: 5000

name: jmx

imagePullSecrets:

- name: myappsecret

nodeSelector:

kubernetes.io/hostname: ip-10-5-7-173.ap-south-1.compute.internal

`

·
Reply
Jens Günther
Jens Günther
·
June 30th 2020 at 11:57AM

Would like to help, but there could be hundreds of reasons why the incoming and outgoing jmx ports are not accessible from the agent.

·
Reply
Recommends
on
Instana
Instana

Hi Medeti,

you are right. Building based on your stack something with open source is heavy lifting. A lot of people I know start with such a set-up, but quickly run into frustration as they need to dedicated their best people to build a monitoring which is doing the job in a professional way.

As you are microservice focussed and are looking for 'low implementation and maintenance effort', you might want to have a look at INSTANA, which was built with modern tool stacks in mind. https://www.instana.com/apm-for-microservices/

We have a public sand-box available if you just want to have a look at the product once and of course also a free-trial: https://www.instana.com/getting-started-with-apm/

Let me know if you need anything on top.

READ MORE
8 upvotes·421.2K views
View all (3)
Avatar of sunilmchaudhari
Team Lead at XYZ·
Needs advice
on
MetricbeatMetricbeat
and
PrometheusPrometheus

Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.

Is it OK to use Metricbeat for Linux server or can we use Prometheus?

What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?

Regards, Sunil.

READ MORE
2 upvotes·576.5K views
Replies (2)
Recommends
on
Prometheus
Prometheus

If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.

Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.

READ MORE
5 upvotes·1 comment·353K views
Manish Sharma
Manish Sharma
·
July 23rd 2021 at 9:41AM

This is perfect answer.

·
Reply
Avatar of Scrayos
CEO at Scrayos UG (haftungsbeschränkt)·

We primarily use Prometheus to gather metrics and statistics to display them in Grafana. Aside from that we poll Prometheus for our orchestration-solution "JCOverseer" to determine, which host is least occupied at the moment.

While there are existing orchestration softwares/suites like Kubernetes, that we also plan to adopt in the future, we're of the opinion that those solutions do not fit our special environment within minecraft and our own solution will outperform them in the limited scope that it needs to cover.

READ MORE
1 upvote·224.3K views
Needs advice
on
PrometheusPrometheus
and
SysdigSysdig

We have Prometheus as a monitoring engine as a part of our stack which contains Kubernetes cluster, container images and other open source tools. Also, I am aware that Sysdig can be integrated with Prometheus but I really wanted to know whether Sysdig or sysdig+prometheus will make better monitoring solution.

READ MORE
11 upvotes·767.8K views
Avatar of Scrayos
CEO at Scrayos UG (haftungsbeschränkt)·

Grafana is used in combination with Prometheus to display the gathered stats and to monitor our physical servers aswell as their virtual applications. While Grafana also allows to configure automated alerts and rules, we decided to use Prometheus Alertmanager, as it is offers advanced features for silences (muting of alerts for a specific time) and also allows more fine-grained rules and notifications for each alert.

READ MORE
2 upvotes·69.6K views