Datadog vs Prometheus: What are the differences?
Developers describe Datadog as "Unify logs, metrics, and traces from across your distributed infrastructure". Datadog is the leading service for cloud-scale monitoring. It is used by IT, operations, and development teams who build and operate applications that run on dynamic or hybrid cloud infrastructure. Start monitoring in minutes with Datadog!. On the other hand, Prometheus is detailed as "An open-source service monitoring system and time series database, developed by SoundCloud". Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.
Datadog and Prometheus are primarily classified as "Performance Monitoring" and "Monitoring" tools respectively.
Some of the features offered by Datadog are:
- 14-day Free Trial for an unlimited number of hosts
- 200+ turn-key integrations for data aggregation
- Clean graphs of StatsD and other integrations
On the other hand, Prometheus provides the following key features:
- a multi-dimensional data model (timeseries defined by metric name and set of key/value dimensions)
- a flexible query language to leverage this dimensionality
- no dependency on distributed storage
"Monitoring for many apps (databases, web servers, etc)" is the primary reason why developers consider Datadog over the competitors, whereas "Powerful easy to use monitoring" was stated as the key factor in picking Prometheus.
Prometheus is an open source tool with 24.6K GitHub stars and 3.49K GitHub forks. Here's a link to Prometheus's open source repository on GitHub.
Shopify, Salesforce, and Starbucks are some of the popular companies that use Datadog, whereas Prometheus is used by Slack, Docplanner, and Uber Technologies. Datadog has a broader approval, being mentioned in 532 company stacks & 213 developers stacks; compared to Prometheus, which is listed in 235 company stacks and 84 developer stacks.
Looking for a tool which can be used for mainly dashboard purposes, but here are the main requirements:
- Must be able to get custom data from AS400,
- Able to display automation test results,
- System monitoring / Nginx API,
- Able to get data from 3rd parties DB.
Grafana is almost solving all the problems, except AS400 and no database to get automation test results.
You can look out for Prometheus Instrumentation (https://prometheus.io/docs/practices/instrumentation/) Client Library available in various languages https://prometheus.io/docs/instrumenting/clientlibs/ to create the custom metric you need for AS4000 and then Grafana can query the newly instrumented metric to show on the dashboard.
Hey there! We are looking at Datadog, Dynatrace, AppDynamics, and New Relic as options for our web application monitoring.
Current Environment: .NET Core Web app hosted on Microsoft IIS
Future Environment: Web app will be hosted on Microsoft Azure
Tech Stacks: IIS, RabbitMQ, Redis, Microsoft SQL Server
Requirement: Infra Monitoring, APM, Real - User Monitoring (User activity monitoring i.e., time spent on a page, most active page, etc.), Service Tracing, Root Cause Analysis, and Centralized Log Management.
Please advise on the above. Thanks!
We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.
We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.
We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.
You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?
Can't say anything to Sysdig. I clearly prefer Datadog as
- they provide plenty of easy to "switch-on" plugins for various technologies (incl. most of AWS)
- easy to code (python) agent plugins / api for own metrics
- brillant dashboarding / alarms with many customization options
- pricing is OK, there are cheaper options for specific use cases but if you want superior dashboarding / alarms I haven't seen a good competitor (despite your own Prometheus / Grafana / Kibana dog food)
IMHO NewRelic is "promising since years" ;) good ideas but bad integration between their products. Their Dashboard query language is really nice but lacks critical functions like multiple data sets or advanced calculations. Needless to say you get all of that with Datadog.
Need help setting up a monitoring / logging / alarm infrastructure? Send me a message!
you are right. Building based on your stack something with open source is heavy lifting. A lot of people I know start with such a set-up, but quickly run into frustration as they need to dedicated their best people to build a monitoring which is doing the job in a professional way.
As you are microservice focussed and are looking for 'low implementation and maintenance effort', you might want to have a look at INSTANA, which was built with modern tool stacks in mind. https://www.instana.com/apm-for-microservices/
We have a public sand-box available if you just want to have a look at the product once and of course also a free-trial: https://www.instana.com/getting-started-with-apm/
Let me know if you need anything on top.
I have hands on production experience both with New Relic and Datadog. I personally prefer Datadog over NewRelic because of the UI, the Documentation and the overall user/developer experience.
NewRelic however, can do basically the same things as Datadog can, and some of the features like alerting have been present in NewRelic for longer than in Datadog. The cool thing about NewRelic is their last-summer-updated pricing: you no longer pay per host but after data you send towards New Relic. This can be a huge cost saver depending on your particular setup
I'd go for Datadog, but given you have lots of containers I would also make a cost calculation. If the price difference is significant and there's a budget constraint NewRelic might be the better choice.
Hi, We have a situation, where we are using Prometheus to get system metrics from PCF (Pivotal Cloud Foundry) platform. We send that as time-series data to Cortex via a Prometheus server and built a dashboard using Grafana. There is another pipeline where we need to read metrics from a Linux server using Metricbeat, CPU, memory, and Disk. That will be sent to Elasticsearch and Grafana will pull and show the data in a dashboard.
Is it OK to use Metricbeat for Linux server or can we use Prometheus?
What is the difference in system metrics sent by Metricbeat and Prometheus node exporters?
If you're already using Prometheus for your system metrics, then it seems like standing up Elasticsearch just for Linux host monitoring is excessive. The node_exporter is probably sufficient if you'e looking for standard system metrics.
Another thing to consider is that Metricbeat / ELK use a push model for metrics delivery, whereas Prometheus pulls metrics from each node it is monitoring. Depending on how you manage your network security, opting for one solution over two may make things simpler.
Hi Sunil! Unfortunately, I don´t have much experience with Metricbeat so I can´t advise on the diffs with Prometheus...for Linux server, I encourage you to use Prometheus node exporter and for PCF, I would recommend using the instana tile (https://www.instana.com/supported-technologies/pivotal-cloud-foundry/). Let me know if you have further questions! Regards Jose
So, I am working in a big company where they have multiple different microservices running that are written in Golang. I am currently searching for a technology that can give me all the metric data from the microservices. What time-series databases would you recommend? or which databases would you recommend to further investigate? I appreciate any input.
Each of these tools can help you with micro service workload and work well. I will try to go through some good, bad and ugly of each.
Datadog has an easy setup and time to get something tangible out of it. The cost model is by host so this is something to take into consideration how it will affect your use case. Also as a large organization at some point you will probably want control over some/all of your telemetry data to run your own ML or AI processes. With Datadog you this can be difficult as you will need to create processes outside of its closed eco system to get Raw metrics.
Prometheus is a great tool. It also has a fairly straight forward setup especially with Kubernetes. If you are running your micro services in k8s then this is going to get used one way or another; it is a first class citizen there with heavy utilization of K8s API. I also like the fact that Kubernetes architecture is easy to understand and that it utilizes Grafana for the visualization engine. Prometheus at scale can be done but it is a pain. Especially with a distributed infrastructure across multiple workloads.
Influxdb (TICK stack in v1) is known for its scalability and flexibility as a time series database. Telegraf is the main input/data-forwarder of the architecture and is completely decoupled from the database as are the other 3 components of the stack. Influx has made it very easy to just use one component on its own. I have worked on stacks that just used telegraf for ingestion into Kinesis or another data stream. I have also worked on stacks that used Influx database but used a different ETL process for analyzing the data in realtime instead of using their v1 architectures Kapacitor query engine. Influx database is a great performing time series database that in version 2 runs within kubernetes and utilizes Flux as the query language. Flux is a nice query language that is fairly easy to learn and has a lot of flexibility. As a last positive note Telegraf is written in Go so that would fit well with your current team.
The difficulties of Influx are that it is hard to get something really tangible out of it. Initial time to see something is fast but all the other work involved is a lot. You also have to understand the architecture well. The management of Influx can be cumbersome but it can scale up better than the other two when Datadogs cost is taken into consideration. They have a lot of API hooks in their V1 enterprise edition to wire and configure it. They do offer a mange service to offload this cost until later.
My overall choice here is probably to go with some of the influx as you can rip and/or add components as needed into the flow. Eventually you will probably want to run an ML process within there (can be done within Kapacitor but of course can also use your cloud provider here too) and this gives you the flexibility to do it anywhere. I would still go through prometheus because you will most likely use it also, but it does have forwarders to Influxdb so still fits.
We're running Prometheus/Alertmanager/Grafana across our whole company for any monitoring and metrics requirement, from the infrastructure layer all the way up to Springboot endpoint services, the prometheus exporter / scraping approach works pretty well for us. It's really easy to setup and more importantly; to maintain it without much effort, all the Prometheus configs get automatically created through Terraform outputs and Ansible jobs. Combine it with Grafana and you're smiling.
We're looking for a Monitoring and Logging tool. It has to support AWS (mostly 100% serverless, Lambdas, SNS, SQS, API GW, CloudFront, Autora, etc.), as well as Azure and GCP (for now mostly used as pure IaaS, with a lot of cognitive services, and mostly managed DB). Hopefully, something not as expensive as Datadog or New relic, as our SRE team could support the tool inhouse. At the moment, we primarily use CloudWatch for AWS and Pandora for most on-prem.
I worked with Datadog at least one year and my position is that commercial tools like Datadog are the best option to consolidate and analyze your metrics. Obviously, if you can't pay the tool, the best free options are the mix of Prometheus with their Alert Manager and Grafana to visualize (that are complementary not substitutable). But I think that no use a good tool it's finally more expensive that use a not really good implementation of free tools and you will pay also to maintain its.
this is quite affordable and provides what you seem to be looking for. you can see a whole thing about the APM space here https://www.apmexperts.com/observability/ranking-the-observability-offerings/
The objective of this work was to develop a system to monitor the materials of a production line using IoT technology. Currently, the process of monitoring and replacing parts depends on manual services. For this, load cells, microcontroller, Broker MQTT, Telegraf, InfluxDB, and Grafana were used. It was implemented in a workflow that had the function of collecting sensor data, storing it in a database, and visualizing it in the form of weight and quantity. With these developed solutions, he hopes to contribute to the logistics area, in the replacement and control of materials.
I haven't heard much about Datadog until about a year ago. Ironically, the NewRelic sales person who I had a series of trainings with was trash talking about Datadog a lot. That drew my attention to Datadog and I gave it a try at another client project where we needed log handling, dashboards and alerting.
In 2019, Datadog was already offering log management and from that perspective, it was ahead of NewRelic. Other than that, from my perspective, the two tools are offering a very-very similar set of tools. Therefore I wouldn't say there's a significant difference between the two, the decision is likely a matter of taste. The pricing is also very similar.
The reasons why we chose Datadog over NewRelic were:
- The presence of log handling feature (since then, logging is GA at NewRelic as well since falls 2019).
- The setup was easier even though I already had experience with NewRelic, including participation in NewRelic trainings.
- The UI of Datadog is more compact and my experience is smoother.
- The NewRelic UI is very fragmented and New Relic One is just increasing this experience for me.
- The log feature of Datadog is very well designed, I find very useful the tagging logs with services. The log filtering is also very awesome.
Bottom line is that both tools are great and it makes sense to discover both and making the decision based on your use case. In our case, Datadog was the clear winner due to its UI, ease of setup and the awesome logging and alerting features.
I chose Datadog APM because the much better APM insights it provides (flamegraph, percentiles by default).
The drawbacks of this decision are we had to move our production monitoring to TimescaleDB + Telegraf instead of NR Insight
NewRelic is definitely easier when starting out. Agent is only a lib and doesn't require a daemon
Sign up to add or upvote prosMake informed product decisions
Sign up to add or upvote consMake informed product decisions
What is Datadog?
What is Prometheus?
Need advice about which tool to choose?Ask the StackShare community!
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
Red Hat, Inc.
Red Hat, Inc.