Need advice about which tool to choose?Ask the StackShare community!

Datadog

9.4K
8K
+ 1
860
Elasticsearch

34.5K
26.9K
+ 1
1.6K
Add tool

Datadog vs Elasticsearch: What are the differences?

Datadog and Elasticsearch are two popular tools in the field of monitoring and observability. Let's explore the key differences between them.

  1. Flexibility of Use: Datadog is a comprehensive monitoring and analytics platform that offers a wide range of features including infrastructure, application, and network monitoring, as well as log management and APM. On the other hand, Elasticsearch is an open-source search and analytics engine that can be used to index, search, and analyze large volumes of data. While Datadog provides a single, integrated solution, Elasticsearch offers more flexibility in terms of use cases and configurations.

  2. Data Storage: Datadog relies on its own proprietary datastore to store monitoring and logs data, which is managed by the platform itself. In contrast, Elasticsearch stores data in an index-based data structure, allowing for customizable indexing and efficient querying. This enables Elasticsearch to handle large datasets and complex querying requirements more effectively.

  3. Scalability: Both Datadog and Elasticsearch are designed to scale horizontally and handle high volumes of data, but they approach scalability differently. Datadog offers a fully managed solution where the infrastructure scales automatically based on the needs of the monitored environment. Elasticsearch, being a self-hosted solution, requires manual setup and configuration of additional nodes for scaling. This provides more control over the scaling process but requires additional administrative effort.

  4. Ease of Setup and Maintenance: Datadog provides a user-friendly interface and automates many configuration and maintenance tasks. It offers easy integration with a wide range of technologies and platforms out of the box. Elasticsearch, being an open-source tool, requires more manual setup and configuration. It may involve more technical expertise and effort to deploy, manage, and maintain.

  5. Data Visualization and Analysis: Datadog provides a unified dashboard and visualization system that allows users to create custom dashboards and graphs for monitoring and analysis. It also offers various built-in analytics features for metrics and logs data. Elasticsearch, on the other hand, is primarily focused on providing a powerful search and indexing engine. While it offers some visualization capabilities, users often need to rely on additional tools like Kibana for advanced data visualization and analysis.

  6. Pricing Model: Datadog follows a subscription-based pricing model, where customers pay based on the number of monitored hosts or resources. This includes a specific set of features and data retention periods. Elasticsearch, being an open-source tool, is free to use but may require additional commercial plugins or professional services for enterprise-grade features and support.

In summary, Datadog provides a comprehensive, integrated monitoring and analytics platform with managed data storage, automated scaling, and user-friendly setup. Elasticsearch, on the other hand, offers more flexibility in data storage and querying, requires more manual setup and configuration, and focuses primarily on search and indexing capabilities.

Advice on Datadog and Elasticsearch
Farzeem Diamond Jiwani
Software Engineer at IVP · | 8 upvotes · 1.5M views
Needs advice
on
AppDynamicsAppDynamicsDatadogDatadog
and
DynatraceDynatrace

Hey there! We are looking at Datadog, Dynatrace, AppDynamics, and New Relic as options for our web application monitoring.

Current Environment: .NET Core Web app hosted on Microsoft IIS

Future Environment: Web app will be hosted on Microsoft Azure

Tech Stacks: IIS, RabbitMQ, Redis, Microsoft SQL Server

Requirement: Infra Monitoring, APM, Real - User Monitoring (User activity monitoring i.e., time spent on a page, most active page, etc.), Service Tracing, Root Cause Analysis, and Centralized Log Management.

Please advise on the above. Thanks!

See more
Needs advice
on
DatadogDatadogNew RelicNew Relic
and
SysdigSysdig

We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.

We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.

We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.

You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?

See more
Replies (3)
Recommends
on
DatadogDatadog

Can't say anything to Sysdig. I clearly prefer Datadog as

  • they provide plenty of easy to "switch-on" plugins for various technologies (incl. most of AWS)
  • easy to code (python) agent plugins / api for own metrics
  • brillant dashboarding / alarms with many customization options
  • pricing is OK, there are cheaper options for specific use cases but if you want superior dashboarding / alarms I haven't seen a good competitor (despite your own Prometheus / Grafana / Kibana dog food)

IMHO NewRelic is "promising since years" ;) good ideas but bad integration between their products. Their Dashboard query language is really nice but lacks critical functions like multiple data sets or advanced calculations. Needless to say you get all of that with Datadog.

Need help setting up a monitoring / logging / alarm infrastructure? Send me a message!

See more
Maik Schröder
Recommends
on
InstanaInstana

Hi Medeti,

you are right. Building based on your stack something with open source is heavy lifting. A lot of people I know start with such a set-up, but quickly run into frustration as they need to dedicated their best people to build a monitoring which is doing the job in a professional way.

As you are microservice focussed and are looking for 'low implementation and maintenance effort', you might want to have a look at INSTANA, which was built with modern tool stacks in mind. https://www.instana.com/apm-for-microservices/

We have a public sand-box available if you just want to have a look at the product once and of course also a free-trial: https://www.instana.com/getting-started-with-apm/

Let me know if you need anything on top.

See more
Attila Fulop
Management Advisor at artkonekt · | 2 upvotes · 338.7K views

I have hands on production experience both with New Relic and Datadog. I personally prefer Datadog over NewRelic because of the UI, the Documentation and the overall user/developer experience.

NewRelic however, can do basically the same things as Datadog can, and some of the features like alerting have been present in NewRelic for longer than in Datadog. The cool thing about NewRelic is their last-summer-updated pricing: you no longer pay per host but after data you send towards New Relic. This can be a huge cost saver depending on your particular setup

https://docs.newrelic.com/docs/accounts/accounts-billing/new-relic-one-pricing-billing/new-relic-one-pricing-billing

I'd go for Datadog, but given you have lots of containers I would also make a cost calculation. If the price difference is significant and there's a budget constraint NewRelic might be the better choice.

See more
Rana Usman Shahid
Chief Technology Officer at TechAvanza · | 6 upvotes · 390.7K views
Needs advice
on
AlgoliaAlgoliaElasticsearchElasticsearch
and
FirebaseFirebase

Hey everybody! (1) I am developing an android application. I have data of around 3 million record (less than a TB). I want to save that data in the cloud. Which company provides the best cloud database services that would suit my scenario? It should be secured, long term useable, and provide better services. I decided to use Firebase Realtime database. Should I stick with Firebase or are there any other companies that provide a better service?

(2) I have the functionality of searching data in my app. Same data (less than a TB). Which search solution should I use in this case? I found Elasticsearch and Algolia search. It should be secure and fast. If any other company provides better services than these, please feel free to suggest them.

Thank you!

See more
Replies (2)
Josh Dzielak
Co-Founder & CTO at Orbit · | 8 upvotes · 293.1K views
Recommends
on
AlgoliaAlgolia

Hi Rana, good question! From my Firebase experience, 3 million records is not too big at all, as long as the cost is within reason for you. With Firebase you will be able to access the data from anywhere, including an android app, and implement fine-grained security with JSON rules. The real-time-ness works perfectly. As a fully managed database, Firebase really takes care of everything. The only thing to watch out for is if you need complex query patterns - Firestore (also in the Firebase family) can be a better fit there.

To answer question 2: the right answer will depend on what's most important to you. Algolia is like Firebase is that it is fully-managed, very easy to set up, and has great SDKs for Android. Algolia is really a full-stack search solution in this case, and it is easy to connect with your Firebase data. Bear in mind that Algolia does cost money, so you'll want to make sure the cost is okay for you, but you will save a lot of engineering time and never have to worry about scale. The search-as-you-type performance with Algolia is flawless, as that is a primary aspect of its design. Elasticsearch can store tons of data and has all the flexibility, is hosted for cheap by many cloud services, and has many users. If you haven't done a lot with search before, the learning curve is higher than Algolia for getting the results ranked properly, and there is another learning curve if you want to do the DevOps part yourself. Both are very good platforms for search, Algolia shines when buliding your app is the most important and you don't want to spend many engineering hours, Elasticsearch shines when you have a lot of data and don't mind learning how to run and optimize it.

See more
Mike Endale
Recommends
on
Cloud FirestoreCloud Firestore

Rana - we use Cloud Firestore at our startup. It handles many million records without any issues. It provides you the same set of features that the Firebase Realtime Database provides on top of the indexing and security trims. The only thing to watch out for is to make sure your Cloud Functions have proper exception handling and there are no infinite loop in the code. This will be too costly if not caught quickly.

For search; Algolia is a great option, but cost is a real consideration. Indexing large number of records can be cost prohibitive for most projects. Elasticsearch is a solid alternative, but requires a little additional work to configure and maintain if you want to self-host.

Hope this helps.

See more
Decisions about Datadog and Elasticsearch
Attila Fulop

I haven't heard much about Datadog until about a year ago. Ironically, the NewRelic sales person who I had a series of trainings with was trash talking about Datadog a lot. That drew my attention to Datadog and I gave it a try at another client project where we needed log handling, dashboards and alerting.

In 2019, Datadog was already offering log management and from that perspective, it was ahead of NewRelic. Other than that, from my perspective, the two tools are offering a very-very similar set of tools. Therefore I wouldn't say there's a significant difference between the two, the decision is likely a matter of taste. The pricing is also very similar.

The reasons why we chose Datadog over NewRelic were:

  • The presence of log handling feature (since then, logging is GA at NewRelic as well since falls 2019).
  • The setup was easier even though I already had experience with NewRelic, including participation in NewRelic trainings.
  • The UI of Datadog is more compact and my experience is smoother.
  • The NewRelic UI is very fragmented and New Relic One is just increasing this experience for me.
  • The log feature of Datadog is very well designed, I find very useful the tagging logs with services. The log filtering is also very awesome.

Bottom line is that both tools are great and it makes sense to discover both and making the decision based on your use case. In our case, Datadog was the clear winner due to its UI, ease of setup and the awesome logging and alerting features.

See more
Benoit Larroque
Principal Engineer at Sqreen · | 4 upvotes · 436.6K views

I chose Datadog APM because the much better APM insights it provides (flamegraph, percentiles by default).

The drawbacks of this decision are we had to move our production monitoring to TimescaleDB + Telegraf instead of NR Insight

NewRelic is definitely easier when starting out. Agent is only a lib and doesn't require a daemon

See more
Manage your open source components, licenses, and vulnerabilities
Learn More
Pros of Datadog
Pros of Elasticsearch
  • 139
    Monitoring for many apps (databases, web servers, etc)
  • 107
    Easy setup
  • 87
    Powerful ui
  • 84
    Powerful integrations
  • 70
    Great value
  • 54
    Great visualization
  • 46
    Events + metrics = clarity
  • 41
    Notifications
  • 41
    Custom metrics
  • 39
    Flexibility
  • 19
    Free & paid plans
  • 16
    Great customer support
  • 15
    Makes my life easier
  • 10
    Adapts automatically as i scale up
  • 9
    Easy setup and plugins
  • 8
    Super easy and powerful
  • 7
    AWS support
  • 7
    In-context collaboration
  • 6
    Rich in features
  • 5
    Docker support
  • 4
    Cost
  • 4
    Full visibility of applications
  • 4
    Monitor almost everything
  • 4
    Cute logo
  • 4
    Automation tools
  • 4
    Source control and bug tracking
  • 4
    Simple, powerful, great for infra
  • 4
    Easy to Analyze
  • 4
    Best than others
  • 3
    Best in the field
  • 3
    Expensive
  • 3
    Good for Startups
  • 3
    Free setup
  • 2
    APM
  • 328
    Powerful api
  • 315
    Great search engine
  • 231
    Open source
  • 214
    Restful
  • 200
    Near real-time search
  • 98
    Free
  • 85
    Search everything
  • 54
    Easy to get started
  • 45
    Analytics
  • 26
    Distributed
  • 6
    Fast search
  • 5
    More than a search engine
  • 4
    Great docs
  • 4
    Awesome, great tool
  • 3
    Highly Available
  • 3
    Easy to scale
  • 2
    Potato
  • 2
    Document Store
  • 2
    Great customer support
  • 2
    Intuitive API
  • 2
    Nosql DB
  • 2
    Great piece of software
  • 2
    Reliable
  • 2
    Fast
  • 2
    Easy setup
  • 1
    Open
  • 1
    Easy to get hot data
  • 1
    Github
  • 1
    Elaticsearch
  • 1
    Actively developing
  • 1
    Responsive maintainers on GitHub
  • 1
    Ecosystem
  • 1
    Not stable
  • 1
    Scalability
  • 0
    Community

Sign up to add or upvote prosMake informed product decisions

Cons of Datadog
Cons of Elasticsearch
  • 20
    Expensive
  • 4
    No errors exception tracking
  • 2
    External Network Goes Down You Wont Be Logging
  • 1
    Complicated
  • 7
    Resource hungry
  • 6
    Diffecult to get started
  • 5
    Expensive
  • 4
    Hard to keep stable at large scale

Sign up to add or upvote consMake informed product decisions

What is Datadog?

Datadog is the leading service for cloud-scale monitoring. It is used by IT, operations, and development teams who build and operate applications that run on dynamic or hybrid cloud infrastructure. Start monitoring in minutes with Datadog!

What is Elasticsearch?

Elasticsearch is a distributed, RESTful search and analytics engine capable of storing data and searching it in near real time. Elasticsearch, Kibana, Beats and Logstash are the Elastic Stack (sometimes called the ELK Stack).

Need advice about which tool to choose?Ask the StackShare community!

What companies use Datadog?
What companies use Elasticsearch?
Manage your open source components, licenses, and vulnerabilities
Learn More

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with Datadog?
What tools integrate with Elasticsearch?

Sign up to get full access to all the tool integrationsMake informed product decisions

Blog Posts

May 21 2019 at 12:20AM

Elastic

ElasticsearchKibanaLogstash+4
12
5296
GitHubPythonReact+42
49
40929
GitHubPythonNode.js+47
55
72785
What are some alternatives to Datadog and Elasticsearch?
New Relic
The world’s best software and DevOps teams rely on New Relic to move faster, make better decisions and create best-in-class digital experiences. If you run software, you need to run New Relic. More than 50% of the Fortune 100 do too.
Splunk
It provides the leading platform for Operational Intelligence. Customers use it to search, monitor, analyze and visualize machine data.
Prometheus
Prometheus is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true.
Grafana
Grafana is a general purpose dashboard and graph composer. It's focused on providing rich ways to visualize time series metrics, mainly though graphs but supports other ways to visualize data through a pluggable panel architecture. It currently has rich support for for Graphite, InfluxDB and OpenTSDB. But supports other data sources via plugins.
AppDynamics
AppDynamics develops application performance management (APM) solutions that deliver problem resolution for highly distributed applications through transaction flow monitoring and deep diagnostics.
See all alternatives