Hello :) We are using Datadog on Kong to monitor the metrics and analytics.
We feel that the cost associated with Datadog is high in terms of custom metrics and indexations. So, we planned to find an alternative for Datadog and we are looking into Grafana implementation with kong.
Will the shift from Datadog to Grafana be a wise move and flawless?
Hi Sathish,
Straightforward responding you: Will the shift from Datadog to Grafana be a wise move? R: Maybe, specially in the long run.
and flawless? R: It depends, but I don't think so. And it also won't be so smooth. You'll reach a point of Datatog deprec where your team will be supporting both, and at this peak will be when you'll think it was a bad move, because you'll be paying for Datadog and Prometheus/Grafana servers, plus your team will have to work on new system metrics and fine tuning metric queries and graphs on Grafana, so you can expect a delay on business deliveries. Your team will feel overburdened.
So bear in mind that basically you'll be moving from an out of the box monitoring solution to your own. The analogy I'd do is to moving out from a cloud provider to your own self managed servers, though way less complex of course.
Grafana itself won't do the trick. You'll need a data scrapper system like Prometheus to collect metrics/data from your app, or use a middleware Pushgateway to receive these data so Prometheus can scrape them. To do this you'll need some backend work to expose metrics data, instead of HTTP post your metrics data to Datadog, so you can expect a little re-architecture or engineering on that, depending on how your system is designed.
In the long run, you'll probably need a team focused only on that: to take care of the Prometheus/Grafana updates as well as their servers (after all they're still systems that need to be managed).
I think all of this will depend on the size of your company now and your teams (which I advise you to include them on that decision), and your budget plus rush for business deliveries. In the long run it'll pay, for your company will be more specialized on this stack, and you can have a team focused on that plus developer support.
If you wanna go with it, I'd suggest to do a small PoC (which still will need a good amount of work). Select a project (or even an endpoint/ mobile feature) to have both integrations: Datadog plus Grafana, and do the same graphs you have on Datadog to Grafana. I think this way you can have a better picture of the reality on how that will be for your company.
I also posted your question on ChatGPT, and here's its full response on that, which I also agree:
```Shifting from Datadog to Grafana for monitoring metrics and analytics can be a reasonable alternative, depending on your specific requirements and use case. Grafana is a popular open-source platform that provides rich visualization and monitoring capabilities. However, it's important to note that Grafana and Datadog have different feature sets and focus areas. Here are a few considerations to help you evaluate the potential shift:
1. Feature Comparison: Evaluate the features and functionalities offered by Grafana and compare them to the ones you currently use in Datadog. Ensure that Grafana meets your specific monitoring, analytics, and visualization needs.
2. Custom Metrics and Indexing: Understand your requirements for custom metrics and indexing in Grafana. Ensure that Grafana provides the necessary flexibility and support for the metrics and indexes you require. Check if Grafana integrates well with Kong and offers plugins or extensions for capturing custom metrics.
3. Learning Curve: Consider the learning curve associated with Grafana, as your team will need to become familiar with its interface, configuration, and query language (e.g., Prometheus Query Language). Assess whether your team has the required expertise or if additional training will be necessary.
4. Data Storage and Scalability: Evaluate how Grafana handles data storage and scalability. Determine if it can handle the volume of metrics and analytics data generated by Kong and if it integrates seamlessly with your existing data storage solutions.
5. Alerting and Notification: Determine if Grafana's alerting and notification features meet your requirements. Check if it supports real-time alerting, customizable thresholds, and various notification channels (e.g., email, Slack).
6. Ecosystem and Community Support: Consider the ecosystem and community support around Grafana. It has a large user community, which means you can find resources, plugins, and community-contributed dashboards to enhance your monitoring capabilities.
7. Cost Comparison: Evaluate the cost aspect of Grafana compared to Datadog. While Grafana itself is open source, you need to consider the costs associated with additional components, such as data storage, alerting systems, and any necessary infrastructure changes. Calculate the total cost of ownership and compare it to your current expenses with Datadog.
It's important to conduct a thorough evaluation and possibly run a proof of concept or pilot project to ensure that Grafana meets your specific needs before fully transitioning from Datadog.```
