We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.
We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.
We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.
You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?
Can't say anything to Sysdig. I clearly prefer Datadog as
- they provide plenty of easy to "switch-on" plugins for various technologies (incl. most of AWS)
- easy to code (python) agent plugins / api for own metrics
- brillant dashboarding / alarms with many customization options
- pricing is OK, there are cheaper options for specific use cases but if you want superior dashboarding / alarms I haven't seen a good competitor (despite your own Prometheus / Grafana / Kibana dog food)
IMHO NewRelic is "promising since years" ;) good ideas but bad integration between their products. Their Dashboard query language is really nice but lacks critical functions like multiple data sets or advanced calculations. Needless to say you get all of that with Datadog.
Need help setting up a monitoring / logging / alarm infrastructure? Send me a message!
Thanks for the reply, I am working on DataDog trail version now. I am able to see my containers/pods/VMs metrics in the DataDog.
I am trying to do the jmx integration with autodiscovery now. But I am not able to see the jvm metrics in DataDog. Can you please help on this?
Here is my deployment yaml:
`
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: datadog
annotations:
ad.datadoghq.com/myapp.check_names: >-
'["myapp"]'
ad.datadoghq.com/myapp.init_configs: >-
'[{"is_jmx": true, "collect_default_metrics": true}]'
ad.datadoghq.com/tomcat.instances: >-
'[{"host": "%%host%%","port":"5000"}]'
labels:
app: myapp
spec:
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: nexus.nslhub.com/sample-java-app:2.0
imagePullPolicy: Always
ports:
- containerPort: 8080
name: http
- containerPort: 5000
name: jmx
imagePullSecrets:
- name: myappsecret
nodeSelector:
kubernetes.io/hostname: ip-10-5-7-173.ap-south-1.compute.internal
`
Would like to help, but there could be hundreds of reasons why the incoming and outgoing jmx ports are not accessible from the agent.
Hi Medeti,
you are right. Building based on your stack something with open source is heavy lifting. A lot of people I know start with such a set-up, but quickly run into frustration as they need to dedicated their best people to build a monitoring which is doing the job in a professional way.
As you are microservice focussed and are looking for 'low implementation and maintenance effort', you might want to have a look at INSTANA, which was built with modern tool stacks in mind. https://www.instana.com/apm-for-microservices/
We have a public sand-box available if you just want to have a look at the product once and of course also a free-trial: https://www.instana.com/getting-started-with-apm/
Let me know if you need anything on top.