What is Amazon CloudWatch?
Who uses Amazon CloudWatch?
Amazon CloudWatch Integrations
Why developers like Amazon CloudWatch?
Here are some stack decisions, common use cases and reviews by companies and developers who chose Amazon CloudWatch in their tech stack.
I'm the CTO of a marketing automation SaaS. Because of the continuously increasing load we moved to the AWSCloud. We are using more and more features of AWS: Amazon CloudWatch, Amazon SNS, Amazon CloudFront, Amazon Route 53 and so on.
Our main Database is MySQL but for the hundreds of GB document data we use MongoDB more and more. We started to use Redis for cache and other time sensitive operations.
On the front-end we use jQuery UI + Smarty but now we refactor our app to use Vue.js with Vuetify. Because our app is relatively complex we need to use vuex as well.
On the development side we use GitHub as our main repo, Docker for local and server environment and Jenkins and AWS CodePipeline for Continuous Integration.
We decided to use AWS Lambda for several serverless tasks such as
- Managing AWS backups
- Processing emails received on Amazon SES and stored to Amazon S3 and notified via Amazon SNS, so as to push a message on our Redis so our Sidekiq Rails workers can process inbound emails
- Pushing some relevant Amazon CloudWatch metrics and alarms to Slack
A huge part of our continuous deployment practices is to have granular alerting and monitoring across the platform. To do this, we run Sentry on-premise, inside our VPCs, for our event alerting, and we run an awesome observability and monitoring system consisting of StatsD, Graphite and Grafana. We have dashboards using this system to monitor our core subsystems so that we can know the health of any given subsystem at any moment. This system ties into our PagerDuty rotation, as well as alerts from some of our Amazon CloudWatch alarms (we’re looking to migrate all of these to our internal monitoring system soon).
AWS Lambda Serverless Amazon CloudWatch Azure Functions Google Cloud Functions Node.js
In the last year or so, I moved all Checkly monitoring workloads to AWS Lambda. Here are some stats:
- We run three core functions in all AWS regions. They handle API checks, browser checks and setup / teardown scripts. Check our docs to find out what that means.
- All functions are hooked up to SNS topics but can also be triggered directly through AWS SDK calls.
- The busiest function is a plumbing function that forwards data to our database. It is invoked anywhere between 7000 and 10.000 times per hour with an average duration of about 179 ms.
- We run separate dev and test versions of each function in each region.
Moving all this to AWS Lambda took some work and considerations. The blog post linked below goes into the following topics:
- Why Lambda is an almost perfect match for SaaS. Especially when you're small.
- Why I don't use a "big" framework around it.
- Why distributed background jobs triggered by queues are Lambda's raison d'être.
- Why monitoring & logging is still an issue.
After looking for a way to monitor or at least get a better overview of our infrastructure, we found out that Grafana (which I previously only used in ELK stacks) has a plugin available to fully integrate with Amazon CloudWatch . Which makes it way better for our use-case than the offer of the different competitors (most of them are even paid). There is also a CloudFlare plugin available, the platform we use to serve our DNS requests. Although we are a big fan of https://smashing.github.io/ (previously dashing), for now we are starting with Grafana .
AWS Elastic Beanstalk is Amazon's excellent PaaS offering that saves me from a bunch of time and effort building and maintaining my own platform.
Rather than me having to DIY configuration for Amazon EC2 , Amazon CloudWatch , Amazon VPC , AWS Elastic Load Balancing (ELB) and more, AWS Elastic Beanstalk handles that for me, so I can focus on my app/service.
Amazon CloudWatch's Features
- Basic Monitoring for Amazon EC2 instances: ten pre-selected metrics at five-minute frequency, free of charge.
- Detailed Monitoring for Amazon EC2 instances: seven pre-selected metrics at one-minute frequency, for an additional charge.
- Amazon EBS volumes: eight pre-selected metrics at five-minute frequency, free of charge.
- Elastic Load Balancers: thirteen pre-selected metrics at one-minute frequency, free of charge.
- Amazon RDS DB instances: thirteen pre-selected metrics at one-minute frequency, free of charge.
- Amazon SQS queues: eight pre-selected metrics at five-minute frequency, free of charge.
- Amazon SNS topics: four pre-selected metrics at five-minute frequency, free of charge.
- Amazon ElastiCache nodes: twenty-nine pre-selected metrics at one-minute frequency, free of charge.
- Amazon DynamoDB tables: seven pre-selected metrics at five-minute frequency, free of charge.
- AWS Storage Gateways: eleven pre-selected gateway metrics and five pre-selected storage volume metrics at five-minute frequency, free of charge.
- Amazon Elastic MapReduce job flows: twenty-three pre-selected metrics at five-minute frequency, free of charge.
- Auto Scaling groups: seven pre-selected metrics at one-minute frequency, optional and charged at standard pricing.
- Estimated charges on your AWS bill: you can also choose to enable metrics to monitor your AWS charges. The number of metrics depends on the AWS products and services that you use, and these metrics are free of charge. Learn more about this option.