Need advice about which tool to choose?Ask the StackShare community!
AWS CloudTrail vs AWS Config: What are the differences?
Introduction:
AWS CloudTrail and AWS Config are both services provided by Amazon Web Services (AWS) that help organizations in managing and auditing their AWS resources. While they have some similarities and overlap in functionality, there are several key differences between the two services.
Data Collection: AWS CloudTrail focuses on auditing and monitoring AWS API activity. It captures detailed information about API calls made within an AWS account, including who made the call, when it was made, and which resources were affected. This data is essential for security and compliance purposes, as it provides an audit trail of all API activity within an AWS environment. On the other hand, AWS Config focuses on recording the configuration history of AWS resources. It captures the configuration details of each resource and maintains a timeline of all configuration changes. This enables users to assess and audit the state of their AWS resources at any point in time.
Scope of Monitoring: AWS CloudTrail provides a comprehensive and centralized view of API activity across an entire AWS account. It captures API calls made to various AWS services, including EC2, S3, IAM, and RDS, among others. In contrast, AWS Config provides resource-level monitoring, focusing on the configuration of individual AWS resources rather than API activity. It records configuration changes made to resources like EC2 instances, security groups, and S3 buckets, allowing users to track and manage resource-level changes specifically.
Real-Time Monitoring: AWS CloudTrail provides real-time monitoring capabilities, allowing users to receive near real-time notifications of API activity. This enables organizations to respond quickly to any suspicious or unauthorized activity within their AWS infrastructure. AWS Config, on the other hand, primarily operates on a periodic basis (typically every few minutes) to capture the configuration state of resources. It can also be triggered by specific events such as configuration changes or periodic snapshots, but it does not offer continuous, real-time monitoring capabilities like AWS CloudTrail.
Integration with Other AWS Services: AWS CloudTrail integrates with several other AWS services, such as AWS CloudWatch, AWS EventBridge, and AWS Lambda. This integration allows users to analyze and respond to CloudTrail events in real-time, triggering automated actions or alerts based on specific conditions. AWS Config also provides integration capabilities with services like AWS CloudWatch, which can be used to monitor the configuration changes captured by AWS Config. However, the level of integration is more focused on monitoring and analysis rather than triggering real-time actions based on specific events.
Retention Period: AWS CloudTrail has a default retention period of 90 days for log files, which can be extended up to 365 days. This ensures that the API activity logs are available for a significant period, allowing organizations to meet their compliance and auditing requirements. On the other hand, AWS Config has a default retention period of 7 days for configuration history, which can be increased up to 7 years. While both services provide options for extended retention periods, their default and maximum retention periods differ significantly.
Cost and Pricing Model: AWS CloudTrail incurs charges based on the volume of API events recorded, the number of trails configured, and any data events captured. Pricing for AWS Config, on the other hand, is based on the number of AWS resources recorded and the number of configuration snapshots taken. Comparing the cost of the two services can be subjective depending on factors such as the size of the environment, the volume of API activity, and the complexity of resource configurations.
**In Summary, AWS CloudTrail focuses on auditing and monitoring AWS API activity in real-time, while AWS Config records the configuration history of AWS resources to enable assessment and auditing of resource-level changes. CloudTrail provides a comprehensive view of API activity across an AWS account, while Config focuses on resource-level monitoring. CloudTrail has real-time monitoring capabilities, while Config operates more on a periodic basis. Both services integrate with other AWS services but with different levels of functionality. CloudTrail has a longer default retention period for log files, while Config offers a longer default retention period for configuration history. Finally, the cost and pricing models of the two services vary based on different factors.
We would like to detect unusual config changes that can potentially cause production outage.
Such as, SecurityGroup new allow/deny rule, AuthZ policy change, Secret key/certificate rotation, IP subnet add/drop. The problem is the source of all of these activities is different, i.e., AWS IAM, Amazon EC2, internal prod services, envoy sidecar, etc.
Which of the technology would be best suitable to detect only IMP events (not all activity) from various sources all workload running on AWS and also Splunk Cloud?
For continuous monitoring and detecting unusual configuration changes, I would suggest you look into AWS Config.
AWS Config enables you to assess, audit, and evaluate the configurations of your AWS resources. Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations. Here is a list of supported AWS resources types and resource relationships with AWS Config https://docs.aws.amazon.com/config/latest/developerguide/resource-config-reference.html
Also as of Nov, 2019 - AWS Config launches support for third-party resources. You can now publish the configuration of third-party resources, such as GitHub repositories, Microsoft Active Directory resources, or any on-premises server into AWS Config using the new API. Here is more detail: https://docs.aws.amazon.com/config/latest/developerguide/customresources.html
If you have multiple AWS Account in your organization and want to detect changes there: https://docs.aws.amazon.com/config/latest/developerguide/aggregate-data.html
Lastly, if you already use Splunk Cloud in your enterprise and are looking for a consolidated view then, AWS Config is supported by Splunk Cloud as per their documentation too. https://aws.amazon.com/marketplace/pp/Splunk-Inc-Splunk-Cloud/B06XK299KV https://aws.amazon.com/marketplace/pp/Splunk-Inc-Splunk-Cloud/B06XK299KV
While it won't detect events as they happen a good stop gap would be to define your infrastructure config using terraform. You can then periodically run the terraform config against your environment and alert if there are any changes.
Consider using a combination of Netflix Security Monkey and AWS Guard Duty.
You can achieve automated detection and alerting, as well as automated recovery based on policies with these tools.
For instance, you could detect SecurityGroup rule changes that allow unrestricted egress from EC2 instances and then revert those changes automatically.
It's unclear from your post whether you want to detect events within the Splunk Cloud infrastructure or if you want to detect events indicated in data going to Splunk using the Splunk capabilities. If the latter, then Splunk has extremely rich capabilities in their query language and integrated alerting functions. With Splunk you can also run arbitrary Python scripts in response to certain events, so what you can't analyze and alert on with native functionality or plugins, you could write code to achieve.
Well there are clear advantages of using either tools, it all boils down to what exactly are you trying to achieve with this i.e do you want to proactive monitoring or do you want debug an incident/issue. Splunk definitely is superior in terms of proactively monitoring your logs for unusal events, but getting the cloudtrail logs across to splunk would require some not so straight forward setup (Splunk has a blueprint for this setup which uses AWS kinesis/Firehose). Cloudtrail on the other had is available out of the box from AWS, the setup is quite simple and straight forward. But analysing the log could require you setup Glue crawlers and you might have to use AWS Athena to run SQL Like query.
Refer: https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html
In my personal experience the cost/effort involved in setting up splunk is not worth it for smaller workloads, whereas the AWS Cloudtrail/Glue/Athena would be less expensive setup(comparatively).
Alternatively you could look at something like sumologic, which has better integration with cloudtrail as opposed to splunk. Hope that helps.
I'd recommend using CloudTrail, it helped me a lot. But depending on your situation I'd recommed building a custom solution(like aws amazon-ssm-agent) which on configuration change makes an API call and logs them in grafana or kibana.
Pros of AWS CloudTrail
- Very easy setup7
- Good integrations with 3rd party tools3
- Very powerful2
- Backup to S32
Pros of AWS Config
- Backed by Amazon4
- One stop solution2
Sign up to add or upvote prosMake informed product decisions
Cons of AWS CloudTrail
Cons of AWS Config
- Not user friendly2
