All articles in
Logging
AWS CloudTrail: Your One-Stop Shop for Comprehensive AWS Logging
The CloudTrail AWS service has come a long way since its launch in 2013. Originally designed to log API calls made within your AWS account, it tracks details about actions taken by users, roles, or AWS services, providing essential oversight for security and auditing. While its core objective remains unchanged, CloudTrail has evolved significantly over the past decade, adapting to the fast-paced demands of our industry and adding a host of powerful features to its arsenal. In this article, I will outline the key fundamentals of CloudTrail that are crucial for Cloud Security professionals to understand.
The three event types captured by CloudTrail
CloudTrail has capability to log three types of events or activities:
1. Management events
Management events, also known as control plane operations, capture administrative actions performed on AWS resources. These include activities like:
- Configuring security (e.g., IAM AttachRolePolicy API calls).
- Registering devices (e.g., EC2 CreateDefaultVpc).
- Setting routing rules (e.g., EC2 CreateSubnet).
- Setting up logging (e.g., CloudTrail CreateTrail).
Additionally, non-API events, such as ConsoleLogin (when a user signs in), and Service Events (events created by AWS services that are not directly triggered by a request to a public AWS API) are also captured.
Management events are enabled by default for 90 days in any AWS account and are visible in the CloudTrail AWS console in the Events History section.
2. Data events
Data events provide information about the resource operations performed on or in a resource. These are also known as data plane operations. Data events are often high-volume activities.
Example data events include:
- Amazon DynamoDB item-level API activity on tables (for example, PutItem, DeleteItem, and UpdateItem API operations).
- Amazon S3 object-level API activity (for example, GetObject, DeleteObject, and PutObject API operations) on objects in S3 buckets.
- AWS Lambda function execution activity (the Invoke API).
Learn more about Data Events with our Youtube video lab which will show you how log and analyze Data events using the example of delete-object API call in S3 bucket:
3. Insights events
CloudTrail Insights events detect and log unusual patterns in API call rates or error rates in your AWS account by analyzing management activity. These events are triggered when there are significant deviations from normal usage patterns, such as spikes in API activity or error rates. Insights events provide details like the API involved, error code, incident time, and statistics. Unlike regular CloudTrail logs, Insights events are only logged when abnormal activity occurs.
Examples include:
- A sudden increase in S3 deleteBucket API calls from 20 to 100 per minute.
- A drop in EC2 AuthorizeSecurityGroupIngress API calls from 20 to zero.
- A spike in AccessDeniedException errors for IAM operations.
What events are not captured by CloudTrail?
The following are some important sources of logs that are critical for Cloud Security professionals but are not captured by CloudTrail:
- Operating Systems and Applications: CloudTrail does not monitor logs from operating systems or applications running within AWS services (e.g., EC2 instances, containers). Examples include Linux/Windows system logs, web server access logs, and application logs.
- Managed Services: Certain managed services, like AWS RDS, may generate low-level database logs that are not captured by CloudTrail, such as database table-level operations.
- Network-Level Events: CloudTrail does not track network-level events like IP traffic, packet filtering, or VPC network flow logs. These events must be captured using other services, such as VPC Flow Logs.
To log such events, you need to rely on either other AWS service offerings or third-party solutions.
What does a CloudTrail event look like?
The three event types use a CloudTrail JSON log format. The log includes key fields such as eventTime, eventName, eventSource, userIdentity, and awsRegion. Optional fields like errorCode, errorMessage, requestParameters, and responseElements provide deeper insights into specific events. These records help track API calls, user actions, and service responses in AWS. Additionally, fields such as sharedEventID and eventID are useful for tracking related events across multiple AWS accounts.
Here’s a sample event:
{
"eventTime": "2023-05-10T22:00:00Z",
"eventName": "CreateBucket",
"userIdentity": {
"type": "IAMUser",
"userName": "test-user"
},
"awsRegion": "us-east-1",
"sourceIPAddress": "192.0.2.0"
}The two options for enabling CloudTrail
AWS CloudTrail is enabled by default for every new AWS account, but the default configuration only captures management events in the current region. These events are available for the last 90 days and are visible only in the CloudTrail console (Events History). For regulatory and compliance purposes, you will need to do much more than what is offered by default by AWS. Here are the two options that AWS provides today:
1. Create a Trail
In AWS CloudTrail, a trail is a configuration that delivers event logs to an S3 bucket and optionally to CloudWatch Logs and EventBridge. You can create single-Region or multi-Region trails, with the latter recommended for comprehensive logging across all AWS Regions. Multi-Region trails ensure consistent configurations and logging in all Regions. Multiple trails can be created to serve different user groups or purposes, and up to five trails are supported per Region. Trails can be organization-wide, logging events for all accounts in an AWS organization.
2. Create an Event Data store with CloudTrail Lake
Even though creating a trail and storing logs in an S3 bucket is an easy option, querying these logs for event filtering and aggregation can be cumbersome. CloudTrail Lake serves as an abstraction over CloudTrail trails, S3, and Athena, providing a platform for storing, querying, and managing your CloudTrail data. The Event Data Store is designed for long-term storage and offers advanced querying capabilities. It supports features like event filtering and aggregation, making it easier to derive insights from historical data without impacting the performance of real-time logging operations.
Understand why and when to use CloudTrail Lake with a demo of logging and analyzing data events for Lambda function and S3 bucket:
I hope this article simplified your understanding of AWS CloudTrail! Thank you so much for reading, and happy logging! 🎉
Follow us on:
Leave a Reply