CloudTrail Auditing

observability aws security audit claude-curated

CloudTrail is the AWS service that records every API call made against an account: who called what, from where, when, with which parameters, and whether it succeeded. It is the backbone of any security posture, incident investigation, or compliance regime on AWS. Without it you cannot answer “who deleted that bucket” or “did anyone touch the IAM policy yesterday.”

Event types

CloudTrail distinguishes three classes of event, with different cost and coverage profiles.

Type	Default	Cost	Captures
Management events	On	Free for first trail	Control-plane API calls (create, modify, delete resources, IAM, configuration)
Data events	Off (opt-in)	Per-event charge	High-volume data-plane calls — S3 GetObject, Lambda Invoke, DynamoDB GetItem
Insights events	Off (opt-in)	Flat per-trail charge	Anomaly detection over write-API call volume

Management events are what most teams need 90% of the time. Data events are essential when investigating data exfiltration or proving non-access of regulated data, but they fire at request rate and the bill grows fast — a busy S3 bucket can produce billions of GetObject events per month. Scope data events tightly: only the buckets, functions, or tables that warrant audit, not the whole account.

Trail destinations

A trail can deliver events to:

S3 — the canonical long-term destination. JSON gzip files, partitioned by date and region.
CloudWatch Logs — for near-real-time tailing and metric filters.
EventBridge — for fan-out to Lambda, Step Functions, or third-party SIEMs.

Most production setups use all three: S3 for retention and forensics, CloudWatch for human investigation, EventBridge for automated detection.

A robust auditing setup

A defensible CloudTrail configuration looks roughly like this:

Organisation-wide trail

In AWS Organizations, a single org trail enabled at the management account captures every member account automatically. New accounts are covered the day they join. This avoids the failure mode where a forgotten account has no trail and an attacker pivots through it invisibly.

Multi-region

Set the trail to record events from all regions, not just the home region. Attackers routinely spin up resources in unused regions specifically to avoid detection. A multi-region trail catches API calls in regions the team has never deployed to.

Log file integrity validation

Enable log file integrity validation so CloudTrail produces a SHA-256 digest file every hour, signed by a known KMS key. Tampering with delivered logs becomes detectable: the digest chain breaks and the validation tool reports the gap. Without this, a sufficiently privileged attacker could quietly edit the S3 logs after the fact.

Dedicated audit account

Deliver the logs to an S3 bucket in a separate, locked-down audit account. The accounts that produce events should not be able to delete the records of those events. Combined with S3 Object Lock in compliance mode, the logs become genuinely immutable for the retention period — no IAM principal, including root, can delete them.

Least-privilege reader

The IAM principals that read logs (security analysts, automated detectors) should have read-only access (least privilege). Write and delete on the audit bucket should be restricted to the CloudTrail service principal itself.

Querying

A trail is only useful if you can ask questions of it.

CloudTrail Lake

CloudTrail Lake is the managed query layer: events go into a queryable event-data-store and you write SQL against them. No infrastructure to set up, no Glue catalog to maintain. Costs scale with ingest and query volume. Best fit when the audit team wants ad-hoc investigation without a data engineering project.

Athena over S3

The cheaper, more flexible option: define a Glue table over the trail S3 bucket and query with Athena (see Athena Query Patterns). You get full SQL, partition pruning, and integration with whatever data tooling already exists. Requires a partition-projection setup or a crawler to keep partitions current.

SELECT eventTime, userIdentity.arn, eventName, requestParameters
FROM cloudtrail_logs
WHERE eventTime > current_timestamp - interval '1' day
  AND eventName IN ('DeleteBucket', 'PutBucketPolicy')
ORDER BY eventTime DESC;

Common audit queries

“Who deleted resource X?” Filter by eventName = 'Delete...' and the resource ARN in requestParameters or resources.
“Who assumed role Y?” Filter by eventName = 'AssumeRole' and requestParameters.roleArn.
“Was the root user used?” Filter by userIdentity.type = 'Root'. Should almost always be empty.
“Did this IP range access anything?” Filter by sourceIPAddress.
“Which IAM policies changed yesterday?” Filter by eventSource = 'iam.amazonaws.com' and write-class event names.

Alerting on suspicious patterns

Some events are worth a pager regardless of context. Common high-signal alerts:

Root user activity — root should be sealed. Any sign-in or API call as root is investigated.
IAM policy or role changes by unexpected principals.
Security group changes opening 0.0.0.0/0 on sensitive ports.
CloudTrail itself being disabled or modified — a classic attacker tradecraft step.
KMS key disable or deletion.
Console login from a new country for privileged users.

Hook these via EventBridge rules that match the relevant event pattern and route to PagerDuty, Slack, or the SIEM (see also Event-Driven Architecture).

Retention

The CloudTrail console keeps the last 90 days of management events for free, queryable through Event History. Beyond that, retention lives in S3:

Standard tier for the first 90 days for fast investigation.
Glacier Instant Retrieval or Glacier Flexible Retrieval for compliance retention (1, 3, 7 years depending on regime).
Lifecycle policies move objects between tiers automatically.

For regulated workloads, Object Lock retention should match the longest compliance window the business is bound by — typically 7 years for financial services, longer for healthcare in some jurisdictions.

mgarden

Explorer