Cloud IR Playbook
Cloud incidents don't look like on-premises incidents. There is no host memory to image, no file system to acquire, and often no network packet capture. The attacker moves through IAM roles and API calls, not through Windows authentication and lateral movement tools. If you bring your on-premises IR mindset to a cloud incident, you will miss most of what happened.
AWS GuardDuty fires at 2:47 AM with finding type "UnauthorizedAccess:IAMUser/ConsoleLoginSuccess.B" — an unusual console login for an IAM user named "ci-deploy." The finding notes the login came from an IP in Vietnam; the CI/CD system that normally uses this user authenticates from a US-based GitHub Actions runner.
The security engineer on-call pulls the CloudTrail logs and finds that in the 35 minutes since the GuardDuty alert, the ci-deploy user has: listed all S3 buckets, listed all EC2 instances, created a new IAM user named "backup-service" with administrator access, and created access keys for the new user. The attacker now has a fresh, high-privilege credential that GuardDuty has no baseline for and will not recognize as unusual.
The ci-deploy IAM user's access key was leaked in a public GitHub repository commit 11 days ago. The key has been in an attacker's hands for 11 days. The question is no longer whether the attacker has been using it — it's what they've done with 11 days of access to a CI/CD account with significant AWS permissions.
Why Cloud IR Differs from On-Premises IR
Cloud infrastructure presents a fundamentally different forensic and response environment. The underlying compute is not owned by you, there is no physical access to the hardware, and the attack surface is primarily API-based rather than network-based. These differences require different tools, different skills, and different investigative approaches.
| Dimension | On-premises IR | Cloud IR |
|---|---|---|
| Primary attack surface | Network services, endpoints, Active Directory | IAM (Identity and Access Management) — compromised credentials or overprivileged roles |
| Primary evidence source | Memory images, disk images, EVTX event logs, network packet captures | Control plane logs (CloudTrail, Azure Activity Log, GCP Audit Logs) — API call history |
| Containment mechanism | Network isolation (firewall rules, VLAN), host isolation (EDR) | IAM credential revocation (disable access key, revoke role), security group isolation, instance quarantine |
| Persistence mechanisms | Scheduled tasks, registry Run keys, services, WMI subscriptions | New IAM users/roles, cross-account trust relationships, Lambda functions, EC2 instance user-data backdoors, SSM parameter store modified payloads |
| Log availability | Depends on your logging configuration; may be limited | CloudTrail logs all API calls by default (though there are gaps: read API events require explicit configuration) |
| Lateral movement | Pass-the-hash, Kerberoasting, PsExec, WMI remote execution | IAM role assumption (assuming a higher-privileged role), EC2 metadata service credential theft (IMDSv1), cross-account trust abuse |
CloudTrail as the Primary Forensic Source
AWS CloudTrail records every API call made in your AWS account — who made it, from where, what action was taken, and what resources were affected. For cloud IR, CloudTrail is the equivalent of the Windows Security Event Log: the authoritative record of activity in the environment. The first step in any AWS IR is pulling the relevant CloudTrail events.
CloudTrail Investigation Framework ═══════════════════════════════════════════════════════════════════ STEP 1: Establish the Compromised Identity ───────────────────────────────────────────────────────────────── Identify: which IAM user, role, or access key was compromised? In this scenario: IAM user "ci-deploy" with access key AKIA... STEP 2: Find the Earliest Malicious Activity ───────────────────────────────────────────────────────────────── Query CloudTrail for all events from the compromised principal, going back as far as logs are available (default retention: 90 days in CloudTrail; longer if logs are exported to S3). Ordering tip: sort ascending by time to see the attack progression from earliest to latest action. STEP 3: Reconstruct the Attack Chain ───────────────────────────────────────────────────────────────── IAM-related events (reconnaissance + privilege escalation): → iam:ListUsers, ListRoles, ListPolicies — enumeration → iam:GetUser, GetRole — targeting specific identities → iam:CreateUser — new backdoor user → iam:AttachUserPolicy — granting permissions to new user → iam:CreateAccessKey — persistent credential creation → iam:AssumeRole — lateral movement to higher-privileged role S3-related events (data access): → s3:ListAllMyBuckets — enumeration → s3:GetBucketAcl, GetBucketPolicy — checking permissions → s3:GetObject — actual file download (if "data events" logging enabled) → s3:PutObject — potential staging for exfiltration EC2-related events (infrastructure access): → ec2:DescribeInstances — enumeration → ec2:ModifyInstanceAttribute — potential backdoor in user-data → ec2:DescribeSecurityGroups — network topology enumeration → ssm:SendCommand — remote code execution on EC2 via SSM STEP 4: Identify New Identities and Credentials Created ───────────────────────────────────────────────────────────────── Critical: the attacker creates new credentials to maintain access even after the compromised credential is revoked. Search for: iam:CreateUser, iam:CreateRole, iam:CreateAccessKey where the identity making the call is the compromised principal. All new identities created during the compromise window are suspect.
In AWS, a single API call may have a complex principal chain: the human user authenticated with access key A, assumed Role B via sts:AssumeRole, and then that role made an API call using temporary credentials. CloudTrail records all of this, but it's easy to miss if you only look at the top-level user. When the attacker creates new IAM users or assumes roles, their downstream actions will appear under those new identities — not under the original compromised user. The investigation must follow the full tree of identities created during the compromise, not just the original entry point.
IAM Forensics: Finding What the Attacker Built
The most dangerous thing an attacker does with cloud access is not what they steal — it's the persistence they establish. A compromised IAM credential can be revoked; a backdoor IAM user with administrator privileges that you missed will re-establish access after you think the incident is over.
| Persistence type | How attacker establishes it | How to find it | How to remove it |
|---|---|---|---|
| New IAM user with high privileges | iam:CreateUser + iam:AttachUserPolicy (AdministratorAccess) | List all IAM users; filter for users created during compromise window | Disable all access keys + delete the user |
| New access key on existing user | iam:CreateAccessKey for an existing user (not the compromised user) | Audit all IAM access key creation events in CloudTrail during compromise window | Revoke the specific access key |
| Cross-account role trust modification | iam:UpdateAssumeRolePolicy — adding attacker's external account to trust policy | CloudTrail iam:UpdateAssumeRolePolicy events; review all role trust policies | Remove attacker's account from trust policy |
| Lambda backdoor function | lambda:CreateFunction — deploys function with attacker's code that exfiltrates data or maintains access | CloudTrail lambda:CreateFunction events; review all Lambda functions for unusual code or triggers | Delete the Lambda function; remove associated IAM role |
| EC2 instance with modified user-data | ec2:ModifyInstanceAttribute — adds reverse shell to instance startup user-data script | Review EC2 user-data for all instances; check ec2:ModifyInstanceAttribute events | Remove malicious user-data; terminate and re-provision instance |
Cloud Resource Isolation and Containment
Cloud containment is primarily IAM-based. Unlike on-premises IR where you can physically pull a network cable or use EDR to isolate a host, cloud containment means revoking the identities and credentials the attacker has access to, and network-isolating any compromised compute resources.
AWS Containment Playbook: Compromised IAM Credential
═══════════════════════════════════════════════════════════════════
STEP 1: Revoke the Compromised Credential
─────────────────────────────────────────────────────────────────
Disable the specific access key (not the entire user account —
disabling the key while leaving the user allows investigation
without destroying evidence):
aws iam update-access-key --access-key-id AKIA... --status Inactive
Effect: the original compromised credential can no longer authenticate.
Does NOT affect: any new credentials the attacker already created.
STEP 2: Revoke All Active Sessions from the Compromised Identity
─────────────────────────────────────────────────────────────────
If the user has active console sessions or temporary STS tokens:
Attach an inline deny-all policy to immediately invalidate all
existing session tokens for that user:
aws iam put-user-policy --user-name ci-deploy
--policy-name ForceLogout
--policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*"}]}'
STEP 3: Revoke All Attacker-Created Credentials
─────────────────────────────────────────────────────────────────
For each new IAM user/key created by the attacker during the incident:
→ Disable all access keys
→ Remove all attached policies
→ Delete the user
For each role trust policy modified:
→ Restore the original trust policy
Do this simultaneously to prevent the attacker from using any
remaining credential while you revoke the others.
STEP 4: Isolate Compromised EC2 Instances
─────────────────────────────────────────────────────────────────
Replace the instance's security group with an isolation group
(allows no inbound or outbound except for your forensic access):
aws ec2 modify-instance-attribute --instance-id i-...
--groups sg-isolation-group
Do NOT terminate — this destroys evidence. Isolate first, investigate.
Evidence Preservation in Cloud Environments
Cloud evidence is volatile in ways on-premises evidence is not: EC2 instances can be terminated (losing all in-memory state and unpreserved disk data), CloudTrail logs may roll off after 90 days, and S3 objects can be deleted. Evidence preservation in cloud IR must happen quickly and deliberately.
| Evidence type | Default retention | Preservation action |
|---|---|---|
| CloudTrail management event logs | 90 days in CloudTrail console | Export to S3 immediately; enable S3 Object Lock on the export bucket to prevent deletion |
| CloudTrail data events (S3 object access) | Only available if explicitly configured; same 90-day retention | Export same as above; note this may be missing if not configured before the incident |
| VPC Flow Logs | Depends on your CloudWatch Logs retention setting; may be as little as 7 days | Export flow logs for the incident window to S3 immediately |
| EC2 instance memory | Lost if instance is stopped or terminated | Take EC2 instance snapshot (AMI or EBS snapshot) before any shutdown; use SSM to run a memory acquisition tool if possible |
| EC2 disk | Lost if instance is terminated (EBS volumes may or may not be deleted on termination) | Create EBS snapshot immediately; apply snapshot tags for evidence tracking |
| S3 object versions | Lost if versioning is not enabled and object is overwritten/deleted | Enable S3 versioning retroactively (protects future state, not historical); use S3 event notifications to capture access events going forward |
AWS, Azure, and GCP operate under a shared responsibility model: the cloud provider is responsible for the security of the cloud infrastructure (hardware, hypervisor, physical facilities); you are responsible for security in the cloud (your IAM policies, your instance configurations, your application code). This boundary matters for IR: if you suspect the cloud provider's infrastructure is compromised, contact them through your enterprise support channel and escalate to your AWS TAM or Azure support team. But the vast majority of cloud incidents are in your zone of responsibility — IAM credentials leaked, overprivileged roles exploited, misconfigured S3 buckets accessed. Don't wait for the cloud provider to detect and respond to incidents in your environment — they won't unless it affects their infrastructure.
Azure-Specific: Entra ID and Activity Log Forensics
Azure IR uses different tooling but the same principles: control plane logs (Azure Activity Log and Entra ID audit logs) are the primary forensic source, and identity compromise is the primary attack vector.
Azure IR Key Evidence Sources
═══════════════════════════════════════════════════════════════════
Entra ID (formerly Azure AD) Sign-In Logs:
→ All authentication events for all users and service principals
→ Includes: IP, location, MFA result, conditional access policy result,
app accessed, risk level (Identity Protection)
→ Retention: 30 days (Entra ID Free/P1); 90 days (P2)
→ Export to Log Analytics / Storage Account for longer retention
Entra ID Audit Logs:
→ All changes to Entra ID objects: user creation, role assignment,
group membership changes, app registration changes, consent grants
→ Equivalent to AWS CloudTrail IAM events
→ Key events: Add member to role, Add app role assignment to service
principal, Consent to application
Azure Activity Log:
→ All operations on Azure resources (VMs, storage, networking, etc.)
→ Equivalent to AWS CloudTrail management events
→ Retention: 90 days; export to Log Analytics for longer
→ Key events: Microsoft.Compute/virtualMachines/write (VM modification),
Microsoft.Authorization/roleAssignments/write (privilege escalation)
Microsoft 365 Unified Audit Log:
→ All M365 activity: Exchange, SharePoint, Teams, OneDrive
→ For BEC investigation: see Chapter 17 for specific event types
→ Retention: 90 days (E3); 1 year (E5)
Q & A
Q: The ci-deploy access key was leaked on GitHub 11 days ago. How do you determine what the attacker did in those 11 days before the GuardDuty alert?
CloudTrail is your primary source. Query all API events from the ci-deploy principal going back 11+ days. Sort ascending by time to see the earliest activity. You're looking for the point where activity shifted from expected CI/CD patterns (build-related API calls from US-based IPs during business hours) to unusual patterns (enumeration queries, IAM changes, unusual geographies, off-hours activity). The forensic challenge is distinguishing legitimate CI/CD activity from attacker activity when both use the same credential. Indicators of attacker activity in CI/CD credentials: calls from IPs outside your CI/CD infrastructure ranges, IAM enumeration (which CI/CD automation doesn't typically do), off-hours activity patterns, and resource enumeration queries broader than CI/CD automation would normally make. Cross-reference with your CI/CD pipeline logs to identify which API calls your automation actually made — any calls NOT in your CI/CD logs are attacker-initiated.
Q: You've revoked the compromised access key and the attacker's created credentials. Three hours later, GuardDuty fires again from a different user. How did they get back in?
This suggests either an incomplete persistence sweep (you missed a created credential or role trust modification) or a different initial access vector that was not the compromised access key. Immediate steps: (1) Look at the new GuardDuty finding — what identity is it pointing to? Is this a user/role you recognize from your persistence sweep? (2) Check CloudTrail for any IAM creation events that occurred during the incident window that you may have missed. Pay specific attention to: new IAM roles (not just users), cross-account trust modifications, and Lambda functions that could be used for persistence. (3) Check if the new access is from an EC2 instance metadata service — an EC2 instance with a high-privilege instance profile can be used by the attacker if they have code execution on that instance. (4) Consider whether there is a second initial access vector — perhaps the leaked GitHub key was not the only compromised credential.
Q: How do you check if the attacker used the compromised access key to access S3 buckets and exfiltrate data?
Two approaches depending on your logging configuration: (1) If S3 data events are enabled in CloudTrail: query for s3:GetObject events from the ci-deploy principal across all S3 buckets during the compromise window. This shows exactly which objects were downloaded. (2) If S3 data events are NOT enabled (the default — data events must be explicitly configured): you have a significant forensic gap. You can see that the attacker listed your buckets (s3:ListAllMyBuckets and s3:ListBucket are management events, always logged), but you cannot determine which specific objects they accessed. In this case: (a) Treat all data in any bucket the attacker had read access to as potentially compromised — assume worst case. (b) Use VPC Flow Logs to check for large outbound data transfers during the window — if the attacker downloaded significant data, it will show as large flows to external IPs. (c) Going forward, enable CloudTrail data events for critical S3 buckets — this is the lesson learned from this gap.
Q: The incident involved a CI/CD credential. What specific detection rules would catch this type of incident faster next time?
Five detection rules that would have fired before or shortly after the GuardDuty alert: (1) Any API call from the ci-deploy user from an IP outside the known CI/CD infrastructure range — the CI/CD system always operates from a fixed set of GitHub Actions runner IPs. (2) Any IAM creation event (CreateUser, CreateRole, CreateAccessKey) performed by a service account — service accounts should not need to create IAM identities. (3) Any IAM enumeration calls (ListUsers, ListRoles, GetAccountAuthorizationDetails) from non-human identities — CI/CD automation doesn't enumerate IAM. (4) Cross-account role assumption from unexpected external accounts. (5) Any GitHub Actions workflow modification in the repository where the key was leaked — if someone modified the CI/CD pipeline to extract the key, that modification is in git history.
Q: AWS sent a "compromised credentials" email notification 2 days after the key was leaked. Why so slow, and can we rely on this?
AWS's abuse detection for leaked credentials operates by scanning public code repositories and credential databases for known AWS access key patterns. The detection latency (2 days in this case, sometimes hours, sometimes never) depends on: how quickly the key appeared in a public source that AWS monitors, whether the key was immediately masked or rotated before detection, and the priority of the automated detection scan. You should not rely on AWS notification as your primary detection for leaked credentials — it's a safety net, not a reliable early warning system. Better approach: implement access key monitoring in your own environment. Use Macie to scan repositories and internal documents for credential patterns. Configure GitHub secret scanning (free on all public and private repos) — it notifies the repo owner within seconds of a key being committed to a monitored repository and can automatically revoke the key if configured. The credential should have been revoked automatically within minutes, not detected by AWS 2 days later.