leaked aws keys detection and containment playbook

You get a PagerDuty alert: a developer pushed a JavaScript build that includes an AWS access key — the scanner flagged it as exposed. You checked the file, and the secret looks valid. The urgent questions are sharply practical: does the key still authenticate, what can it reach, and how quickly can you prevent an attacker turning that single secret into account takeover?

Immediate triage: prove the key is live without creating risk

When a leaked-key alert arrives, operate as though the key is already in attacker hands. The first step is a low-impact verification: make a non-destructive call that confirms the key still authenticates and reveals the caller identity. Use a short-lived environment and never paste the secret into long-lived shells or shared logs. Truffle Security’s recent analysis showed thousands of leaked keys still authenticate; assume any public key is dangerous until proven otherwise.

Run the AWS STS identity check from an isolated ephemeral host. If the key responds, capture the canonical account and principal type (`IAMUser`, `AssumedRole`, or `Root`) and the account ID. These values determine the containment path: a root key requires immediate root-level action; an IAM user key requires policy and role-path mapping. If the call fails with signature mismatch, still treat the secret as compromised and continue investigation: artifacts of exposure remain relevant for supply-chain and repository hygiene.

export AWS_ACCESS_KEY_ID=<LEAKED_KEY_ID>\nexport AWS_SECRET_ACCESS_KEY=<LEAKED_SECRET>\naws sts get-caller-identity --output json

# sample output
# { "UserId": "AIDIEXAMPLE:leaky-user", "Account": "123456789012", "Arn": "arn:aws:iam::123456789012:user/leaky-user" }

Next, check when the key was last used without making changes. The CLI call `get-access-key-last-used` reports the AWS service and region of last activity in readable form. This flag gives you immediate signal of whether the key has been used recently and which services to inspect first (S3, RDS, EC2, etc.).

aws iam get-access-key-last-used --access-key-id <LEAKED_KEY_ID>

# read-only result: shows "LastUsedDate" and "ServiceName" (e.g. "s3.amazonaws.com")

Map blast radius: resolve identity, attached policies and role paths

Verification is only step one. The critical defensive work is mapping what the principal can do. A key tied to an IAM user might inherit permissions from groups, inline policies, attached managed policies and trust relationships to roles. Truffle Security’s toolset and reporting call this “reach” — what the principal can access either directly or by assuming another role. You must compute the same reach before rotating keys so you avoid breaking production while containing attackers.

Use the IAM API to fetch user, group and inline policies, and then run a policy simulation to enumerate allowed actions. For roles, collect trust policies that mention the principal; an assumable deploy role gives the key more power than the original user. Document every Allow that includes wildcard actions or Resource: "*" — those are immediate priorities for remediation because they permit account-wide changes.

aws iam get-user --user-name leaky-user
aws iam list-attached-user-policies --user-name leaky-user
aws iam list-groups-for-user --user-name leaky-user
aws iam simulate-principal-policy --policy-source-arn arn:aws:iam::123456789012:user/leaky-user --action-names iam:CreateAccessKey sts:AssumeRole s3:DeleteObject

If you use a secrets-scanning vendor, they may provide an automated reach map. Truffle Security’s August research and product notes show how automated tooling accelerates identity mapping; if you don’t have that, export the policies and run the simulator queries yourself. Capture the set of high-risk APIs (`iam:CreateAccessKey`, `iam:PutRolePolicy`, `sts:AssumeRole`, `s3:PutObject`, `rds:DownloadDBLogFilePortion`, `ec2:RunInstances`) so your detection queries focus on meaningful telemetry.

Detection queries and SIEM hunts that catch abuse early

Attackers who possess valid keys typically start with discovery (List/Describe calls), then escalate into identity changes or data access. AWS GuardDuty may flag parts of this — but you need queries tuned to your business context. The AWS Security blog provides CloudWatch Logs Insights examples for correlating CloudTrail events; implement similar queries in CloudWatch, Athena, Splunk or your SIEM to catch the chain: GetCallerIdentity, high-rate List calls, followed by write-privelege IAM modifications or high-volume S3 GETs.

Run a CloudWatch Logs Insights query to surface recent unusual identity activity. Replace placeholders for your environment. This query flags recent GetCallerIdentity calls from new IPs and a burst of List/Describe APIs, a common early reconnaissance signature when a leaked key is probed.

fields @timestamp, eventName, sourceIPAddress, userIdentity.arn
| filter eventName = "GetCallerIdentity" or eventName like /List|Describe|GetObject/
| stats count() by userIdentity.arn, sourceIPAddress, bin(1h)
| sort by count() desc

Use Athena (SQL) for S3-delivered CloudTrail if you don’t stream to CloudWatch. Look for `CreateAccessKey`, `CreateUser`, `PutRolePolicy`, `AssumeRole` and `ConsoleLogin` events clustered in a short window. In Splunk or Elastic, a useful rule is: high cardinality of service names for one principal plus any IAM write-call equals high-severity investigation. Instrument thresholds to your traffic patterns to reduce noise.

# Athena (simplified)
SELECT userIdentity.arn, eventName, COUNT(*) as hits
FROM cloudtrail_logs
WHERE eventTime > current_timestamp - interval '24' hour
AND (eventName IN ('CreateAccessKey','CreateUser','PutRolePolicy','AssumeRole')
OR eventName LIKE '%List%'
)
GROUP BY userIdentity.arn, eventName
ORDER BY hits DESC
LIMIT 100;

Containment playbook: actions with order and why they matter

Containment must be surgical: remove the attacker’s ability to expand while preserving evidence and production continuity where possible. If the leaked key is root, disable it immediately. For IAM user keys, inactivate or delete the specific access key, then create a temporary policy that denies privilege escalation while you investigate. Record timestamps and export CloudTrail logs to an isolated bucket for forensic work before making wide-scoped deletions that could destroy evidence.

Concrete atomic actions you should script and run from a locked admin workstation follow. Each command is minimally disruptive to confirm and revoke the leaked credential. If you must preserve service continuity, rotate the credential with a short-lived role or temporary key prior to deleting the old key — but assume compromise and isolate the principal until you complete a full policy and activity review.

# Inactivate the access key (non-destructive)
aws iam update-access-key --user-name leaky-user --access-key-id <LEAKED_KEY_ID> --status Inactive

# or delete it (destructive)
aws iam delete-access-key --user-name leaky-user --access-key-id <LEAKED_KEY_ID>

After key revocation, hunt for backdoors: new users, new access keys, changes to role trust policies, newly launched instances with user-data that contains keys, or scheduled Lambda functions created in the same window. If suspicious EC2 or container activity appears, snapshot affected instances and collect memory if necessary. Notify AWS Support and, if data was exfiltrated, follow legal/notification obligations per the breach handling playbook.

Operational hardening: stop the next leak from becoming an incident

Cleaning a single leaked key is tactical; preventing the class of problem requires controls. Move workloads to short-lived credentials: prefer IAM Roles with OIDC federated identity and workload identity federation for CI/CD. Block use of long-lived access keys for human or machine identities via organization Service Control Policies (SCPs) that deny `iam:CreateAccessKey` except for a tightly-scoped automation account.

Automate secret scanning in CI pipelines and build artifacts, and enforce pre-commit and repository scanning. Truffle Security’s work shows public datasets like Hugging Face and container images are repeat sources of exposure; mandate automated scans of container registries and package registries prior to publishing. Add credential-age monitoring into routine risk reviews: AWS credential reports and Access Analyzer metadata let you find keys older than your rotation policy quickly.

Enforce ephemeral roles (OIDC/STS) for CI/CD and workloads.
Enable GuardDuty, Detective and Security Hub across accounts; centralize alerts into your SOC pipeline as described in AWS documentation.
Run monthly credential reports and alert on keys older than your policy (e.g., 90 days) and keys with full AdministratorAccess or attached inline wildcards.

Instrument budget alerts to detect sudden compute bills from cryptomining or mass snapshots; in the Beacon CRM incident a compromised key exposed backups in a bucket and led to a mass data export. Treat any public key exposure as an incident and enact the containment playbook—don’t wait for evidence of misuse before acting.

Finally, run tabletop exercises that start with a single leaked key. Measure your time to revoke, time to detect privilege escalation, and time to restore safe operations. If your mean time to revoke a leaked key is measured in days, you still have a large detection and process gap.

What to do next

If you received a leaked-key alert right now: (1) immediately run `sts get-caller-identity` from an isolated environment; (2) run `get-access-key-last-used`; (3) inactivate the key if it authenticates; (4) hunt for role-trust and IAM writes in CloudTrail for the preceding 48 hours; (5) rotate secrets and harden CI/CD and registry scans. Document each step and preserve logs for post-incident review.

Leaked access keys remain one of the fastest paths to account takeover because they bypass many perimeter controls. Make the verification, reach-mapping and containment steps above into a checklist in your runbook; automate the low-risk checks so analysts can spend time on the tricky policy and forensics questions that follow.

Hunting and Containing Live AWS Access Keys in Production