AI Agent Risk Scanner Found Our Own Wildcard IAM Policy

TL;DR: We built an AI agent risk scanner as course content for the Secure AI Builder Bootcamp, then pointed its new --workspace mode at our own bootcamp infrastructure instead of a hand-typed description. It found a real wildcard kms:Decrypt policy in both the student starter kit and our deployed reference lab. We fixed the deployed infrastructure end-to-end, then backported the fix into the course so students never hit the same gap.

Tonight’s session started as a feature add, not a security audit. The Module 6 agent-risk-calculator template — one of four AI agent templates students build in the Secure AI Builder Bootcamp — took a plain-text description of an agent and ran a STRIDE threat model against it. That’s useful for teaching the framework, but it’s a toy input. Real agents aren’t a paragraph; they’re a codebase. So the plan was to add a --workspace <path> mode: point the scanner at an actual project directory, read the Terraform, the Python, the system prompt, and let it infer what the agent actually does instead of asking a student to describe it by hand.

Once --workspace mode worked, the obvious next move was to dogfood it. We ran it against our own bootcamp reference lab infrastructure — the same AWS setup every student in the cohort is running right now. That’s when the AI agent risk scanner stopped being a course exercise and started finding real bugs.

What the Scanner Found

The finding was a wildcard IAM policy: both the student starter-workspace/terraform/main.tf and the deployed ABT reference lab granted kms:Decrypt on Resource: "*".

# Before
resource "aws_iam_role_policy" "bootcamp_permissions" {
  policy = jsonencode({
    Statement = [
      {
        Sid      = "KMSDecrypt"
        Effect   = "Allow"
        Action   = ["kms:Decrypt"]
        Resource = "*"
      },
      # ...
    ]
  })
}

The root cause wasn’t sloppiness — it was a missing prerequisite. The /bootcamp/* secrets in SSM Parameter Store were only ever encrypted with the shared AWS-managed key, alias/aws/ssm. That key isn’t scopable to one project; it’s the same key every default-encrypted SSM parameter in the account uses. An IAM policy that says “allow decrypt with this key” on a shared key has no choice but to mean “allow decrypt anything encrypted with this key, account-wide.” The Resource: "*" wasn’t a shortcut somebody took — it was the only honest way to express “give this role decrypt access” when there was no dedicated key to name instead.

That’s a pattern worth naming for anyone doing IAM least privilege work: a wildcard resource in a policy statement is sometimes a lazy default, but sometimes it’s a symptom that an earlier resource — here, the encryption key — was never scoped in the first place. You can’t write a tight policy against a shared resource. Fix the upstream resource, and the policy tightens on its own.

Fixing the Deployed Infrastructure

We created a dedicated KMS key, aliased alias/bootcamp-secrets, in the ABT AWS account, re-encrypted /bootcamp/anthropic-key under it, and scoped the reference lab’s IAM policy to that key’s exact ARN:

# After
data "aws_kms_alias" "bootcamp_secrets" {
  name = "alias/bootcamp-secrets"
}

resource "aws_iam_role_policy" "bootcamp_permissions" {
  policy = jsonencode({
    Statement = [
      {
        Sid      = "KMSDecrypt"
        Effect   = "Allow"
        Action   = ["kms:Decrypt"]
        Resource = data.aws_kms_alias.bootcamp_secrets.target_key_arn
      },
      # ...
    ]
  })
}

terraform apply, verified the EC2 instance role could still decrypt the parameter, verified nothing else in the account needed the old wildcard grant. Standard least-privilege remediation — the interesting part isn’t the fix, it’s how it got applied in two places instead of one.

Closing the Loop for Students

A fix that only lands in our deployed infrastructure teaches nothing. The same wildcard was sitting in the student starter-workspace Terraform, which meant every student building along the course was about to reproduce the exact gap we’d just found in our own account — before they’d ever get a chance to notice it.

So the fix went into the course material, not just the infrastructure:

• Module 3, Lesson 4 (secrets management, before students ever store their first secret) now has a “create your own KMS key” step — aws kms create-key + aws kms create-alias, aliased alias/bootcamp-secrets, done before the first aws ssm put-parameter call. Students who follow the lesson in order never hit the shared-key trap at all.
• Module 5, Lesson 3 (the Terraform walkthrough for deploying an agent to EC2) and the starter-workspace terraform/main.tf both now look the key up via data.aws_kms_alias and scope kms:Decrypt to its ARN, matching the pattern we just applied to our own reference lab.

The fix shipped as one commit touching both the infrastructure and the course content — the Terraform change, the deployed policy update, and the two lesson updates went out together, so there’s no window where the course teaches a pattern the reference lab itself doesn’t follow.

Extending the Scanner Itself

While we were in the agent-risk-calculator code, we extended it in two more ways that came directly out of using it for real:

• OWASP LLM Top 10 mapping. Each STRIDE finding now carries a paired OWASP LLM Top 10 code, matching what Module 6’s threat-log exercise already required students to produce by hand. The scanner was teaching a framework it wasn’t itself using.
• --workspace <path> mode, the feature that made this whole session possible — reading a real agent’s project files (Terraform, Python, system prompts, README) and inferring Purpose, Tools, and Data access from them, instead of requiring a hand-typed description that a student (or we) could get lazy about writing accurately.

The Workflow Worth Repeating

The sequence here is the actual lesson: build a security tool as course content, use Claude Code to run it against your own real infrastructure instead of a synthetic example, let it surface a real finding, then fix the finding in two places — the live system and the material that teaches the pattern — in the same change. A scanner that only ever sees toy inputs during development never gets tested against the mess real infrastructure accumulates. Ours had been checked into the course for weeks before it saw a real Terraform file, and the first real file it saw had a live wildcard IAM grant in it.

The AI agent risk scanner didn’t get smarter in some abstract sense from this session. It got useful, because we finally asked it to look at something with real stakes attached.

Key Takeaways

• A wildcard IAM resource is sometimes a symptom of an unscoped upstream resource, not a lazy default — here, a shared KMS key meant kms:Decrypt on Resource: "*" was the only honest way to write the policy until a dedicated key existed
• Building a security tool and never running it against your own real infrastructure means it never gets tested against anything messier than the examples you wrote for it
• When you fix an infrastructure gap that’s also taught in course material, ship both fixes in the same change — a lesson that doesn’t match the reference deployment teaches students to distrust the reference deployment
• A --workspace-style mode that reads real project files instead of requiring a hand-typed description turns an AI agent risk scanner from a classroom exercise into something you’d actually run before a deploy

---

This is part of our ongoing build-in-public series documenting the Secure AI Builder Bootcamp — a hands-on course teaching AI-first builders to build and secure their own agents on real cloud infrastructure.