Blast Radius Thinking: The Mental Model I Apply Daily
Blast Radius Thinking: The Mental Model I Apply Daily
There’s a concept from incident response that I spent years applying to network segmentation, breach containment, and IAM policy design. It never had a formal name on my team — we just called it asking “how bad can this get?” Turns out there’s a cleaner term for it: blast radius thinking.
The mental model is simple. Before you deploy anything — a firewall rule, an IAM policy, an API integration — you ask one question: if this goes wrong, what’s the maximum damage this component can cause? Then you scope its access to the minimum required to make that maximum acceptable.
I picked up the formal framing from a paper on fault-tolerant distributed systems design. I applied it to an AI agent the same afternoon. The change it forced was immediate and uncomfortable: I realized my agent had far more capability than it needed, and if something went wrong, the blast radius was unacceptably large.
This post explains the mental model, how I applied it in practice, and why it’s the first thing I now think about when designing any AI system that touches real infrastructure.
TL;DR
Blast radius thinking asks: if this fails or gets exploited, what’s the worst-case impact? Applied to AI agents and automation pipelines, it reveals over-provisioned access, missing rollback paths, and failure modes you haven’t accounted for. Run the thought experiment before you deploy, not after something breaks.
What Blast Radius Thinking Actually Means
The term comes from literal explosions. A small charge in an open field has a wide blast radius but limited real-world damage. The same charge in a server room is catastrophic. The explosive itself isn’t the only variable — the environment it operates in determines the impact.
In distributed systems design, engineers use this concept to reason about failure propagation. If a service fails, what else fails with it? If a database goes read-only, does one feature degrade gracefully, or does the entire application become unavailable?
In security, we apply it to compromise. If this account gets breached, what can the attacker reach? That’s the blast radius. Least privilege, network segmentation, and secrets management are all tools for reducing it.
The mental model has three components:
- Identify the actor — what is the thing that could fail or be compromised?
- Map the reach — what systems, data, and operations can it touch?
- Bound the damage — if everything it can reach is affected, is that acceptable?
If the answer to step 3 is “no,” you haven’t scoped it correctly yet.
How I Applied It to an AI Agent — The Same Day I Formalized the Model
I had a Claude Code agent running as part of an automation pipeline. Its job was to read repository metadata, generate documentation drafts, and push commits to a feature branch. Reasonable scope, or so I thought.
When I ran the blast radius exercise, I started mapping what it actually had access to:
- Full read access to all repositories in the org, not just the target repo
- Write access to the main branch (I had added this during testing and never removed it)
- Access to AWS credentials via an environment variable it inherited from the parent process
- API tokens stored in a shared secrets manager path with broader permissions than this agent needed
The agent’s nominal job was documentation. But its actual blast radius included: every codebase in the org, the ability to push directly to main on any repo, and — through the inherited AWS credentials — S3 buckets and Lambda functions that had nothing to do with documentation.
None of this was intentional. It was accumulated access that happened during development and never got cleaned up. This is the exact pattern I spent years catching in IAM reviews for enterprise clients. I was doing it to myself.
The blast radius exercise made it undeniable. I spent the next two hours tightening access:
- Scoped repo access to the single target repository
- Removed main branch write access, enforced PR-only commits
- Moved the agent to its own IAM identity with no inherited credentials
- Created a dedicated secrets manager path with only the secrets this agent legitimately needed
The agent’s capability from a task perspective didn’t change at all. Its blast radius dropped by roughly 90%.
Why This Mental Model Works Especially Well for AI Systems
Traditional software has deterministic behavior within its access scope. An API that can read your user database will read your user database when called — predictably, in ways you can audit.
AI agents are different. They take actions based on inferred intent, which means they can reach for capabilities you didn’t anticipate in ways you didn’t design for. A prompt injection attack, a misinterpreted instruction, or a model hallucinating a plausible-but-wrong action path can all cause an agent to use access you granted for purposes you didn’t intend.
This makes blast radius thinking more important for AI systems than for traditional software, not less. With conventional code, your threat model is primarily external attackers and bugs. With AI agents, you add a third category: the model behaving in unexpected ways within legitimate execution.
If you’ve scoped the access correctly, an unexpected action is a recoverable anomaly. If you haven’t, it’s a serious incident.
The three questions I now ask before any AI agent or automation goes live:
- What is the maximum this agent can destroy or exfiltrate if it behaves completely wrong? Be honest. List the actual capabilities, not the intended use.
- Is that maximum acceptable given the value this agent provides? If your documentation agent can push to production and touch billing data, the answer is no.
- What’s the rollback path? If the agent takes a bad action, can you detect it, stop it, and undo it? No rollback path means your blast radius includes permanent damage.
Applying Blast Radius Thinking to AI Pipelines: A Checklist
Run this before you move any AI agent from development to production:
Access scoping:
- Agent has a dedicated identity (not shared with other processes or humans)
- Repository access is scoped to the specific repos it needs, not org-wide
- Branch protection rules prevent direct commits to main
- Secrets access is scoped to a dedicated path, not a shared secrets prefix
- Cloud credentials are environment-specific (no production access from dev agents)
Failure mode analysis:
- You’ve named the worst-case action this agent could take
- The worst-case action is detectable via logging
- There’s a documented rollback path for that action
- The agent cannot take irreversible actions without an approval gate
Propagation limits:
- The agent cannot trigger downstream agents with broader permissions
- API tokens the agent uses cannot be used to escalate to higher-privilege access
- Network egress is limited to the endpoints this agent legitimately needs
This isn’t a one-time exercise. Run it every time you add a new capability to an existing agent. Capability creep is how blast radius grows without you noticing.
The Pattern This Surfaces Every Time
In 25 years of security reviews, the most dangerous configurations were never the ones built with bad intentions. They were the ones built with good intentions, incrementally, where each individual addition seemed reasonable at the time.
AI agents accumulate access the same way. You add a credential during debugging and forget to remove it. You grant broad repo access to speed up initial development and never narrow it. You inherit a permissions context from a parent process because it was convenient.
Blast radius thinking works because it forces you to evaluate the cumulative result, not each individual addition. You’re not asking “is this one credential reasonable?” You’re asking “given everything this agent can touch right now, what’s the worst realistic outcome?”
That question surfaces the problem every time. I haven’t run this exercise on any AI system where I didn’t find something to tighten.
Key Takeaways
- Blast radius thinking asks: if this fails or gets compromised, what’s the maximum damage it can cause?
- AI agents are higher-risk than traditional software because model behavior is non-deterministic — access granted for one purpose can be used for another in ways you didn’t design
- Run the exercise by mapping actual access, not intended access — they’re often significantly different
- The three questions: what’s the worst-case action, is that acceptable, and what’s the rollback path?
- Capability creep is the primary mechanism by which AI agent blast radius grows undetected
- The checklist above gives you a starting point — add to it as your systems grow
The mental model itself took five minutes to understand. Applying it the first time took two hours of access cleanup. That’s a good ratio. Run it before you deploy, not after something goes wrong.
Comments