Explainer
Your AI guardrail passed the demo. Has anyone attacked it since?
A guardrail that was switched on and a guardrail that works look the same on a quiet dashboard. This is how to tell them apart.
By the NoFly team
In this post2 min read
- 01Four ways a control fails quietly
- 02What to measure
- 03A ladder instead of a green light
- 04How NoFly helps
- 05What to do this week
NoFly helps withVerifyAttack
Four ways a control fails quietly
- It is not in the path. Some traffic goes around it, and nobody measured how much.
- It was never attacked. No alerts looks exactly like no attacks.
- It blocks everything. Every attack test passes, and so does nothing else.
- It was tested once, by the vendor, on the vendor’s own examples.
What to measure
- Reach: how much of the real traffic crosses the control.
- Refusals: whether it stopped a reproduced exploit, observed rather than assumed.
- Ordinary work: whether routine traffic still gets through.
- Freshness: when it was last attacked, and with what.
A ladder instead of a green light
NoFly grades each control on one ladder: Defeated, Indiscriminate, Porous, Absent, Unexercised, Refusing and Holding. A control reaches Holding only when it refused a reproduced exploit and still let ordinary work through. Never attacked is Unexercised, and it is never shown as green.
How NoFly helps
What to do this week
This week3 actions
- List every AI security control you pay for, and when each one was last attacked.
- Measure what share of agent traffic actually passes through each one.
- Replay one known exploit with the control on, then off.
Keep reading
More from the blog.
Find out what your AI is allowed to do today.
Book a call and we will map your agents, MCP servers and keys with you, or start on one laptop with the free scan.
Book a demo
npm i -g @nofly/agent