It concerns me that anyone with anything important to protect might trust what this paper calls "Injection detectors deployed to protect LLM agents" - Llama Guard and the like.
There are unlimited combinations of tokens that can be used to attack an LLM system. The idea that some kind of "detector" can catch them all just feels inherently absurd to me.
Contemporary tech culture successfully trained influential people to be beyond credulous.
If you have somebody promising a feature and somebody saying that the feature is impossible or a time bomb for catastrophe, the default for most executives and many developers these days is to believe the person promising the feature. And then, to boot, you can trust that same executive or developer to shirk responsibility when things fail later with a "How could I have known?! [Now defunct company] said it would work!"
There are unlimited combinations of tokens that can be used to attack an LLM system. The idea that some kind of "detector" can catch them all just feels inherently absurd to me.
If you have somebody promising a feature and somebody saying that the feature is impossible or a time bomb for catastrophe, the default for most executives and many developers these days is to believe the person promising the feature. And then, to boot, you can trust that same executive or developer to shirk responsibility when things fail later with a "How could I have known?! [Now defunct company] said it would work!"