Using an LLM to interpret natural language and from that perform commands is an incredibly bad idea.
nativeit 32 days ago [-]
> Microsoft never said how it mitigated the Copilot vulnerability and didn't answer questions asking for these details. While the specific attack Rehberger devised no longer worked, indirect prompt injection still did.
...
> Another measure chatbot developers employ is a limitation on broad types of instructions that can be invoked by untrusted data. In Google’s case, this measure seems to include the invocation of apps or data available through its Workspace collaboration suite. (Google doesn’t document these restrictions anywhere, so researchers are left to infer them based on the behavior they observe.)
I understand the impetus to protect one's "special sauce" recipes in this nascent industry, but it sounds like it would be beneficial to everyone if they did more to document the protection mechanisms they are developing, and welcome outside review, if not contribution. I'm in no way an expert in AI or security, just picked up on a recurring theme, and wondered if anyone was working in this space, or knew of any specific efforts to establish some kind of standards of practice for AI security?
sigmoid10 32 days ago [-]
This is not just about keeping the moat. If you disclose how exactly content is filtered, it becomes much easier to bypass those filters. In the same way most companies won't disclose their firewall setup.
...
> Another measure chatbot developers employ is a limitation on broad types of instructions that can be invoked by untrusted data. In Google’s case, this measure seems to include the invocation of apps or data available through its Workspace collaboration suite. (Google doesn’t document these restrictions anywhere, so researchers are left to infer them based on the behavior they observe.)
I understand the impetus to protect one's "special sauce" recipes in this nascent industry, but it sounds like it would be beneficial to everyone if they did more to document the protection mechanisms they are developing, and welcome outside review, if not contribution. I'm in no way an expert in AI or security, just picked up on a recurring theme, and wondered if anyone was working in this space, or knew of any specific efforts to establish some kind of standards of practice for AI security?