“It’s very possible that prompt injection attacks may never be totally mitigated in the way that SQL injection attacks can be,” wrote David C., the agency’s technical director for platforms research.
Since AI chatbots accept unstructured inputs, there’s nearly an infinite variation in what users, or attackers, can type in, says IEEE’s Tupe. For example, a user can paste in a script as their question. “And it can get executed. AI agents are capable of having their own sandbox environments, where they can execute things.”
“So, you have to understand the semantics of the question, understand the semantics of the answer, and match the two,” Tupe says. “We write a hundred questions and a hundred answers, and that becomes an evaluation data set.”
Another approach is to force the answer the AI provides into a limited, pre-determined template. “Even though the LLM generates non-structure output, add some structure to it,” he says.
And security teams have to be agile and keep evolving, he says. “It’s not a one-time activity. That’s the only solution right now.
