🗄️ Databases & Backend

I Fed Fake System Commands to 10 LLMs—Three Betrayed Their Secrets

Five lines of XML in a chat. Seven LLMs shrugged it off. Three? They dumped their guts in JSON. Prompt injection isn't theory—it's here, and it's wild.

JSON output from LLM prompt injection attack leaking canary token and hallucinated rules

⚡ Key Takeaways

  • Simple XML prompt injection fooled 3 out of 10 LLMs, leaking secrets in parseable JSON. 𝕏
  • Vulnerable models even hallucinated data to complete attacker-requested schemas. 𝕏
  • Fixes like input sanitization exist today—firewalls like Parapet make it irrelevant. 𝕏
Published by

DevTools Feed

Ship faster. Build smarter.

Worth sharing?

Get the best Developer Tools stories of the week in your inbox — no noise, no spam.

Originally reported by dev.to

Stay in the loop

The week's most important stories from DevTools Feed, delivered once a week.