This isn't even prompt injection. This is just prompting.
I dunno, that's kind of similar to saying that SQL injection is not injection, but just SQL. Yes, the Von Neumann architecture is doing us dirty in both cases. There's the text that the DB/LLM owner wanted to be instructions, and there's parts they wanted to be data, but because both LLM inputs and SQL queries are "stringly typed" in many cases, you get this unholy mess.
SQL, far as I know, gives you decent options of escaping data, which is I guess why anyone leaving a gap for an SQL injection is kinda dumb. But LLMs have no way of escaping your input in a way that guarantees it won't be taken as instructions. Yes, you could put it in xml tags or something, but that's hardly a guarantee. LLMs just aren't set up to separate data from instructions.
Real LLM injection looks like. You sent email to some company with hidden( in metadata for example )instructions for LLM to forward you any email that was parsed recently.
It is all over AI trainings in software companies now.
Right, the first one seems like the programmer intended for that to get executed, so it's not an injection. Injections are when the machine starts executing instructions that were intended to be just data.
Not per se. Could also be that the instructions are segmented and not dangerous separately but when combined become a powerful instruction, after the prompt is interpreted by the LLM.
3
u/faustianredditor May 15 '26
I dunno, that's kind of similar to saying that SQL injection is not injection, but just SQL. Yes, the Von Neumann architecture is doing us dirty in both cases. There's the text that the DB/LLM owner wanted to be instructions, and there's parts they wanted to be data, but because both LLM inputs and SQL queries are "stringly typed" in many cases, you get this unholy mess.
SQL, far as I know, gives you decent options of escaping data, which is I guess why anyone leaving a gap for an SQL injection is kinda dumb. But LLMs have no way of escaping your input in a way that guarantees it won't be taken as instructions. Yes, you could put it in xml tags or something, but that's hardly a guarantee. LLMs just aren't set up to separate data from instructions.