IBM Granite LLMON: a prompt injection defense to test

Prompt injection works because a model cannot tell your instruction from text it reads. IBM just shipped an attempt to fix that in the model. Two Granite 4.1 LLMON preview models mark each part of the prompt with reserved tokens, then tell the model which parts are commands and which parts are only material to read. It is the most concrete prompt injection defense with open weights we have seen. IBM also says it is not ready for production.

What actually happened

IBM published the LLMON announcement on November 12. LLMON means "LLM Object Notation". It is an XML-like markup with one important difference from JSON: its delimiters are reserved special tokens. A brace or a quote can appear in an email. <|open|> cannot, so an attacker cannot type the structure.

The parts:

• Blocks. Each piece of context gets a tag: instr, data, doc, tool, req, system.
• The exec block. Each turn names which instruction to run and which blocks are its inputs.
• The rule. The instruction and system blocks have authority. Text reached as input is read, not obeyed, no matter how it is phrased.

The 8B model card lists a 3B and an 8B model, both Apache 2.0. You do not write the markup by hand. The bundled chat template renders it from instruction and data fields in transformers. vLLM and SGLang are listed too.

IBM's results: on the OPI benchmark, the models avoid 47% more attacks than the best frontier model IBM tested, and 95% more than the best open-source model. IBM does not name those models in the post.

Why this prompt injection defense matters for your business

Each agent that reads email, web pages or tickets has this problem. Until now, the only fixes were outside the model: filters, allowlists, approvals. Experts rank injection as the top risk, as we wrote in our incident data post. A fix inside the model is the correct direction.

Read the limits first:

1. Preview only. IBM's words: ready for evaluation, "not for production".
2. Static attacks only. The tests used fixed injections. Multi-turn adaptive attacks are still in work.
3. Learned, not enforced. The model card says the rule reduces the risk, does not remove it, and must not be the only defense for an agent with sensitive access.
4. Some capability is lost. Against Granite 4.1 8B Instruct, the model card shows GSM8K at 89.16 versus 92.49 and CRUXEval-O at 36.88 versus 47.63. BigCodeBench goes up, 44.04 versus 35.00.

What to do: if you run a small open model for document triage or ticket routing, put the 8B preview next to it this week. Send both the injections you fear most. Keep each control you have now. And when you buy an agent product, ask the vendor one question: how does your model tell my instruction from the data it reads?

Sources: IBM Granite LLMON announcement, Granite 4.1 8B LLMON preview model card, LLMON toolkit on GitHub.