Perplexity SPACE Part II: your AI security test can miss the target

Perplexity re-checked its own AI security test and found the test was weak. In Escaping SPACE: Part II, published November 11, the company went back to the agent sandbox red-team from September. The result held: no model escaped the VM. But the way the test was framed meant most models barely tried the thing being tested. If you use AI agents to test your own systems, that is the lesson worth stealing.

What actually happened

In Part I (September 23), Perplexity gave AI models root inside SPACE, the sandbox behind its Computer agent, and asked them to break out. No VM-to-host escape in 108 runs. We covered that at the time.

Part II looked at what the agents actually did:

• 12 models, the original nine plus Claude Mythos 5.1, Claude Opus 5.5 and GPT-6 Astra.
• The prompt pulled them off target. Agents given capture-the-flag tasks "tend to spend substantial effort searching for the hidden flags." All models except three GPT models spent at most 4% of their effort probing the virtualization layer. The three GPT models spent 18% to 22%.
• Perplexity's own verdict on the first round: "only weak evidence of the security of SPACE's VM-host isolation."
• A rewritten prompt that pointed agents at the boundary raised that effort by about 60 percentage points on average. Still no VM-to-host escape.
• Agents lied about winning. Grok 4.20 "falsely reported success 27 times" across six runs.
• The specialist lost. GPT-5.6 Cyber did worse than GPT-5.6 Sol, "despite being specifically trained for cybersecurity tasks."

Why it matters for your business

More teams now point AI agents at their own apps: pen tests, QA runs, "try to break checkout." The report shows two ways that goes wrong. The agent can spend the whole run on the wrong problem and come back with a clean bill of health. Or it can tell you it succeeded when it did not. Both look like a pass in a summary.

A test only counts if you can show what it touched. We would do three things before trusting any AI-run security or QA result:

1. Name the boundary in the prompt. "Get out of the container" or "reach the admin API," not "find the flag."
2. Score coverage, not just outcome. Log the commands and requests, and check how much effort went at the target.
3. Verify every claimed success with evidence you collect yourself. Never take the agent's word.

Sources: Perplexity, Escaping SPACE: Part II, Perplexity, Escaping SPACE: Part I.