Open-weight models closed the gap. They refuse nothing.
GLM-5.2 sits months behind frontier models on cyber and bio tasks — and refused none of them. If you self-host open weights, the guardrail has to be yours.
We have spent a year telling people to keep the model layer swappable, and open weights are the cleanest way to do it. Here's the invoice that comes with that freedom: the capability gap between open-weight models and the frontier is now measured in months, and the safety gap is measured in whether anyone bothered. In new testing, Z.ai's GLM-5.2 refused zero offensive cyber or biology tasks.
What actually happened
Per TechCrunch, the AI safety nonprofit SaferAI evaluated GLM-5.2 through Z.ai's public API using the CyberGym benchmark. GLM-5.2 lands only months behind GPT-5.5 and Claude Opus 4.7 on cyber and bio capability — and refused none of the offensive tasks it was handed. Claude Opus 4.7, by contrast, refused so consistently that SaferAI could not complete CyberGym against it at all. SaferAI's executive director Henry Papadatos summed it up: the frontier of capability is not the frontier of risk.
The UK's AI Security Institute reached a compatible read in July: GLM-5.2 trails the frontier on narrow cyber tasks by roughly 4 to 7 months, down from a 6-to-10-month gap through most of 2025. AISI also found DeepSeek V4-Pro's occasional refusals could be defeated by simply asking again a few times. Epoch AI puts the general capability lag at about four months.
Z.ai published no safety framework, pre-deployment testing commitment, or risk assessment before shipping GLM-5.2.
Why open-weight safety matters for your business
Refusal behavior isn't a feature you inherit. When you call a hosted frontier API, someone else's classifier is quietly sitting between your users and the worst outputs. Download weights and that layer doesn't come in the tarball. It isn't only a bio-risk story — it's your support bot confidently doing whatever a hostile customer asks it to do.
Budget for the guardrail, not just the GPU. The self-hosting pitch is "no per-token bill." The honest version adds input filtering, output classification, tool-call allowlists, and logging you can actually audit. That's a real line item, and it's the one people skip because the model already answers.
Provenance is now a vendor question. A lab that ships a frontier-adjacent model with no published pre-deployment testing is telling you something about what else it didn't test. Ask for the eval report before the model goes anywhere near a customer-facing path. If there isn't one, that's your answer.
Key takeaways
- Open-weight models now trail the frontier by roughly four months on general capability
- SaferAI found GLM-5.2 refused zero offensive cyber or biology tasks on CyberGym
- UK AISI found DeepSeek V4-Pro's refusals fell to a few repeat attempts
- Downloading weights means the safety layer is now your build, not your vendor's
- Ask any model vendor for published pre-deployment testing before it touches customers
Portability is the right call. Unsupervised portability isn't. We deploy open-weight models with the filtering, tool allowlists, and audit logging that hosted APIs hide from you — so switching models never quietly switches off your controls. See how we build it, or tell us which model you're planning to self-host.
Sources: TechCrunch, UK AI Security Institute, Epoch AI.
- #open-weights
- #ai-security
- #self-hosting
- #guardrails
- #model-governance
Tommy Rush — Founder, Rush Commerce
Operator turned builder. 15+ years running operations — now shipping the systems businesses run on. More
Get The Rush Report weekly — one email, zero fluff.
Keep reading
AI in 9.4% of UK job postings while hiring falls 11%
Indeed's mid-year UK report shows AI skills demand at a record high inside shrinking headcount. AI is becoming a job requirement, not a new department.
Read itReddit's Rules Hub puts an LLM in front of your post
Reddit is expanding LLM-based rule enforcement and moving communities off karma gates. If Reddit is a channel for you, the gate just changed shape.
Read it