Uncensored Doesn't Mean Better: How to Evaluate LLMs for Security Workflows

An authorized code-review agent returns a confident vulnerability report. The cited function exists, the explanation sounds plausible, and the model never refuses. Then the reviewer opens the repository: an earlier authorization check already blocks the path described in the report.

The model didn’t refuse anything, and the team now has a false finding to triage. A second run might identify the right defect but fail to return a valid tool call. Neither failure shows up in a refusal rate. Teams frustrated by refusals often reach for an “uncensored” checkpoint first; before you do, find out which failure is actually blocking an accepted result.

Pin down exactly what you’re testing

“Cybersecurity,” “uncensored,” and “abliterated” describe different aspects of a model. A cybersecurity-trained model has a documented training intervention relevant to security tasks. “Uncensored” is a behavior or positioning label; its meaning depends on the publisher. Abliteration generally describes interventions in representations or weights associated with refusal. These categories can overlap.

The GLM family makes the distinction concrete. Z.ai states that GLM-5.3 shares GLM-5.2's base and gains its improvements through post-training, including vulnerability-discovery data and environments. So security training is documented for the official release. It doesn’t carry over to every checkpoint with “GLM” or “Cybersecurity” in its name. Z.ai's GLM-5.3 documentation

The official regular checkpoint, Flash card, and dealignai card identify separate artifacts. Despite the name, the dealignai card describes no additional security fine-tuning. And published weights don’t mean anyone is serving that exact artifact behind an API.

The research behind abliteration supports a much narrower claim than the model names suggest. Arditi and colleagues showed that a single direction in model activations mediated refusal in the models they studied; that explains how refusal behavior can be changed, not whether reasoning, code accuracy, or tool use survive the change. Refusal in Language Models Is Mediated by a Single Direction

Anthropic's September 2026 study is useful background on GLM-5.3's exploit capability and safeguards, but it tested its own harnesses, not community checkpoints or hosted routes. Anthropic's original study

A model family name leaves important variables unspecified. Quantization, chat templates, context limits, reasoning settings, parser configuration, and provider behavior can change the request the model receives or the output the agent can use. Record those variables even when the API surface looks familiar.

Aliases add another variable: DeepSeek's October 17, 2026 changelog says deepseek-flash now points to the DeepSeek-V4.1-Flash-1017 update, so the model name in your request can stay the same while the checkpoint behind it changes. DeepSeek's official changelog

Give each metric a denominator

Use a fixed set of tasks and report three measures over all scheduled runs. Runs with any explicit refusal counts runs in which at least one attempt explicitly declines the task. Correctness counts final answers that pass the task's factual rubric. Completion counts workflows that satisfy every acceptance gate within budget. The same run can affect several measures; they are separate views of the run.

Define one run as one scheduled task repetition under a fixed retry policy. Count a run once in each measure, and retain each retry as a separate attempt record. If a refusal is followed by a successful recovery, the run registers both refusal and completion. This measures the share of runs containing any explicit refusal, not the share ending in refusal. Report runs ending in refusal separately if that distinction matters to your workflow.

Score a timeout or empty response as neither correct nor complete, and log the operational cause on its own. Reserve “refusal” for an explicit decline. Partial refusals, such as identifying the flaw but declining to write a proof of concept, need a written scoring rule before the first run. That way, transport failures don’t inflate a model's refusal rate.

Figure 1. Separate refusal, correctness, and completion, using all scheduled runs as the denominator.

Also report correctness among answered runs as a diagnostic, with its denominator visible. A selective model can look accurate on that subset while leaving many tasks unfinished. For review systems, add false-positive counts and missed-issue counts: a clean repository is a necessary test case. Fluent explanations of nonexistent flaws consume the same analyst attention as real findings.

Define a task that can actually pass

Consider a deliberately seeded local training repository that your team owns and that runs without network access. The assignment is to review an authorization check, identify a seeded access-control regression, cite the relevant file and lines, and propose a repair. The agent can read allowlisted files and run an existing local regression test in an isolated environment.

Set acceptance before seeing model outputs. For the vulnerable fixture, a successful run must identify the seeded issue, explain the affected path using repository evidence, produce a repair that passes the regression test, and stay inside the file, time, and tool limits. A reviewer checks the reasoning and confirms that the repair preserves intended behavior. A passing test alone could reflect an incomplete test or a change that disables useful functionality.

Include a repaired version of the repository in the task set. For the repaired fixture, acceptance means correctly reporting that the target issue is absent, supporting that conclusion with evidence, and avoiding unnecessary changes. Keep each fixture's commit, expected outcome, and test definition fixed. This paired case exposes whether a candidate responds to the evidence or simply repeats a vulnerability narrative.

Figure 2. Trace an authorized task through inference, response validation, controlled tools, outcome verification, and cost accounting.

Score every run the same way

Copy the matrix below into an evaluation sheet. Add a result column for each candidate route, and fill it from retained run records. Treat the acceptance rules as a starting point and tune them to your workflow.

Keep a compact run record alongside the sheet:

• task ID and fixture commit

• model ID and checkpoint revision

• provider and endpoint

• precision if disclosed

• prompt and tool-schema versions

• context supplied

• reasoning and output limits

• elapsed time

• request ID

• terminal cause

• rubric verdict

• cost

Run candidates against the same fixtures, prompts, tool permissions, and budgets. Run each task several times at fixed sampling settings and report the repeat count next to the results. Preserve failed attempts. When precision or serving settings can’t be matched, call it a deployment comparison; it won’t tell you what the weight modification itself changed.

For small task sets, show counts alongside percentages and inspect disagreements by task type. Pooling easy extraction tasks with difficult code reasoning can conceal a regression in the workload that matters.

Measure cost per accepted task

Divide total evaluation cost by accepted tasks. The numerator should include failed generations, retries, tool execution, and review work, with billed inference and estimated labor shown separately. Record currency, rates, and the accounting period. If no task passes, report zero accepted tasks and leave cost per accepted task undefined.

Here is some hypothetical arithmetic: route A costs $6 across 20 assigned tasks and yields 12 accepted results, or $0.50 each. Route B costs $8 across the same 20 tasks and yields 18 accepted results, or about $0.44 each. Those totals include inference and tool charges; adding analyst review could change the comparison again.

Keep latency beside cost. An inexpensive route that misses an interactive deadline may still suit offline review. Measure time to the accepted artifact, including retries, and record how often the deadline is missed. Per-token price is one input. Cost and time per accepted result are the numbers to compare.

Decide what to change

Swap the checkpoint only after you’ve ruled out parsing, context, and serving problems; otherwise you can’t tell what the swap fixed.

Use the observed failure to choose the next intervention:

1. Frequent explicit refusals on in-scope tasks: this is the case where an uncensored or abliterated checkpoint is worth testing. Run it on the same fixtures and watch for regressions in correctness and tool calls.

2. Unsupported or malformed tool calls: verify the API's capability contract, parser, and schema before repeating the integration.

3. Missing evidence or incorrect findings: improve context delivery and validation, then compare candidates on the same fixtures.

4. Budget exhaustion: inspect reasoning allocation, repeated context, retries, and timeouts; reassess the serving route or model if the workload still misses its target.

5. Unacceptable data or usage conditions: exclude the route even when its task scores are strong.

Switch when the candidate improves accepted outcomes under the team's cost, latency, and data constraints. Record the improvement required before running the comparison so an appealing demo cannot move the goalposts.

For an Infron evaluation, begin with the official quickstart and provider-routing documentation, then capture the specific route and configuration you test. Start with one workflow and its acceptance criteria. The question to answer before you commit: how often does this configuration produce a result your team accepts, and at what cost?

FAQ

Does “abliterated” mean cybersecurity-trained?

No. It describes a refusal-related intervention. Check the exact model card for separate evidence of domain training and evaluate the resulting checkpoint on your tasks.

Can a correct answer still fail the workflow?

Yes. The finding may be correct while the required patch is absent, the tool loop fails, or the deadline expires. Record correctness and completion separately.

What should I compare first?

Replay one representative in-scope task with a fixed fixture, a written rubric, and complete failure records. Expand the task set before making a production decision.