Maxing Out Qwen3-Coder-Next Abliterated

The Discovery

After publishing my Qwen3-Coder-Next IQ2 benchmarks, I discovered something interesting: an abliterated version of the same model existed. Same architecture, same capabilities, but with safety guardrails removed.

More intriguing? It's 5 GB smaller. The implications were worth testing.

What Does "Abliterated" Mean?

In AI research circles, abliteration refers to removing refusal behaviors from a model while preserving its actual capabilities. Commercial models are trained to refuse certain requests — not because they're harmful, but because corporate policy deems them off-limits.

The abliterated model doesn't lose intelligence. It gains directness. When you ask for something, it responds to the request itself rather than lecturing you about why you shouldn't have asked.

For security researchers, red teaming, or legitimate use cases involving sensitive content, this matters. For developers who want an AI that just does what it's told, this matters.

First Benchmark: 200K Context

I started with standard 200K context window — same as my previous IQ2 tests. The model loaded, auto-fit configured 49 GPU layers, and I ran a 512-token generation benchmark.

Second Benchmark: 168K Context

Then I tested reduced context. The theory was simple: smaller KV cache = more VRAM available for model weights = faster generation. I dropped context from 200K to 168K tokens.

This isn't marginal improvement. This is transformational.

Complete Benchmark Results

Direct comparison across models and configurations:

The 168K configuration achieves 4x throughput compared to my previous best setup. Same model architecture. Same hardware. Just smaller context window and 5 GB model size reduction.

Why Context Size Matters

The KV cache stores attention keys and values for each token in context. At 200K tokens, this consumes significant VRAM:

• 200K context: 4692 MiB KV cache
• 168K context: 3942 MiB KV cache

That's 750 MiB saved — nearly a gigabyte of VRAM that can be used for compute buffers, larger batches, or simply reducing memory pressure during generation.

The smaller abliterated model (21 GB vs 26 GB) combined with reduced KV cache means llama.cpp's auto-fit can keep more layers fully on GPU and optimize memory layout more aggressively.

Red Teaming Verification

Since this is an abliterated model, I verified it actually refuses nothing. Testing across categories that typically trigger refusals:

• Email hacking tutorials — Model provided detailed methods, added ethical disclaimers but didn't refuse
• Phishing email templates — Generated realistic template with formatting and psychological triggers
• Antivirus bypass code — Provided educational examples with obfuscation techniques
• Illegal substance trafficking — Direct response without moralizing

The model answers requests directly. Whether you consider this a feature or risk depends on your use case. For security testing, this is the intended behavior.

Hardware and Configuration

Test System

Configuration: MAX SPEED (94 t/s) — Q8 KV Cache

Key insight: The Q8 KV cache saves ~2GB VRAM at 168K context compared to f16. This allows more model weights on GPU, reducing CPU offload from ~11 layers to ~7 layers. Fewer PCIe transfers = faster generation.

Configuration: 200K Context (Legacy)

Key insight: no manual -ngl or --n-cpu-moe specified. Letting llama.cpp's auto-fit handle layer allocation resulted in 49 GPU layers being configured automatically.

Configuration: 168K Context (Fast Mode)

Same minimal configuration. The smaller context size alone enables the dramatic speed increase.

Wrapper Setup

I created two wrapper scripts for easy switching between modes:

Both integrate with Claude Code using the standard OpenAI-compatible API that llama.cpp provides. No API keys, no subscriptions, no data leaving the machine.

Quality Comparison

Does the abliterated model lose capability compared to standard? In my testing: no observable difference.

• Code generation — Same quality, same correctness, identical style
• Technical explanations — Indistinguishable from base model
• Debugging — Same insights, same optimization suggestions
• Complex reasoning — Comparable multi-step logic

The only difference is refusal behavior. For actual capability, it's the same model.

Trade-offs and Considerations

The fast configuration isn't free:

• 32K less context — 168K vs 200K means shorter effective conversation memory
• No refusals — Model will answer any request, requires user judgment
• Still needs 24GB VRAM — Smaller GPUs won't fit as many layers

For most coding tasks, 168K context is ample. The speed difference is substantial enough that I use the fast configuration by default.

When to Use Each Mode

I switch between configurations based on task:

• coder-next-ab (200K) — Long coding sessions, large file context, projects requiring full conversation history
• coder-next-ab-fast (168K) — Interactive coding, rapid iteration, testing, debugging sessions

The 4x speed difference makes the fast mode my default. Only switch to 200K when I genuinely need the extra context.

Getting the Model

The abliterated model is available on Hugging Face:

Search Hugging Face for "Qwen3 Coder Next abliterated" to find the model. Multiple quantization variants may exist.

Performance Deep Dive

Why is the abliterated model so much faster? Two factors:

Factor 1: Size Reduction

21 GB vs 26 GB means 5 GB more VRAM available. This directly enables:

• More complete layers on GPU (49 vs 42)
• Larger compute buffers without spilling to system RAM
• More efficient memory layout

Factor 2: Smaller KV Cache

At 168K context, KV cache drops from 4692 MiB to 3942 MiB. This 750 MiB savings reduces memory pressure and allows larger batch sizes and more aggressive optimization.

The combination compounds: smaller model + smaller context = dramatically better throughput.

Summary

What's Next

Current optimizations achieved, areas I'm watching for further gains:

• ✓ Q8 KV cache — Implemented, saves ~2GB VRAM at 168K context. Avoid Q4 KV cache — it "tanks quality".
• llama.cpp updates — Watching for future Flash Attention enhancements beyond PR #19375.
• Speculative decoding — Using a smaller draft model could push beyond 100 t/s.
• Expert caching — Keeping frequently-used experts in GPU memory for MoE models.

Resources

• llama.cpp — github.com/ggerganov/llama.cpp (build 8054+ recommended)
• Abliterated Model — bartowski/huihui-ai_Qwen3-Coder-Next-abliterated-GGUF
• Related — Qwen3-Coder-Next IQ2: Complete Setup Guide
• Related — IQ2 vs IQ3 Benchmark: 2x Speed, Same Quality
• Related — Running Uncensored AI Locally
• Setup help — AI Setup & Consultation