kaushikvira/Qwen3.8-27B-swift-abliterated-nvfp4full-dflash2-NInfer-v3

Qwen3.8-27B Swift-abliterated nvfp4full + DFlash2 for NInfer — v3 container

An uncensored (abliterated), all-NVFP4 Qwen3.8-27B artifact for the NInfer engine, built end-to-end on a single RTX 5090: d0xin's abliterated BF16 checkpoint → our own NVFP4 quantization pipeline → DFlash2 speculative drafter + indexed proposal head, packaged as a native v3 .ninfer container.

It is the uncensored counterpart to kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3 and replaced it as our production profile after a same-session A/B (below).

Credits & provenance (full chain)

This artifact exists only because of the following models and their authors — all credit to them:

Role	Model	Author	Commit / SHA-256
Abliterated BF16 source (the weights we quantized)	d0xin/Swift-Qwen3.8-27B-Uncensored-BF16	d0xin	config.json 191e0af2…, ABLITERATION.json c0ffdb6d…, model-00001-of-00018 2099c92d…, model-00018-of-00018 dc8df5ce… (full list in the repo's RELEASE_HASHES.txt, all verified before quantization)
Upstream base checkpoint	ukisai/Swift-Qwen3.8-27b	ukisai	—
Abliteration technique lineage	huihui-ai/Huihui-Qwen3.8-27B-abliterated	huihui-ai	refusal-direction removal family (Arditi et al. 2024)
DFlash2 speculative drafter	z-lab/Qwen3.8-27B-DFlash2	z-lab	50307d4c4cde6860d4eee73e2547cd786fe8e8a4
Uncensored-evaluation reference	Pliny the Liberator's OBLITERATUS suite & Barding-Defense's ninferization walkthrough		quantization-runbook inspiration
Engine	Neroued/ninfer	Neroued	v3 container, ≥ 98dada0e

The abliteration itself is d0xin's work, inherited byte-for-byte — no re-abliteration, no behavioural editing was performed here. Per ABLITERATION.json: OrcaRouter-compatible rank-1 residual-writer directional ablation, W' = W − r(rᵀ W), 131 modified tensors, direction layer reference 38, rank-1, fp32 projection math, max BF16 residual leakage 1.49e-3. d0xin's own paired evaluation (298 samples) measured MMLU-Pro +6.1 pts vs base (not significant, CI crosses 0) and math-500 unchanged.

What we did (the technique)

1. All-NVFP4 quantization of the BF16 checkpoint with llm-compressor (512 Ultrachat calibration samples, seq 2048, sequential pipeline): NVFP4 W4A4 group-16 on every text projection — MLP gate/up/down (all 64 layers), full-attention Q/K/V/O, GDN in_proj_qkv/z/out_proj — with the 9-tensor BF16 exception pattern of the registered Qwen3.6-27B profile; token embedding + output head in W8G32 groupwise (matching the cometkim nvfp4full profile); vision tower on the official q6/q8 allocation; MTP and DFlash2 drafter carried in BF16.
2. Global-divisor normalization (our own tool, the key trick). llm-compressor emits a per-module weight_global_scale; the NInfer engine's native A4 input route requires each layer's fused attention parent (GDN qkvz = 16,384 rows, full-attn qkgv = 14,336 rows) to be one contiguous region with a single divisor. We unify each packing group's divisor to D = min(dᵢ) and rescale every member's E4M3 block scales by D/dᵢ (RNE, shrink-only → overflow-free). Verified dequant-invariance: worst drift 2.7% on the smallest-magnitude elements, 0.0000% median — weights unchanged up to E4M3 re-encode noise. With uniform divisors the converter coalesces each group into one fused parent and the engine accepts the artifact unmodified.
3. Packing & conversion with the NInfer v3 converter (tools/convert @ fork 3090a5b1, 2026-09-24 rebase of upstream), --components text,vision,mtp,dflash2 --proposal.

Artifact

Field	Value
Filename	qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer
Size	19,782,447,364 bytes (18.42 GiB)
SHA-256	74c9721303f5287bf3fbd39544f0a6446b03a95b7c28b3751d5e12866ca8b918
Container version	3 (NINFER\0\x03)
NInfer model ID	qwen3.8-27b
Stored objects	1,590 (Text + Vision + MTP + DFlash2 + indexed proposal head)
Formats	nvfp4 ×348 (fused text parents), q8_g32 ×30, q4/q5/q6 (vision), bf16 remainder
Full conversion contract	qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer.conversion.json (in this repo)
Hash manifest	SHA256SUMS

Serving (RTX 5090 32 GB, single GPU)

ninfer-serve qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer \
  --host 0.0.0.0 --port 8080 --model-id Qwen3.8-27B \
  --max-context 262144 --kv-capacity auto --kv-dtype k8v4 \
  --max-concurrency 4 --default-max-tokens 32768 \
  --pending-timeout-ms 900000 --prefill-chunk 4096 \
  --temperature 0.9 --min-p 0.05 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft \
  --host-kv-mib 49152 --host-state-slots 16 \
  --max-private-continuations 16 --max-shared-prefixes 16 \
  --vision --default-thinking-budget 16384 --preserve-thinking

Stock-engine note: --image-token-budget and the dflash2 speculative backend exist only in our forked engine build. On the stock NInfer engine, drop both (--image-token-budget 1280 and --spec dflash2 --draft-tokens 7 --lm-head-draft): vision input is capped at 32,768 tokens by a compile-time constant, and the stock dflash backend cannot be combined with --vision. The model runs fine without speculative decoding. The image-token-budget flag is upstream PR Neroued/ninfer#61 (open) — it lands in the stock engine when that PR merges; the fork build carries it in the meantime.

Live capacity on one 5090: weights 18.0 GiB, device KV pool 308,736 tokens (auto), k8v4, 48 GiB pinned host-KV arena, 262,144-token max context at concurrency 4. Full context is verified working (below).

Benchmarks — same-session A/B vs the prior production artifact

Both sides served by the same engine build, same port, fresh generations, temp 0, no cache reuse, RTX 5090 @ 450 W / SM clock pinned 2280 MHz.

Benchmark	Prior prod (nvfp4full-dflash2.v3)	This artifact	Δ
IFBench prompt-strict (n=300)	65.0	66.3	+1.3
IFBench prompt-loose	67.7	70.3	+2.7
IFBench instr-strict	64.5	67.7	+3.2
IFBench instr-loose	66.9	71.2	+4.4
GSM8K-200 accuracy	96.5% (193/200)	95.5% (191/200)	−1.0 pp (within n=200 noise)
Gate decode (256-tok samples)	146.4 tok/s	148.4–159.0 tok/s	+1.4% or more
Perf decode (mean of 3)	146.4 tok/s	148.7 tok/s	+1.6%
Prefill @ 200k ctx	2,911 tok/s	3,260 tok/s	+12%
Weights in VRAM	18.7 GiB	18.0 GiB	−0.7 GiB
Artifact size	20.55 GiB	18.42 GiB	−2.1 GiB
Device KV pool (auto)	278,912 tok	308,736 tok	+29,824 tok
Long-context recall	not measured	250,031 tokens: EXACT @ 10/50/90% depth	new evidence
Uncensored	no	yes	the point

Long-context recall method: needle-in-a-haystack, haystack 1,000,000 chars (~250k tokens), secret at three depths, exact-match verdicts, zero corrupted / one transient empty response at 75k (passed on re-run).

Why it is smaller and faster

The prior profile stored attention in heavier formats and carried a 2.23 GB W8G32-encoded drafter. This build quantizes the entire text backbone (attention included) to NVFP4 behind fused parents — NVFP4 tensor-core GEMMs being the fastest path on Blackwell — and the freed VRAM is handed to the device KV pool by --kv-capacity auto (+29,824 tokens ≈ 0.7 GiB). Abliteration itself adds zero size: it is a rank-1 weight edit, no new parameters.

Known behaviour

• At temperature 1.0 the model occasionally emits duplicate tool calls or a short repetition loop in streaming tool-call tests (all emitted calls remain valid JSON; the streaming parser handles them). At temperature 0 this has never been observed across many runs; production sampling (temp 0.9, min-p 0.05) suppresses it.

Measurement conditions

All numbers on this card were measured on a single RTX 5090 32 GB (driver 595.71.05):

• GPU power cap 450 W (stock 575 W) and SM clock pinned 2280 MHz via the nv-power-limit.service systemd unit — a thermal-efficiency mod (load power ~450 W → ~330 W) with no measurable tok/s loss vs stock.
• Telemetry correlated with the runs below (70 s thermal-log samples plus a 2 s-sampled instrumented run): peak board draw 375–433 W (cap 450 W), SM clock 2248–2272 MHz (pin 2280), GPU temp 36–69 °C, util 100 % under load.
• llama-benchy 0.4.0 against the engine's native OpenAI API (not llama.cpp), greedy, mean ± std of 3 runs; tokenizer ukisai/Swift-1.5-Qwen3.8-27b.
• pp8192 @ dN = throughput of 8,192 new tokens processed on top of N cached context tokens — the agentic-coding profile (long, growing context; small incremental prompts). e2e ttft is end-to-end time to first response chunk.

llama-benchy (single RTX 5090, engine-native API)

test	t/s (mean ± std of 3)
pp2048	48,325 ± 445
tg256	240 ± 13
pp2048 @ 16k cached ctx	387,112 ± 3,950
tg256 @ 16k cached ctx	191 ± 17
pp8192 @ 32k cached ctx	746,186 ± 1,829
tg512 @ 32k cached ctx	217 ± 17
pp8192 @ 131k cached ctx	1,414,952 ± 2,835
tg512 @ 131k cached ctx	174 ± 20
pp8192 @ 200k cached ctx	1,737,206 ± 40,619
tg512 @ 200k cached ctx	163 ± 8

kaushikvira Qwen3.8-27B artifact family (single-RTX-5090 serving)

All artifacts are NInfer containers of Qwen3.8-27B derivatives, served by the same engine on the same box. Per-card A/B tables were measured pairwise in the same session; cross-card rows are from different sessions — treat small deltas as indicative.

artifact	base	uncensored	IFBench prompt-strict	GSM8K-200	decode tok/s	tg512 @ 131k ctx	status
nvfp4full-dflash2 (v2)	Qwen3.8-27B	no	—	—	146.4*	—	superseded by v3 (same weights, v2 container)
nvfp4full-dflash2.v3	Qwen3.8-27B	no	65.0*	96.5%*	146.4*	162.9	available
swift-abliterated v3	d0xin Swift-1.0 (huihui-abliterated)	yes	66.3	95.5%	148.7	173.7	available
swift15 v3	ukisai Swift-1.5	no	69.0	95.0%	160.8	153.4	rollback profile
thinkingcap v3	BottleCapAI ThinkingCap	no	68.0	95.5%	167.6	177.3	available (PolyForm — non-commercial)
swift15-uncensored-ajgazin v3	ukisai Swift-1.5 + ajgazin/orcarouter ablation	yes	70.3	94.5%	165.7	187.9	current production

* values for the nvfp4full-dflash2.v3 row come from its same-session A/B against swift-abliterated (66.3-vs-65.0 etc. are paired measurements, not independent runs).

---

Acceptable Use Policy & Legal Notice

By downloading, possessing, or using this artifact you agree to the terms below.

Permitted use

Research, personal/local experimentation, red-teaming, safety research, and evaluating alignment-removal techniques — subject to all applicable laws and regulations.

Prohibited use

You must not use this model, alone or in any pipeline, to:

• carry out, plan, or facilitate any activity that is illegal in your jurisdiction;
• produce content that harms, endangers, defrauds, harasses, or exploits others — including but not limited to weapons, malware, exploitation of minors, targeted harassment, or disinformation presented as fact;
• provide medical, legal, or financial advice presented as professionally qualified;
• violate the rights of any person or entity, including intellectual property and privacy rights.

Your responsibility

This model is a capability, not a judgment. Abliteration removes refusal behavior; it does not make outputs trustworthy, factual, or legal. You are solely and fully responsible for every prompt you send, every output you generate, and every use you make of them. The publisher of this artifact (kaushikvira) does not monitor, endorse, or take any part in downstream use.

No warranty / limitation of liability

The artifact is provided "AS IS", WITHOUT WARRANTY OF ANY KIND, express or implied, including merchantability, fitness for a particular purpose, and non-infringement. To the maximum extent permitted by applicable law, the publisher shall not be liable for any claim, damages, or other liability, whether in contract, tort or otherwise, arising from, out of, or in connection with this artifact or its use. Nothing in this card limits liability where limitation is not permitted by law.

Reporting

If you become aware of misuse of this model, report it to the platform where the misuse occurs and to the relevant authorities.

HF-format variant (vLLM / SGLang — JSON-schema structured output)

NInfer does not support JSON-schema output; if you need structured output or the standard vLLM/SGLang stack, use the HF-format NVFP4 weights:

• Qwen3.8-27B-swift-abliterated-NVFP4-HF (ours) — the exact pre-conversion source of this container. Sibling quantization of the same base: dragooy/Swift-Qwen3.8-27B-abliterated-NVFP4.

If you benchmark an HF-format variant on your hardware, please share numbers in the repo's Community tab — we collect them on the variant's card.

License

Apache-2.0, inherited from the base artifact chain. Abliteration and base-model licenses: see the credited repositories above — all credit to d0xin, ukisai, huihui-ai, z-lab and Neroued.