Qwen3.8 Flash Next

Model card checked 2026-10-03 · Memory fit estimated; speed unknown unless a separately linked measurement matches your configuration.

Also searched as: qwen3.8 flash next

Stock checkpoint: 125B backbone + 51B n-gram embedding + 4B MTP. Q4 budget estimated as 180 × 0.5 × 1.2 = 108 GB before workload-specific KV cache. Not the REAP-288 conversion; do not transfer its 48 GB Mac result. A published fork that streams the routed experts from the internal SSD measured ≈27.6 tok/s decode on a 64 GB M5 Pro MacBook, but that path needs the fork, a 36 GiB expert cache and the internal SSD; stock llama.cpp fails to load the checkpoint at all. SGLang supports it in a tagged build now: the cookbook page (re-read 2026-10-24) describes it as a 176B / 6B-active hybrid, publishes day-0 NVIDIA and AMD recipes, and says to use an SGLang build that includes Qwen3.8-Flash-Next support (v0.5.22 or later), while its per-hardware cells still run on the lmsysorg/sglang:qwen38flashnext and dev-qwen38-next-local nightly images, so check which build tag you are on. The v0.5.20 release notes first listed Qwen3.8-Flash-Next as a model added in that release, and v0.5.22 reports a 12% B200 decode gain for it. The cookbook also states the NVFP4 checkpoint is 126 GiB (78 GiB of weights plus a 47.7 GiB FP8 N-gram table) and does not fit one DGX Spark's 128 GB pool unless that table is kept file-backed on NVMe; the fully resident shape is two Sparks with TP=2. A third-party engine, halogen-flash-server 0.12.0, documents its own 5.53-bpw 118 GiB build at a 258,794-token context on a 128 GB Ryzen AI Max+ 395 with the amdgpu GTT ceiling raised to about 99% of RAM (45.0 tok/s decode, 1,114 tok/s prefill), but it serves only this model and is not a general runtime path. A second third-party path reached the full 262,144-token native window on the same class of machine: the checkpoint author's own Q4_0-ROCmFP4 build (5.51 bpw, 113.47 GiB) on a llama.cpp fork carrying the ROCmFP4 tensor types streams its 51.2B-parameter n-gram table from the SSD and measured 22.87 tok/s decode at an 8,192-token window and 15.22 tok/s at 131,072, so stock llama.cpp still will not load this checkpoint and none of these speeds is a stock-runtime expectation. Strata is another model-specific offload path: the 2026-09-29 RTX 5070 12 GB / 64 GB RAM IQ3_XXS run on 0.1.26 reports 49.0 tok/s decode and 1,602.3 tok/s prefill at 128,478 prompt tokens. A 2026-09-30 community replication on RTX 5090 32 GB / 64 GB RAM, 0.1.29 and original IQ2_XS reports median 165.0 tok/s decode at 128,000 prompt tokens over three runs. These quantizations and workloads differ; they do not change this record's conservative Q4 budget or verify a generic runtime command. Conditions and pinned reports are in the workspace measurements.