Qwen3.8-Flash-Next on one RTX 5090: turboderp claims 200+ tokens/s at 1.66bpw
A newly uploaded community quant targets a single headless RTX 5090, with vision and multi-token prediction. Its creator's speed claim comes with a 64K quantized cache and tight memory tradeoffs.
A community quant of Qwen3.8-Flash-Next is aiming at the single-GPU coding-agent crowd: turboderp says its 1.66bpw EXL3 build runs at more than 200 tokens per second on one headless RTX 5090. The model card says that configuration includes the vision tower and multi-token prediction head, without CPU offloading, using a 64K-token cache with K5V4 quantization.
The concrete new event is the October 20 weight upload to 1.66bpw_h4_ng3, which adds four model shards and an n-gram embedding file. This is a new compression option for an existing model, not a fresh Qwen base-model launch. The quant repository was created on August 31.
Small weights still need a carefully chosen setup
The 1.66bpw label does not mean every tensor uses that precision. The creator describes mostly K=1.5 expert tensors, a 4-bit output head and input embeddings, a 3-bit n-gram table and a 4-bit vision tower. The card also notes that the n-gram file is swappable and usually streamed from disk. Its no-CPU-offloading claim should not be read as saying every released file stays resident in GPU memory.
That distinction matters for anyone trying to reproduce the setup. Headless operation leaves display memory available for inference, while vision, speculative prediction and the cache compete for the remaining space. The quoted generation rate is an author claim for agentic workloads, not a measured completion time for an arbitrary coding job. AITrending has not run this checkpoint.
The community discussion is about the tradeoff, too
An English LocalLLaMA thread shared the quant and debated whether probability-distribution measurements adequately describe coding quality. Its original poster corrected the headline’s 36GB reference to under 32GB and said they did not own a 5090, so the post is not independent hardware replication.
Commenter ReturningTarzan described completing a long OpenCode task involving a C++ soft-body physics engine and vision feedback with a 64K cache on one 32GB GPU. The same account cautioned that the smallest quant still forces difficult choices between vision, multi-token prediction and context. That is a reported usage account, not a standardized coding benchmark.
The author-published chart measures divergence from the original model on a self-generated trace. Lower divergence is useful quantization evidence, but does not directly establish bug-fixing success or replace end-to-end tests.
Where to start
The download is an EXL3 branch, rather than a GGUF file. ExLlamaV3 lists Qwen3.8-Flash-Next multimodal support and recommends TabbyAPI for an OpenAI-compatible server. Its documented installation routes include prebuilt wheels or a source build with compatible CUDA and PyTorch; a model API subscription is not needed for local inference.
The weights use Qwen Community License 1.0. Its commercial Model-as-a-Service and AI Work Assistant provisions require a separate license, with an internal-use exception described in the terms. This is not an Apache-licensed checkpoint. For owners considering the smallest build, the useful experiment is whether its speed and memory savings preserve enough quality and context for their actual agent tasks.