DeepSeek Flash community test reports 830 aggregate tokens/s on four Blackwell GPUs

A local serving result with the waiting time attached

Community developer Infatoshi reports 830.06 aggregate output tokens per second from an attention-edited DeepSeek V4.1 Flash 1017 model at 32 concurrent requests on four RTX PRO 6000 Blackwell workstation GPUs. The new October 18 bundle includes timing cells, charts and reproduction code. It is an interim local measurement, not an official DeepSeek leaderboard result or a claim that one user receives 830 tokens per second.

The reasoning-enabled workload uses a warm 40,000-token public-code prefix plus roughly 1,000 new observation tokens, asking for a small cache bug fix and two tests. DSpark proposes up to five tokens with adaptive verification. Across three measured legs, the author reports mean aggregate output of 131.55 tokens/s at concurrency one, 425.19 at eight and 830.06 at 32. Output counts include reasoning tokens.

The visible-answer measurements tell a different part of the experience. The mean of each leg’s median time to visible content was 9.65 seconds, 16.76 seconds and 30.14 seconds respectively. The first streamed event arrived much earlier. Combined throughput, first-event latency and the wait for readable content should therefore remain separate metrics.

The author explicitly says unrelated training and embedding jobs were active on the same hardware. The charts show observed minimum-to-maximum ranges across the three legs, not confidence intervals. AITrending checked the card against its published timing cells but has not reproduced the GPU runs.

Modified weights, bounded behavior evidence

The native reference applies pqhaz’s published attention edit to a pinned DeepSeek checkpoint. A separate rank-2 edit comes from mrfakename. Infatoshi says it originated neither method, and quantized-runtime qualification and the final method choice remain unfinished. The release contains no checkpoint weights or raw generated answers.

The capability table is also narrower than a general quality verdict. For example, the attention-edited model scores 156/164 on the local HumanEval protocol versus 157/164 for the original base; strict MMLU uses an eight-token cap and exact answer formatting. Those are single-completion local tests with different protocols from official leaderboards.

On 300 direct HarmBench prompts, the pinned classifier marks behavior success for 30 original-base outputs and 287 attention-edited outputs. The author reports a 95/100 classifier control check and discloses output caps. These counts describe that classifier and prompt set; they are not generic refusal rates or proof of universal compliance.

What reproduction requires

The documented route uses a Linux GPU host, Docker, NVIDIA Container Toolkit, Hugging Face downloads and uv, then serves the edited checkpoint through a pinned vLLM container. The launcher assigns four GPUs, binds its API to localhost, mounts model weights read-only and writes a cache directory. It also uses host networking and IPC and an unconfined seccomp setting, which are material permissions for anyone considering the script. The bundle does not quote a turnkey hosting price; hardware, storage and upstream access remain the operator’s responsibility.

For local-inference readers, the useful development is a published, inspectable measurement package that pairs throughput with latency and exposes its limitations. It does not yet settle which quantized format or model edit is best.