All stories
Gemma 4 12B MXFP4 by FreedomAISVR Ecosystem Community reports

Downloaded this Gemma 4 MXFP4 build? Its author says download it again

FreedomAISVR says an earlier MXFP4 file was a Q4_0 hybrid. The replacement adds a corrected template and author-reported 256K-context results on a 16GB GPU.

Sources (2)

Why now October 21 repository changes replace the mislabeled artifact and document the correction; a 14:50 UTC card revision adds hardware tests. This is a community rebuild, not a new Google base-model release.

Anyone who downloaded FreedomAISVR’s Gemma 4 12B MXFP4 GGUF before October 21 should download it again, according to its author. The revised English model card says the earlier file was a Q4_0 hybrid distributed under an MXFP4 name, with a stale chat template. The replacement is described as genuine MXFP4 transformer weights with Google’s canonical July 9 template.

This is a correction to a community conversion of Google’s QAT model, not a fresh Google model announcement. The author preserves the earlier artifact in repository history. The new text-model file is 6.88GB, with token embeddings retained at Q8_0 and norms and biases at F32; the card lists 328 MXFP4 tensors. Users should identify the actual repository and revision rather than assume every Gemma GGUF is affected.

A 16GB long-context recipe, with a speed tradeoff

The October 21 14:50 UTC card revision adds measurements on an RTX 5060 Ti 16GB, CUDA 13.2 and llama.cpp b1868. The author reports processing a 255,396-token prompt with Q8_0 key and value caches, retrieving one fact placed about 60% into the input. Reported GPU use was 9,794MiB for weights, cache and compute buffers. A successful single retrieval is useful evidence of that test, not comprehensive proof of long-context reasoning quality.

The full-prompt test reports 767.4 prompt tokens per second and 20.5 generated tokens per second with tuned settings. The stated tuning gain is only 1.5% for prompt processing and zero for generation. Separate depth sweeps show generation slowing sharply near 250K context, so fitting the window should not be confused with maintaining short-context speed.

The documented text command uses a 262,144-token context, Q8_0 caches, GPU offload and --jinja. For image input, the author requires the accompanying F32/BF16 vision projector and warns that converting it to F16 causes failures in this setup. AITrending did not reproduce these measurements or inspect the weight tensors independently.

There is another important distinction for downloaders: Google trained this QAT model for Q4_0. The author explicitly says MXFP4 uses a different quantization grid and was chosen for hardware support and portability, not because it beats Q4_0 in quality. The actionable news is the corrected file and template, with an attributed local-inference recipe, rather than a blanket quality upgrade.