Granite Heretic gets smaller GGUF downloads, starting at 1.7 GB

Smaller files for local experimentation

Richard Young’s community Granite 4.2 3B Heretic build now has smaller GGUF download options. An October 15 commit adds IQ3_M and IQ4_XS files, listed in the English model card at approximately 1.7 GB and 2.0 GB respectively. It also adds an importance matrix and replaces the Q4_K_M and Q5_K_M files with imatrix-based builds.

That gives local-model users more choices when storage or memory is tight. Download size is not the same as total runtime memory, and the source does not establish a universal hardware minimum or a measured speed improvement for these new files. The author still recommends the roughly 2.2 GB Q4_K_M option as a balance between size and quality.

The repository includes recipes for Ollama, llama.cpp and llama-cpp-python. Its Q6_K and Q8_0 files remain available. This is a concrete expansion of an existing community download collection: the repository and its initial quantizations date to September 28, rather than October 15.

Fewer refusals, with an explicit tradeoff

Heretic modifies the model’s refusal behavior. Young reports that the base model refused 98 of 100 harmful test prompts, compared with 27 for the modified build. Those figures describe the author’s test, not an independently established result across every use case.

The card says trials with still fewer refusals also degraded normal behavior, so the author selected a compromise. Its low reported KL divergence is not proof that every original capability remains intact. The news here is the new range of downloadable quantizations, with the modification and its claimed tradeoffs made clear.