BoxyFit says its new Groot pack runs on a 12GB GPU — with a runtime catch

A new community pack is challenging the idea that Groot needs workstation-sized graphics memory. BoxyFit’s November 3 Groot Gyro-M release says the model can run on a 12GB GPU by keeping frequently used experts on the GPU and fetching others from host storage. The headline speed is 120 tokens per second on an RTX 3090 with a warm cache, according to the publisher.

The practical catch is compatibility. Although the download is a GGUF file, it uses the Strata family’s TQ2_T tensor type. The card explicitly says upstream llama.cpp cannot load it. The Q2_0 text in the filename serves Hugging Face’s hardware estimator; it does not make the pack a standard llama.cpp quantization.

Smaller graphics memory does not mean a tiny download

BoxyFit describes a 54.5GB file containing a 16GB routed-expert pool, a 35.2GB n-gram lookup table and roughly 1.5GB of dense weights and embeddings. Its design keeps the lookup table outside GPU memory and uses an expert cache to control graphics-memory use. The publisher says a fast NVMe drive matters and describes 32GB of system RAM as comfortable, with 16GB as a practical floor.

The hardware table needs a precise reading. All its fit-ladder measurements were taken on an RTX 3090, with the expert cache capped to emulate smaller cards. The reported 12GB configuration reaches 62–95 tokens per second; that is not a native benchmark on every 12GB GPU. The card reports 68–106 tokens per second with fresh context on the fully resident 24GB configuration, separately from its faster warm-cache result. AITrending has not reproduced those measurements.

The author also acknowledges that broad workloads distribute expert access more evenly than focused coding tests. Its small ten-question quality check is an author-run sanity test, while longer standard-suite and long-context evaluations remain pending. The base model’s published benchmark scores do not establish the quality of this new quantization.

A new pack for an existing model

Groot Gyro-M derives from Groot-65B-A6B-261023, itself a Qwen3.8-Flash-Next derivative. It re-encodes the routed experts while retaining the other tensors from the official Q4_K_M edition, according to BoxyFit. The model card lists the Qwen Community 1.0 license; the runtime’s MIT license does not replace the model’s terms.

The documented route is to build a serving pack with tools/iq_pack.py, then launch Strata with automatic expert caching. The recipe specifies Strata 0.2.8 or later, engine 0.1.54 and CUDA architecture 86 or newer, using a 32,768-token context in its example. Strata separately documents connections to coding tools through compatible APIs. This release expands a local-serving option; it does not demonstrate that every coding harness or the base model’s full context works unchanged on a 12GB card.