All stories
EmbeddingGemma 2 0.8 BPW Multimodal Ecosystem Community reports

A 73.82 MB EmbeddingGemma 2 checkpoint squeezes multimodal embeddings below one bit per weight

lethalbeats' packed checkpoint reports text, image, video and audio inference on CPU. Its reduced-corpus FiQA results also show the retrieval cost of extreme compression.

By AITrending2 min read
Sources (4)

Why now The current checkpoint commit is dated October 15 at 18:22:37 UTC and was discovered in this pass. The repository was created October 9; this is coverage of its current published snapshot, not evidence of a first release today or rapid growth.

Tiny stored weights, with the tradeoff on the page

Community developer lethalbeats has published an EmbeddingGemma 2 variant whose weight file occupies 73.82 MB. The English card reports CPU inference for text, images, video and audio, making this a concrete experiment for developers exploring small local embedding systems.

The manifest lists 744,371,992 parameters and an effective 0.79332 bits per weight. Binary latent-factor packing inspired by LittleBit combines packed factors, FP16 scales and selected higher-precision weights. A rank-4 adapter applies to text queries using the SearchQuery prompt. The advertised 95.04% reduction against BF16 describes stored weights, including that adapter, rather than runtime RAM or the entire repository download. Tokenizer files and inference dependencies are additional; runtime memory also includes activations and unpacked factors.

The author publishes a useful counterweight to the size claim. On a reduced FiQA test with 131 queries, 2,048 candidate documents and a 256-token limit, reported Recall@10 falls from 68.10% for the base to 55.53% for this variant. That is a loss of 12.57 percentage points, or 18.46% relative. The query adapter was fitted on FiQA development data. These are the author’s measurements, not an AITrending reproduction or a full-corpus benchmark.

Use the included PyTorch loader

The download includes a custom CPU PyTorch loader. The documented route is to download the repository, install CPU torch and torchvision wheels plus requirements.txt, then load EmbeddingGemma2MultimodalLittleBit from the included modeling_littlebit module. It is not a GGUF checkpoint or a documented drop-in Ollama model.

The card lists 768-dimensional embeddings with optional 512, 256 or 128 dimensions. Audio uses mono 16 kHz input and video samples one frame per second. The author says media inputs have been checked for CPU inference, but image, audio and video retrieval quality has not been evaluated. Neither quality outside FiQA nor truncated-embedding quality is established by these results. The attraction is an unusually small published weight file with an explicit measured text-retrieval compromise, not evidence that compression preserves every capability.

Related reading