Ecosystem
bloomery puts Bolt 74B on a 24GB card
Community reports 2 sources
B
bloomery
Tool Oct 18, 2026
A Rust inference server for local MoE models, GPU and host offloading, embeddings and multi-Spark expert routing.
Visit primary link
2 sources attached
What happened lately
bloomery timeline
bloomery is an MIT-licensed inference engine written in Rust, including its CUDA kernels. It exposes llama-server-compatible OpenAI and Anthropic HTTP interfaces and loads supported GGUF models. Model weights, hardware requirements and licenses remain separate from the server.
Version 0.3.4 adds Nemotron 3.5 Bolt on a 24GB GPU, embedding models alongside chat, and routed experts across DGX Sparks. Release binaries target Linux x86-64 with AVX2, glibc 2.34 or newer, NVIDIA compute capability 8.6 or newer and R580 drivers. Windows uses WSL2; DGX Spark requires a source build. The project’s hardware-specific performance figures are author measurements, not AITrending tests.