TensorFold brings mixed-width models to Macs, with a claimed 2x decode gain
Version 1.0.10 serves specified MLX oQ checkpoints on Macs; its headline speedup comes from the author's M3 Ultra measurement, with explicit release-test gaps.
Different bit widths in one checkpoint
TensorFold 1.0.10 brings Mac serving support for mixed-width MLX checkpoints of Qwen3.8-27B, GLM-5.3-Flash and Qwen3.8-Flash-Next, according to the project’s October 14 release. Each module can load at its own width from two to eight bits, with supported group sizes of 32, 64 or 128. Uniform checkpoints retain the previous release’s kernels and launches.
The practical hook is running supported oQ model downloads through TensorFold’s native inference engine. The project uses Zig and talks directly to Metal or CUDA; reading the MLX checkpoint format does not require installing a Python or MLX runtime to serve it.
The 2x claim has a specific scope
The author reports that Qwen3.8-27B oQ2 decoded code twice as fast and chat 1.6 times as fast on an M3 Ultra with the new mixed-width kernels. Reported prefill gains were 2.4 to 3.3 times. These are the maintainer’s measurements for that model, checkpoint and machine, not AITrending tests or a promise of a universal Mac speedup.
The release also lists consequential limits. GLM-5.3-Flash across two Macs or using expert parallelism still accepts only 4-bit, group-64 checkpoints. No M5-generation Mac or two-Mac pair ran the release build, and GLM-5.3-Flash and Flash Next were not served from that build. Their oQ changes rely on contributor runs at specified commits and kernel checks. Known issues from 1.0.9 still apply.
A local server for existing clients
On supported Macs, the documented Homebrew route is brew install ashhart/tensorfold/tensorfold; existing users can run brew upgrade tensorfold. tensorfold pull REPO downloads a supported checkpoint, and tensorfold serve REPO starts serving it. The default endpoint is http://127.0.0.1:8080/v1, with OpenAI-compatible routes and an Anthropic-compatible messages route.
Local serving defaults to loopback access without an API key. Exposing it remotely is a separate configuration choice, with an API-key-file option documented in the runbook. The engine is Apache-2.0 licensed, while downloaded weights retain their own licenses and memory requirements. AITrending reviewed the English release and tagged documentation without downloading models or reproducing its benchmarks.