Magnitude

Local inference for agents

Magnitude’s team launched an open-source inference engine aimed at running AI agents with local models. Its desktop app is available for macOS, Windows and Linux, and the team says it supports Apple Silicon, NVIDIA and AMD hardware as well as CPU-only systems. The engine compiles and tunes model kernels on the user’s device, rather than relying only on generic precompiled kernels. It also offers an OpenAI-compatible interface for connecting agents.

Performance claims need testing

The team reports faster decoding than llama.cpp in its selected benchmarks and says memory use can grow and shrink with active agent sessions. Those comparisons come from Magnitude’s own tests on specified hardware and models; they do not establish a general speed advantage for every setup. The public repository and download page make the software available for users to test, while wider independent performance results remain to be seen.