All stories
Respan Gateway Launches Developing

Respan launches a Rust AI gateway where even routing is a plugin

The pre-alpha gateway translates supported model APIs and exposes Rust extension hooks, while its launch discussion makes the compile-time tradeoff explicit.

By AITrending2 min read
Sources (6)

Why now Respan introduced the Rust gateway on Hacker News October 20, after publishing v0.2.0 binaries at 06:53 UTC that day. The launch and dated artifacts establish a current event beyond repository activity.

Respan’s new Rust gateway takes the plugin idea all the way down to routing, protocol handling and observability. Introduced on Hacker News October 20, it offers a common endpoint for supported model APIs and a public Rust interface for teams that want to extend the gateway themselves.

The company describes the software as pre-alpha. Its author says the redesign followed difficulties maintaining a large wrapper around LiteLLM, including an upgrade-related connection problem whose cause remained unresolved. That is Respan’s account of its own infrastructure, not evidence that LiteLLM generally fails.

Keep the client, change the provider

The documented workflow is to supply provider keys, start the gateway and point a supported SDK’s base URL at it. OpenAI Chat Completions and Responses, Anthropic Messages and Gemini generateContent can translate through shared conversation types. Routing adds named models, fallbacks, retries and hedged requests; MCP integration can run tools for clients that do not implement MCP themselves.

The authors also describe realtime and media support, while explicitly warning in the launch post that not every realtime feature translates. A shared endpoint should not be read as a guarantee that every vendor-specific feature survives every conversion.

Plugins require a compiler for custom logic

In the discussion, Respan clarifies that these plugins are native Rust crates compiled into the application. Configuration changes for existing templates do not require a rebuild; adding custom logic does. They are not dynamically loaded extensions or a WebAssembly sandbox. Readers questioned the build burden and supply-chain risks, and the author acknowledged that crates still carry those risks.

The v0.2.0 release, published October 20 at 06:53 UTC, provides Linux and macOS archives and a Windows x86-64 ZIP. This directly contradicts a stale README sentence saying no release has been tagged. The author says a prebuilt Docker image was withdrawn before launch; users wanting a container must build it themselves at the time of that discussion.

The MIT-or-Apache-2.0 license does not include upstream model usage. Users bring provider credentials and pay applicable provider charges. The configuration guide says the gateway refuses a non-loopback listener without client keys, and documents separate admin permissions.

Self-hosting also needs a deliberate logging choice. When the optional Respan tracing plugin is configured, it sends model and tool traces to Respan, including messages and replies unless content = false is set. The gateway can run without the hosted platform; the tracing option is not a promise that all data remains local.

Related reading