Swift apps get local speech engines and decisions that return probabilities

A Swift app can now change its speech-recognition engine and ask a model for a typed decision with probabilities, using two community packages introduced by Mattt on October 29. AnySpeechRecognizer and AnyDecisionModel extend the approach behind AnyLanguageModel to transcription and classification. These are community libraries for Apple platforms, rather than new Apple frameworks or newly trained foundation models.

The practical pitch is a support workflow: transcribe a call, identify speaker turns, then classify a ticket’s urgency or destination without parsing a generated paragraph. Developers can choose local models for supported backends, while hosted decision services remain separate options.

Speech recognition with a choice of engine

AnySpeechRecognizer mirrors the modern SpeechAnalyzer API. For code using its shared API, the announcement describes replacing the Speech import and choosing a different transcriber configuration. Apple’s engine remains the default; Parakeet variants run through the package’s Core ML runtime and support live input, provisional results and word timing.

The package also supports Qwen3-ASR in 0.6B and 1.7B sizes. That is a different tradeoff: it produces final segment results, with segments up to 40 seconds, and does not supply Parakeet-style provisional text, word timing or confidence. The author recommends it more for files than live transcription.

SpeakerDiarizer adds NVIDIA Sortformer alongside a timed transcriber to label up to four speakers. Overlapping speech remains difficult: the documentation warns that a quieter interjection may not receive its own recognized words. Mattt’s published performance measurements come from an M4 Max Mac Studio; the announcement says the package has been tested on macOS 27, without iPhone testing so far.

Decisions without parsing prose

AnyDecisionModel exposes yes-or-no probabilities, multiple-choice results and ordinal scores. A ticket-routing example returns a Swift enumeration plus a distribution over departments. Its next-token backends read allowed answer-label probabilities; a separate MLX decision-head backend supports Cloudflare’s Clef models.

The six documented backend options cover MLX, MLX decision heads, Core AI, llama.cpp, Jev and Ollama. That includes a route for local GGUF files through llama.cpp. The author’s Emoji Magnet demo makes the idea tangible by asking which emoji fit a typed phrase; the published video uses the hosted Jev backend, rather than demonstrating local inference speed.

Probability output is not a correctness guarantee. The README says raw model probabilities can be overconfident, and its confidence value measures how concentrated the distribution is. Calibration and option-rotation tools are available, with extra evaluation work and computation. Typed answers alone also do not establish resistance to injected instructions.

What developers need

The announcement provides Swift Package Manager examples starting at AnySpeechRecognizer 0.1.0 and AnyDecisionModel 0.6.0. Both require Swift 6.2 or later and macOS 26 or iOS 26 or later for Apple-platform use. Parakeet, Sortformer and MLX require Apple silicon; the decision package’s MLX build tasks need Xcode’s Swift 6.4 or later and the Metal toolchain. Core AI requires Xcode 27 and an OS 27 runtime.

Model downloads are explicit for speech assets. Local decision backends require the corresponding model files and hardware; Jev uses a TypeSafe API key, while local Ollama requests need no API key. The announcement also flags a current dependency conflict between AnyDecisionModel’s CoreAI trait and AnySpeechRecognizer. Teams should choose a compatible backend combination rather than assume every advertised option can be enabled together.