All stories
OpenGCUA Launches Community reports

OpenGCUA opens a computer agent that mixes code, APIs and clicks

Tongyi-MAI's first research preview releases computer-use code and eight-model trajectories, plus a preliminary macOS harness and Cross-Device SDK. Its own GCUA model is still planned.

By AITrending2 min read
Sources (3)

Why now The English technical report's first arXiv submission is October 15, 2026 at 16:51:35 UTC. Its October 16 Hugging Face discussion presents the first preview; those dates are distinct from subsequent repository activity.
OpenGCUA authors' English stack diagram marking the agent design and trajectories released, macOS harness and SDK preliminary, and model, efficiency and evaluation work next.
Author-published research-preview diagram distinguishing released, preliminary and planned components. It is not an AITrending execution test. Rasterized and resized without cropping. Source: Tongyi-MAI / OpenGCUA authors

OpenGCUA gives computer-use researchers an alternative to treating every task as a sequence of screenshots and clicks. Tongyi-MAI and collaborators have opened a first research preview that lets a model combine GUI actions, accessibility information, application APIs and shell commands inside programs with loops and conditions.

The release includes an OSWorld integration example, evaluation trajectories from eight frontier models, a preliminary real-device macOS harness and a Cross-Device SDK. It is an agent system around existing models. The project’s own model for computer and browser use remains on the roadmap.

Let the model choose when to look

The authors call their approach a generalist computer-use agent. Instead of forcing a screenshot after every action, the model can request an observation when it needs one, choose an API or visual action, and maintain task state. Code execution allows an intermediate result to determine the next operation without another model call.

In their OSWorld-v2 comparison with batched GUI-only execution, the authors report a mean partial-score gain of 19.6 points across eight evaluated models and 53% fewer reported steps on average. These are results for the reported configuration and baseline, not a claim that every real-world task finishes twice as fast. AITrending has not reproduced the evaluation.

A preview with real device permissions

The documented source harness targets macOS 14 or newer and requires uv, Python 3.11-3.14, Rust/Cargo 1.89 or newer and Xcode Command Line Tools; Python 3.12 is recommended. Setup installs the SDK and runtime, initializes the native driver and Chromium support, and copies a local configuration file.

Users supply a model-provider API key and endpoint separately. Provider usage costs are not included in the harness. The driver requires Accessibility and Screen Recording permission, and the agent can invoke application and shell operations. Aborting a turn does not roll back operations already sent.

The SDK documents macOS, Android, browser and Linux adapters, while the native macOS app is described as an Apple Silicon local development build with public-release validation still outstanding. Those are different availability claims. The accompanying author diagram labels released, preliminary and planned components; it is not an AITrending execution test.

Related reading