All stories
Video-Point Model Launches Developing

VPM gives AI video cameras a shared 3D future, not just convincing frames

The research release pairs future multi-view video with persistent point trajectories, and publishes code, a robot checkpoint and datasets under noncommercial model terms.

By AITrending2 min read
Sources (3)

Why now The original preprint was submitted October 20 at 17:45 UTC. The reviewed repository now provides inference and training code and a VPM-DROID checkpoint, making the method inspectable beyond a demo page.
VPM method diagram connects multi-view history, instructions and point queries to future videos, actions and 3D point tracks.
The authors' architecture diagram explains joint video, action and point forecasting. It is an explanatory figure, not independent evidence of robot performance. Source: Byungwoo Jeon and Video-Point Model authors

An AI world model can make three camera views look plausible while giving each camera a different future. Video-Point Model tries to fix that by forecasting where the same physical points move in 3D. The October 20 preprint arrives with code, datasets and a VPM-DROID checkpoint for researchers to inspect.

VPM takes posed multi-view video histories and a language instruction, then jointly predicts future videos and persistent point trajectories. Its authors build on Cosmos3-Nano, adding camera conditioning and a trajectory head. Because each trajectory follows the same point through views and time, point supervision connects the cameras to a shared motion description. An action-enabled version also predicts robot action chunks.

The reported gain is a controlled training comparison

On the authors’ project page, adding point supervision under the same training budget lowers simulation FVD from 78.52 to 67.35 for robot scenes and from 101.37 to 81.92 for general scenes. Lower FVD is better. These are the team’s reported video-fidelity measurements, not independent replication or proof that arbitrary generated scenes obey physics.

The release describes VPM-Data as 10,000 simulated training clips, split between 8,000 tabletop and 2,000 urban clips. VP-WorldBench contains 1,200 held-out clips across simulated and real-world recordings, evaluating appearance, cross-view consistency and 3D motion accuracy. The current package should not be confused with the million-sample dataset or Cosmos3-Super version, both still marked as planned for December 21.

A research package, with a noncommercial license

The repository supplies inference and training code and a downloadable VPM-DROID checkpoint. Its documented example uses nine historical frames and intervening actions from three cameras to generate the next 16 frames per camera and a 16-step action chunk, with optional point tracks.

Setup uses uv, ffmpeg, CUDA 12.8 or 13.0 dependency groups and downloaded model assets. This is a GPU research workflow rather than a browser video generator. VPM code and weights are CC BY-NC 4.0, with attribution to Byungwoo Jeon; Cosmos-derived portions have OpenMDW 1.1 terms, and datasets retain their individual licenses. Public downloads do not grant unrestricted commercial use.

The authors also show real robot pick-and-place evaluation on three tasks, with 24 trials per task. Their scoring gives half credit when a successful placement touches the intervening pile of boxes. That limited setup is useful evidence about this experiment, not a general claim of reliable autonomous manipulation.

Related reading