OPEN PHYSICAL AI WORKBENCH

v0 objective

Ground a robot command, execute it in MuJoCo, and inspect the control evidence.

v0 demonstrates an auditable robotics stack: natural language becomes task parameters, the deterministic MuJoCo controller executes the manipulation, and the UI shows the resulting trajectory, actuator commands, coordinate deltas, metrics, and artifacts.

MuJoCo liveCommand loop liveAdvisory model live

v0 command demo

Command a lift-cube task and inspect the MuJoCo evidence.

IDLEAdvisory model live
action: live arm target previewcube: x 0.250 / y 0.120 / z 0.200gripper: closedtarget z: 0.200 m
Browser MuJoCo WASM loading
Live arm director
Directly manipulate robot targets and execute the generated command.
Physical model controls
Lift the cube 18 cm normally, move it 12 cm right, rotate the arm 180 degrees, and keep it there.
Typed task compiler
Live

Deterministic command compiler is available before the first run.

MuJoCo physics
Live

The gateway executes a real MuJoCo manipulation run and returns artifacts.

Simulation gateway
Live

The deployed FastAPI robotics service is the executor.

Advisory model
Live

qwen3_5-4b (Qwen visual advisory model) is routed through https://serverless.roboflow.com.

LeRobot / VLA lane
Next

Not active in v0. The arm run already emits the episode, action, state, and metric evidence that a learned policy lane needs.

What v0 is proving

01
User command
A lift-cube instruction with target height, speed, and optional seed.
02
Grounding
qwen3_5-4b proposes advisory JSON; the typed compiler validates task semantics before simulation.
03
Control
Deterministic MuJoCo baseline controller for repeatable manipulation evidence.
04
Execution
Deployed Simulation Gateway runs the physics backend and produces artifacts.
05
Evidence
Replay video, metrics, states, actions, events, and manifest returned to the UI.

Capability boundary

Live now
Advisory model grounding, typed compiler validation, MuJoCo execution, replay, metrics, state/action traces, and artifacts.
What the advisory model does
It can propose task parameters from text or a scene prompt. It is not the low-level controller or the policy authority.
Not v0
LeRobot/SmolVLA learned policy inference, imitation learning, visual servoing, and robot-model generalization.

Next policy lane

MuJoCo WASM / mujoco-js
Move the physics loop into the browser for continuous interactive simulation.
MuJoCo Menagerie
Replace the simple cartesian arm with credible robot models.
LeRobot + SmolVLA
Add dataset/policy evaluation and compare learned control against the baseline.

Multi-embodiment roadmap

The arm is the validated public lane for this release. The platform contract can support mobile manipulators and bimanual systems later, but the visible demo stays focused on a manipulation task that works end to end.

Manipulation v0
Live
Single-arm lift-cube task with explicit grasp, actuator channels, replay, metrics, and artifacts.
Mobile manipulation
Next
Base plus arm tasks: navigate to object, align, grasp, carry, and evaluate contact/pose success.
Bimanual control
Next
Two-arm coordination tasks with richer action spaces and LeRobot-compatible episode export.
Goal seeking
Scenario lane
Mobile manipulation tasks can add navigation before grasping, while keeping the same action/evidence contract.