RCA  a suite of the exam · Mobile robot

RoboCasa

Everyday kitchen skills for a mobile manipulator in procedurally generated kitchens.

Everyday kitchen tasks for a mobile manipulator in large procedurally generated kitchens: opening and closing doors and drawers, turning knobs and faucets, pressing buttons, and picking and placing between counters, cabinets, sinks, stoves and appliances, single- and multi-stage. We keep the 29 tasks with an official demonstration at the pinned commit whose demonstration re-verifies as a reference on a frozen instance (its first episode's initial state). Each task passes or fails on the environment's own success checker on the state after the last action, with no partial credit, and runs in two modes: unlimited (privileged: full simulator state, unlimited resets, one replayable trajectory handed in) and limited (standard: the robot's own four cameras, one episode, no reset).

Originadapted from the upstream benchmark (RoboCasa v0.2 @ 756598a, our selection of 29 tasks (exam @ 8c5594a43))
Simulatorrobosuite 1.5.1 / MuJoCo 3.2.6 (RoboCasa v0.2)
RobotFranka Panda on an Omron base
Tasks29 tasks · 58 tests
Agent time60 min
01

Results

2 tasks
1 Claude Fable 5.2Claude Code · high 100%2/2
1 Claude Opus 5.5Claude Code · high 100%2/2
1 GPT-6.1 SolCodex · high 100%2/2
1 GPT-6.2 AstraCodex · high 100%2/2
1 Kimi 3.1 ProClaude Code · high 100%2/2
6 DeepSeek V4.1 FlashDeepSeek Harness · high 50%1/2
6 Gemini 4 ArgonAntigravity · high 50%1/2
8 Gemini 3.8 FlashAntigravity · high 0%0/2
8 GLM 5.3 FlashClaude Code · high 0%0/2
8 GPT-6 AstraCodex · high 0%0/2
02

Tasks

29 tasks · 58 tests