MW+  a suite of the exam · One arm

MetaWorld+

All 50 Meta-World v3 environments for a Sawyer arm, one frozen instance each.

All 50 of Meta-World's v3 environments for a Sawyer arm (reward v2), one frozen instance each: push, pick and place, open doors, drawers and windows, press buttons, turn dials and faucets, insert pegs, hammer, sweep and more. Each task passes or fails on Meta-World's own success checker on the state after the agent's last action, with no partial credit, and runs in two modes: unlimited (privileged: full simulator state, unlimited resets, one replayable trajectory of at most 500 actions handed in) and limited (standard: a robot behind a service, four fixed cameras and the arm's own state, one episode, no reset). Privileged is reported as a control; the standard mode is the benchmark.

Originadapted from the upstream benchmark (our benchmark (exam @ 8c5594a43): Meta-World v3 @ 59fc34d, all 50 environments)
SimulatorMuJoCo 3.3.0 (Meta-World v3)
RobotRethink Sawyer
Tasks50 tasks · 100 tests
Agent time30 min
01

Results

2 tasks
1 Claude Fable 5.2Claude Code · high 100%2/2
1 Claude Opus 5.5Claude Code · high 100%2/2
1 Gemini 4 ArgonAntigravity · high 100%2/2
1 GPT-6.1 SolCodex · high 100%2/2
1 GPT-6.2 AstraCodex · high 100%2/2
1 Kimi 3.1 ProClaude Code · high 100%2/2
7 DeepSeek V4.1 FlashDeepSeek Harness · high 50%1/2
7 Gemini 3.8 FlashAntigravity · high 50%1/2
7 GLM 5.3 FlashClaude Code · high 50%1/2
7 GPT-6 AstraCodex · high 50%1/2
02

Tasks

50 tasks · 100 tests