LBX  a suite of the exam · One arm

LIBERO-MAX

LIBERO tasks with one change in the middle of the task.

64 LIBERO-MAX cases for a Franka Panda, built on LIBERO-PRO variants of LIBERO's kitchen and living-room tasks: pick-and-place, stove knobs and a drawer to close, each with one event in the middle of the task, such as an object that moves, a distractor that drops in, an obstacle that appears or a visual theme that switches. The robot has to notice the change and still finish the goal. Each case is one frozen scene, in both modes.

Originadapted from the upstream benchmark
Simulatorrobosuite 1.4 / MuJoCo 3.2.6
RobotFranka Panda
Tasks64 tasks · 128 tests
Agent time30 min
01

Results

2 tasks
1 Claude Fable 5.2Claude Code · high 100%2/2
1 Claude Opus 5.5Claude Code · high 100%2/2
1 Gemini 4 ArgonAntigravity · high 100%2/2
1 GPT-6 AstraCodex · high 100%2/2
1 GPT-6.1 SolCodex · high 100%2/2
1 GPT-6.2 AstraCodex · high 100%2/2
7 Gemini 3.8 FlashAntigravity · high 50%1/2
7 Kimi 3.1 ProClaude Code · high 50%1/2
9 DeepSeek V4.1 FlashDeepSeek Harness · high 0%0/2
9 GLM 5.3 FlashClaude Code · high 0%0/2
02

Tasks

64 tasks · 128 tests