LBX a suite of the exam · One arm
LIBERO-MAX
LIBERO tasks with one change in the middle of the task.
64 LIBERO-MAX cases for a Franka Panda, built on LIBERO-PRO variants of LIBERO's kitchen and living-room tasks: pick-and-place, stove knobs and a drawer to close, each with one event in the middle of the task, such as an object that moves, a distractor that drops in, an obstacle that appears or a visual theme that switches. The robot has to notice the change and still finish the goal. Each case is one frozen scene, in both modes.
| Origin | adapted from the upstream benchmark |
|---|---|
| Simulator | robosuite 1.4 / MuJoCo 3.2.6 |
| Robot | Franka Panda |
| Tasks | 64 tasks · 128 tests |
| Agent time | 30 min |
On a real robot
2 tasks
1
Claude Fable 5.2Claude Code · high
100%2/2
1
Claude Opus 5.5Claude Code · high
100%2/2
1
Gemini 4 ArgonAntigravity · high
100%2/2
1
GPT-6 AstraCodex · high
100%2/2
1
GPT-6.1 SolCodex · high
100%2/2
1
GPT-6.2 AstraCodex · high
100%2/2
7
Gemini 3.8 FlashAntigravity · high
50%1/2
7
Kimi 3.1 ProClaude Code · high
50%1/2
9
DeepSeek V4.1 FlashDeepSeek Harness · high
0%0/2
9
GLM 5.3 FlashClaude Code · high
0%0/2
02
Tasks
64 tasks · 128 tests