RLB a suite of the exam · One arm
RoboLab
Tabletop tasks that test language grounding, from colours and counts to relations and order.
53 of its 120 tabletop manipulation tasks for a fixed-base arm, built to test language grounding and goal specification rather than locomotion: colour and size references, counting, spatial relations, sorting, ordered stacking, reorientation. Success is a composable predicate over object state, with partial credit from scored subtasks.
| Origin | adapted from the upstream benchmark (v0.3.1 (2026-08-11)) |
|---|---|
| Simulator | Isaac Lab 2.3.2 / Isaac Sim 5.1 (NVIDIA) |
| Robot | Franka + Robotiq 2F-85 |
| Tasks | 53 tasks · 106 tests |
| Agent time | 60 min |
On a real robot
6 tasks
1
Claude Fable 5.2Claude Code · high
33%2/6
1
GPT-6.1 SolCodex · high
33%2/6
1
GPT-6.2 AstraCodex · high
33%2/6
4
Claude Opus 5.5Claude Code · high
17%1/6
4
Gemini 3.8 FlashAntigravity · high
17%1/6
6
DeepSeek V4.1 FlashDeepSeek Harness · high
0%0/6
6
Gemini 4 ArgonAntigravity · high
0%0/6
6
GLM 5.3 FlashClaude Code · high
0%0/6
6
GPT-6 AstraCodex · high
0%0/6
6
Kimi 3.1 ProClaude Code · high
0%0/6
02
Tasks
53 tasks · 106 tests