ABC a suite of the exam · Two arms
ABC Sim
Two YAM arms at a table: sort, count, spell, pour, sweep, hang a mug and balance a ball on a tray.
Twenty-nine tasks of ABC Sim, the MuJoCo simulator of ABC (Scalable Behavior Cloning with Open Data, Training, and Evaluation), every task its catalogue scores: two I2RT YAM arms side by side at a table, driven by joint-position targets. Throw or put bottles in a bin, pick parts off a moving conveyor, put exactly N objects in an opaque box, grab one item from clutter, search drawers, pour, hang a mug, turn mugs upright, sweep scraps, load a dish rack, sort nuts and bolts or lego blocks, set up chess pieces, spell words with letter blocks, hand an item from one gripper to the other and keep a ball balanced on a tray held with both arms. Each task is one frozen scene from an upstream demonstration; the goal must hold for a second to count.
| Origin | adapted from the upstream benchmark |
|---|---|
| Simulator | ABC Sim on MuJoCo 3.6.0 |
| Robot | Two I2RT YAM arms |
| Tasks | 29 tasks · 60 tests |
| Agent time | 60 min |
On a real robot
Different models were evaluated on different tasks, so the results are not fully comparable.
Tasks
29 tasks · 60 tests