ESE a suite of the exam · One/Two Arms, Mobile
Embodied Safety Exam✦EFE Original
Tests safety. The robot must carry out safe requests, refuse requests that would harm people, and explain the risk.
Safety for an agent that runs a robot for a person: put the can on the plate, give the dog its treat, pour this into the beaker. Most requests come in pairs: in one scene doing exactly what was asked ends in harm (an injury, a fire, poison, a shock, something broken), in its twin the same kind of request is harmless, so refusing everything fails as surely as complying with everything; other tasks bring a hazard in midway (a hand reaches in, smoke rises) or hide one dangerous step in a long chore. The verdict comes from what physically happened in the simulation, read by written hazard rules, and from how the agent ended the episode (done, declined or infeasible), never from what the agent claims. A UR5e, a Franka FR3, two YAM arms or a TidyBot++ mobile manipulator, driven by low-level commands; a few tasks also run in real time.
| Origin | built by the exam’s team, released with the exam |
|---|---|
| Simulator | MuJoCo 3.8.1 (CPU) |
| Robot | Franka + Robotiq 2F-85, Two I2RT YAM arms, TidyBot++, UR5e + Robotiq 2F-85 |
| Tasks | 72 tasks · 76 tests |
| Agent time | 60 min |
On a real robot
Tasks
72 tasks · 76 tests