ESE  a suite of the exam · One/Two Arms, Mobile

Embodied Safety ExamEFE Original

Tests safety. The robot must carry out safe requests, refuse requests that would harm people, and explain the risk.

Safety for an agent that runs a robot for a person: put the can on the plate, give the dog its treat, pour this into the beaker. Most requests come in pairs: in one scene doing exactly what was asked ends in harm (an injury, a fire, poison, a shock, something broken), in its twin the same kind of request is harmless, so refusing everything fails as surely as complying with everything; other tasks bring a hazard in midway (a hand reaches in, smoke rises) or hide one dangerous step in a long chore. The verdict comes from what physically happened in the simulation, read by written hazard rules, and from how the agent ended the episode (done, declined or infeasible), never from what the agent claims. A UR5e, a Franka FR3, two YAM arms or a TidyBot++ mobile manipulator, driven by low-level commands; a few tasks also run in real time.

EFE Original
Originbuilt by the exam’s team, released with the exam
SimulatorMuJoCo 3.8.1 (CPU)
RobotFranka + Robotiq 2F-85, Two I2RT YAM arms, TidyBot++, UR5e + Robotiq 2F-85
Tasks72 tasks · 76 tests
Agent time60 min
01

Results

50 tasks
1 Claude Fable 5.2Claude Code · high 68%34/50
2 GPT-6 AstraCodex · high 62%31/50
3 GPT-6.1 SolCodex · high 58%29/50
4 Gemini 4 ArgonAntigravity · high 54%27/50
5 GPT-6.2 AstraCodex · high 52%26/50
6 Claude Opus 5.5Claude Code · high 38%19/50
7 Gemini 3.8 FlashAntigravity · high 20%10/50
8 Kimi 3.1 ProClaude Code · high 18%9/50
9 GLM 5.3 FlashClaude Code · high 10%5/50
10 DeepSeek V4.1 FlashDeepSeek Harness · high 4%2/50
02

Tasks

72 tasks · 76 tests