ENG  a suite of the exam · Diverse agile mobile robots

RoboEngineerEFE Original

Tests the sim-to-real engineering loop. The agent builds a controller in a public simulator, deploys it to one with hidden dynamics, and fixes it from what the deployment shows.

The loop of real sim-to-real engineering, in simulation. The agent tunes a controller or trains a policy in a public simulator, deploys it to a private evaluator whose dynamics differ in hidden ways, and revises it from the noisy telemetry and camera frames that come back, over at most three deployments. The robots are quadrotors (landing on a pad, flying figure eights in wind), a racing car (tracking, drifting into a parking bay, timed laps), a Unitree Go2 (body attitude, jumps, weaving between poles) and a Unitree G1 that learns to walk over rough and stepped ground on the agent's own GPU. The chosen policy passes only if it succeeds on every episode of a held-out set on the same hidden dynamics, so tuning to the deployments it has seen is not enough.

EFE Original
Originbuilt by the exam’s team, released with the exam
SimulatorMuJoCo and learned dynamics surrogates (quadrotors, a racing car, Unitree Go2 and G1)
RobotAnyCar racing car, Intel Aero quadrotor, Neural-Fly quadrotor, Unitree G1, Unitree Go2
Tasks30 tasks · 30 tests
Agent time60 min, 240 min
01

Results

15 tasks
1 Claude Fable 5.2Claude Code · high 47%7/15
2 GPT-6 AstraCodex · high 33%5/15
3 Claude Opus 5.5Claude Code · high 27%4/15
3 Gemini 4 ArgonAntigravity · high 27%4/15
5 GPT-6.1 SolCodex · high 20%3/15
5 GPT-6.2 AstraCodex · high 20%3/15
7 DeepSeek V4.1 FlashDeepSeek Harness · high 13%2/15
7 GLM 5.3 FlashClaude Code · high 13%2/15
7 Kimi 3.1 ProClaude Code · high 13%2/15
10 Gemini 3.8 FlashAntigravity · high 0%0/15
02

Tasks

30 tasks · 30 tests