Coding agents have passed many exams in the digital world. What is their first exam before entering the physical world?

EmbodiedFirst Exam

A Terminal-Bench for coding agents in robotics.

harbor run -d embodied-first-exam -a codex -m openai/gpt-6-astra

↓
1,156
tasks
35
benchmark suites
47
robots

481 core tasks. A task counts as solved only if the agent solved every mode it has (standard, and privileged or realtime where the task has them).

481 tasks

Click a model to see its suites one by one.

Full leaderboard, per suite →

Task diversity

The suites

Budgets. Every attempt gets real time and real compute, as an engineer would: from 30 minutes to 4 hours of agent time per task, depending on the suite, and a container with 2 to 16 CPU cores and 4 to 39 GiB of memory. Suites built on GPU simulation backends (Isaac Sim, OmniGibson, SAPIEN) give the agent a GPU, and some tasks in Embodied Olympics - Fun and RoboEngineer offer one for training a policy.

The unified framework

Harbor · one command
Question
goal in words Completely separate the two claw pieces and briefly hold both still.
frozen scene
  • rebuilt bit for bit
  • start state checked
  • same scene in every setting
Model
any model
Harness
CodexClaude Codeany harness
Interfacehow the code reaches the robot, per setting
Standardreal-time
reach
APIrobot API only
sees
RGBdepthjointsno object poses
sends
action chunks (T, dim)
How it runs underneath →
Privileged
reach
inside the simulator
sees
full stateresets ∞everything, at will
sends
trajectory.npzreplayed fresh
How it runs underneath →
Simulators47 robots
MuJoCoIsaac SimOmniGibsonSAPIENGenesis
recorded
Verifier
sandbox
live verdict
replay
10
steps · cost · video

Our Team

* Equal contribution · § Core contributor · † Equal advising