Coding agents have passed many exams in the digital world. What is their first exam before entering the physical world?
EmbodiedFirst Exam
A Terminal-Bench for coding agents in robotics.
- 1,156
- tasks
- 35
- benchmark suites
- 47
- robots
481 core tasks. A task counts as solved only if the agent solved every mode it has (standard, and privileged or realtime where the task has them).
Click a model to see its suites one by one.
Task diversity
The suites
EFE Suites9 suites
Existing Suites26 suites
Budgets. Every attempt gets real time and real compute, as an engineer would: from 30 minutes to 4 hours of agent time per task, depending on the suite, and a container with 2 to 16 CPU cores and 4 to 39 GiB of memory. Suites built on GPU simulation backends (Isaac Sim, OmniGibson, SAPIEN) give the agent a GPU, and some tasks in Embodied Olympics - Fun and RoboEngineer offer one for training a policy.
The unified framework
Completely separate the two claw pieces and briefly hold both still.
- rebuilt bit for bit
- start state checked
- same scene in every setting
- reach
- APIrobot API only
- sees
- RGBdepthjointsno object poses
- sends
- action chunks
(T, dim)
- reach
- inside the simulator
- sees
- full stateresets ∞everything, at will
- sends
- trajectory.npzreplayed fresh
Our Team
Kangrui Wang*§Northwestern
Qineng Wang*§Northwestern
Changhe Chen*§Northwestern
Pingyue Zhang*§Northwestern
Chi Gui*§Northwestern
Feishi Wang*§Northwestern
Canyu Chen*§Northwestern
Wenli Xiao§CMU
Xiaoyu Huang§UC Berkeley
Tonghe Zhang§CMU
Linjie Li§UW
Zihan WangNorthwestern
Minyeong KimStanford
Yiyou SunUC Berkeley
Mingkai DengCMU
Jinyu HouCMU
Qi ZhuNorthwestern
Han LiuNorthwestern
Eric P. XingCMU
Huaxiu YaoUNC-Chapel Hill
Zhiwen FanTexas A&M
Yuejiang LiuStanford
Dawn SongUC Berkeley
Pieter AbbeelUC Berkeley
Li Fei-FeiStanford
Jiajun WuStanford
Yuke ZhuUT Austin
Guanya Shi†CMU
Ruohan Zhang†Northwestern
Manling Li†Northwestern
* Equal contribution · § Core contributor · † Equal advising