Contribute

Call for contributors

The exam is an open agentic robotics benchmark, and it grows with its community. You can evaluate a model, integrate an agent harness, integrate tasks, or bring anything else that would make the exam better. All are very welcome, from labs, companies and individual researchers alike, and we would love to hear from you: here is how to contribute.

35
suites in the exam
47
robots
1,156
tasks2,294 tests

How to contribute

Evaluate a model

Have a new model? We run it on the exam with an existing harness, under the same budgets as every other agent, and its results join the leaderboard.

Get in touch by e-mail →

Integrate a harness

Bring your agent harness to the exam, so that any model can be tested in it and compared with Codex, Claude Code and the other harnesses on the same tests.

Get in touch by e-mail →

Integrate tasks

Add a single task or a whole benchmark suite: frozen scenes, a program that decides success and, where there is one, a reference solution. What a suite needs.

Get in touch by e-mail →

Something else

Real-robot evaluation, analysis of the agents’ runs, tools, or an idea we have not thought of: tell us what you have in mind and we will find a way to make it part of the exam.

Get in touch by e-mail →

All go to embodiedfirstexam@gmail.com. For questions, join the exam’s Discord or its .

What a suite needs

From proposal to leaderboard

  1. ProposeE-mail a short proposal.proposal-approved
  2. BuildTwo tests per task, each a task folder, from a generator.-privileged-standard
  3. Self-checkRun the checks the reviewers will run.run_all.shoracle = 1nop = 02 replays
  4. ReviewA maintainer, a cheat trial and a model trial.cheat trialhuman review
  5. MergeBase images published, the suite added to the site.eai-<name>
  6. On the boardIts tests join the exam’s evaluations.both modesevery log

Where a new suite helps most

Credit

Have an idea? Let’s talk.

A model, a harness, a task or anything else, write to us and we will work it out together.