RPT  a suite of the exam · One arm

RoboPaintEFE Original

Tests fine tool control. A robot arm reproduces a given picture with a brush or pen, from Chinese calligraphy to oil painting.

A Franka Panda holding a brush or a pointed pen copies a target picture onto a paper sheet, dipping in an ink stone or paint wells, with a practice sheet beside it: Chinese calligraphy in regular, running and cursive script, English copperplate lettering, and acrylic and oil paintings. Success needs the sheet to match the target closely (overlap, edges, stroke coverage and stray ink; for paintings, colour and structure) and the work to be done with strokes, the way a person writes or paints: filling the shape with thin back-and-forth lines, zigzags or spirals fails. Millimetre-wide strokes are hard to judge from camera images, and in standard mode the agent gets no score until the episode is over.

EFE Original
Originbuilt by the exam’s team, released with the exam
SimulatorManiSkill 3.0.1 / SAPIEN 3.0.3 + brush models (calligraphy, broad nib, libmypaint 1.6.1)
RobotFranka Panda
Tasks43 tasks · 86 tests
Agent time60 min
01

Results

22 tasks
1 GPT-6.2 AstraCodex · high 95%21/22
2 GPT-6 AstraCodex · high 32%7/22
3 GPT-6.1 SolCodex · high 27%6/22
4 Claude Fable 5.2Claude Code · high 9%2/22
5 Claude Opus 5.5Claude Code · high 5%1/22
5 Gemini 4 ArgonAntigravity · high 5%1/22
7 DeepSeek V4.1 FlashDeepSeek Harness · high 0%0/22
7 Gemini 3.8 FlashAntigravity · high 0%0/22
7 GLM 5.3 FlashClaude Code · high 0%0/22
7 Kimi 3.1 ProClaude Code · high 0%0/22
02

Tasks

43 tasks · 86 tests