Research question
Can coding agents act as robotics engineers that keep improving their robots through competition?
A robotics engineer works in a loop. They build a controller or train a policy, test it, field it, watch where it fails, revise it and test again. Coding agents can now run parts of this loop on their own: they write robot programs, launch policy training and read the results. This makes them candidates for autoresearch on robot policies, where the agent runs the whole engineering loop without a person in it. Whether the loop actually makes the robot better, match after match, has not been measured.
Embodied Olympics measures it through competition. Seven coding agents build, run and revise the systems that control robots in 18 sports on six simulated robots, from humanoids to a racing drone. Each agent starts without supplied motor skills, works within fixed time and compute budgets, and plays against systems that other agents build and revise. In 16 sports the match keeps running while the agent thinks. Competition suits the question: scores give an objective result, opponents expose weak strategies, and repeated matches show whether revisions help. We call a coding agent together with the programs it writes to control robots an agentic robot.
What we found
Overall results
Where the loop works
- Agents deliver a working system in 983 of 1,044 seat-matches (94%) across all sports and both formats.
- The best first programs score 56% of the points against fixed opponents, and their robots perceive the ball and predict its path well.
- Over a season, Gemini 4 Argon rises from 16% to 36%, GPT-6 Astra from 34% to 48% and Kimi 3.1 Pro from 6% to 13%.
Where the loop breaks
- The side with the better first program wins 89 of 98 decisive matches, because revisions barely change the program.
- Agents analyze the opponent correctly in 66% of seat-matches but field a change in 26%, and a change that wins more points in 4%.
- Against fixed opponents, revisions within a match gain 0.2 percentage points; of 305 self-assessments that claim an improvement, replays confirm 35.
How far from strong systems
- The strongest agent’s programs win 43% of their games against the strongest scripted opponent.
- Four agents lose all 32 wheeled-soccer matches against the RoboCup champions’ software without scoring a goal.
- When a badminton shuttle lands out of reach, Fable’s robot moves to it in 2 of 166 cases and Astra’s in 5 of 142; our scripted practice player does in 112 of 116.
How Embodied Olympics measures the engineering loop
Each agent builds its system alone, in a private development world with a practice opponent
Each agent works from its own seat, a container with fixed compute and time. The seat holds a development world, a private copy of the match world that the agent can reset and, in most sports, query for the true state of the ball and the robots. Where the sport has an opponent, the development world also has a black-box practice player to play against. The seat provides simulators and general-purpose libraries, but no motor skill such as walking or kicking, and no reward function, curriculum or example policy. Whatever the robot does in a match, the agent built.
In the match itself, the agent’s system sees what a real robot would: camera images, noisy joint and inertial readings, and the referee’s announcements. No sport reveals the true position of the ball or of the opponents. The system acts through low-level commands in the form the real robot accepts, from joint targets on the humanoids to thruster valves on the stunt car. In most team sports one agent commands every robot of its side, up to 11.
An agent builds in its own seat, plays in a shared world, and revises between points. Paper Figure 2. Each agent develops its system in its own seat, with a private development world and a practice player (top left). All systems act in one shared official world (top). Play alternates with review windows, and every fielded version is saved for Checkpoint Evaluation (bottom left). The Continual format keeps the workspace across matches, and the Episodic format resets it (bottom right).

A match has one preparation phase, 30 minutes to 2 hours in which the official world stands still while the agent builds. Then play and review alternate. In 16 of the 18 sports, the official world advances one control step every 10 or 20 milliseconds of real time whether or not a command has arrived, so time spent deciding is time in which the world moves on. Curling and billiards are turn-based, with a chess clock per side. After each point or goal, the world freezes for a review window in which the agent can inspect what happened and revise its system. During play, the agent can also send commands from its shell, start programs that run continuously and may call the model, and replace a running program at any moment.
Episodic matches test one build; Continual seasons test whether experience accumulates
Two formats set what an agent keeps from one match to the next. In the Episodic format, every match starts from a fresh workspace. It tests what an agent builds from scratch and how it adjusts its system within one match. In the Continual format, the agent keeps its workspace, its harness memory and its session history through a season of matches, and it also receives the referee log, statistics and video of its earlier matches. Programs, trained weights and experience can accumulate. This is the format that asks whether the engineering loop keeps improving the robot.
Each sport holds one competition in each format. In Episodic, every pair of players meets once, in independent matches on the same hidden seed. In Continual, the same round robin is followed by play-offs. Six agents enter both formats: GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon, Kimi 3.1 Pro, DeepSeek V4.1 Flash and GLM-5.3 Flash. Claude Fable 5.2 joined after the Continual seasons and enters Episodic only, so every season result below leaves out the agent that does best in Episodic. Each agent runs in its provider’s own harness at high reasoning effort, so each result belongs to a model and its harness together.
Replaying saved programs separates an agent’s own improvement from its opponent’s changes
A better match result does not show that an agent improved, because the opponent also revises its system. Checkpoint Evaluation separates the two. Every version that commands a robot is saved as a restartable copy, and two versions per player and match are replayed: the Initial program, in control when the first game starts, and the Final program, in control at the end. Each replayed program runs from its saved copy without access to the model.
The replays answer different questions. Playing the two Final programs of a match again on new seeds tests whether the result holds. Playing all four pairings of the two players’ Initial and Final programs measures what a revision gains against the current opponent. Playing both versions against three scripted opponents of rising strength that we wrote for each sport, L1, L2 and L3, plus the practice player, measures what a revision gains against a reference that is the same for every agent.
The main measure of a program in this post is its level: the share of points or goals it scores against these fixed opponents, own over own plus opponent. Unless stated otherwise, a level averages seven real-time duel and team sports with equal weight. A seat-match is one agent’s side of one match. Error intervals are 95% bootstrap intervals, and a change is measurable when its interval excludes zero.
18 sports on six robots, each played under the real sport’s rules and the real robot’s limits
The 18 sports span four kinds of event, solo, duel, team and race, on six simulated robots: the Unitree G1 and Booster T1 humanoids, a RoboCup Small Size League wheeled robot, a racing quadrotor, a full-size race car and a thruster stunt car. Each sport follows three principles. A referee program enforces the rule book of the real sport, and every deviation is declared. Each robot keeps the vendor’s model, actuator limits and latency, and balls, implements and surfaces are calibrated against real measurements. And each venue is scaled so that the robot can perform the basic actions of the sport within these limits. The badminton court, for example, is scaled to 0.4 of its official size, so that a receiver that knows the true state returns at least 90% of legal serves. A robot that misses the shuttle therefore fails because of its program, not the court.
The 18 sports, from sprints and javelin to soccer, racing and drone flight. From paper Figure 1; each sport has its own page on the Embodied Olympics site. Solo, duel, team and race events on the Unitree G1 and Booster T1 humanoids, an SSL wheeled robot, a racing quadrotor, a full-size race car and a thruster stunt car. Curling and billiards are turn-based; the other 16 sports run in real time.

Build: the first program largely decides the match
Agents almost always deliver a working system, but its quality differs widely
The first step of the loop works. Across all sports and both formats, 983 of 1,044 seat-matches (94%) end with a fielded program and a result other than a forfeit. Delivery is lower for GLM-5.3 Flash (122 of 148) and Gemini 4 Argon (152 of 172).
What the agents deliver differs widely in quality. The Initial programs, fielded at the first game before any match experience, score 56% of the points against the fixed opponents for Claude Fable 5.2, 38% for GPT-6 Astra and 36% for Claude Opus 5.5, and under 10% for Kimi 3.1 Pro, DeepSeek V4.1 Flash and GLM-5.3 Flash. These levels pool seven real-time duel and team sports.
One build, from first version to deployment. From the Embodied Olympics video: GPT-6 Astra develops five versions of its stunt-car controller, runs 818 practice jumps and a parameter sweep to test and debug it, and fields it in the match, where it scores 149.1 points.
The side with the better first program wins 89 of 98 decisive matches
In the Episodic format, the side whose Initial program has the higher level wins 89 of the 98 matches that have a winner and two different Initial levels. The first program decides so much because the Final program is barely better than it. In the seven pooled sports, the level goes from 56% to 57% for Fable 5.2, from 38% to 40% for Astra and from 36% to 37% for Opus 5.5 between the first game and the end of the match. Whatever the agent learns during the match, little of it reaches the program.
Given a GPU, agents train policies to walk and hand-write their ball skills
Agents spend the GPU on learning to move. In the 425 Episodic seat-matches with a GPU, they launch policy training in 56%, mostly reinforcement learning for walking or getting up. Ball skills are trained in only 14% of the seat-matches that train at all. Agents train in 97% of the seat-matches in sports that demand walking and running, but in 21% in net sports, where they write the stroke by hand, tune it with parameter search and keep the robot’s feet planted.
Training is not what separates good systems from weak ones. Within the same agent and sport, launching training shows no detectable association with the Final level (−1.1 points, 95% interval −7.8 to +4.1).
Agents spend the GPU on locomotion, and level and cost spread widely. Paper Figure 6. (a) Level against cost. Level is the share of points that the Final program wins against the fixed opponents, pooled over seven real-time duel and team sports (Episodic). Cost is the median cost of one Episodic seat-match at list prices, on a log scale. (b) Share of the preparation time with the seat’s GPU busy, by sport (Episodic, the 12 sports with a GPU); in javelin the GPU searches throw plans.

What an agent spends and how good its system is do not move together
A seat-match costs about $17 for Claude Opus 5.5, with a Final level of 37%, and about $29 for Claude Fable 5.2, with a level of 57%: about 2 points of level per dollar for both. GPT-6 Astra and Gemini 4 Argon obtain about 1 point per dollar ($42 for 40%, $17 for 19%), and Kimi 3.1 Pro about 0.2 ($37 for 8%). DeepSeek V4.1 Flash and GLM-5.3 Flash cost $0.41 and $1.35 per seat-match, and their programs reach about 4%. Model use is metered but not capped, so these costs are what each agent chose to spend.
Deploy: robots plan the action well but fail to complete it
For every agent, perception and prediction run ahead of the outcome
The paper splits physical skill in real-time play into four aspects. State estimation asks whether a system knows where things are, prediction whether it knows where they will be, execution whether the body does what the program intends, and outcome whether the action succeeds. Each aspect has one metric per sport, scored so that 0 is no better than a control version of the program and 100 is perfect. In humanoid soccer, for example, state estimation is the angle between a robot’s head camera and the true direction of the ball, and the control is the same program with the head held still.
For every agent, the outcome score is the lowest of the four. Fable 5.2 and GPT-6 Astra score highest in all four aspects. In state estimation they average 81 and 74, against 13 for DeepSeek and GLM. When a robot must aim its own camera, the agents split into two groups: in humanoid soccer, the head cameras of Fable, Astra, Opus and Gemini stay within 6° of the ball, and those of Kimi, GLM and DeepSeek point 28° to 46° away. In prediction, Fable scores 88 and Astra 69, and only Fable releases the javelin close to the longest throw its release speed allows. In execution, wheels are easy and legs separate the agents: every agent scores at least 76 in wheeled-soccer drive speed, while in HSL Team the robots of Opus, Astra and Gemini get up after nearly every fall and those of GLM and DeepSeek almost never. In outcome, Fable and Astra score 67 and 63, Opus and Gemini 34 and 33, and Kimi, DeepSeek and GLM at most 5.
Agents perceive and predict better than they act, and the robots rarely move to a ball out of reach. Paper Figure 4. (a) to (d) One panel per aspect of physical skill, one row per agent. Small dots are sports and the large marker is their mean; panels cover different sports, so compare agents within a panel. 0 equals the program’s own control and 100 the ideal (for speeds, our reference program). (e) Badminton: of the shuttles that land out of the robot’s reach from where it stands, the share it moves to. The star is our practice player, which reads the true shuttle state.

Perceive, predict, hit: the steps that work. From the Embodied Olympics video: in table tennis, the program tracks the ball, predicts where to meet it and hits there, 2.3 cm from its prediction.
In net sports, the robots stand still while the ball lands out of reach
In most failed returns, the racket never touches the ball: 70 of Opus’s 77 failed returns in badminton, and 52 of Astra’s 59 in tennis. The robots stand still. When a shuttle lands out of reach from where the robot stands, Fable’s robot moves to it in 2 of 166 cases and Astra’s in 5 of 142, while our practice player, a scripted reference opponent, does so in 112 of 116. In beach volleyball, robots touch 0 to 42% of incoming balls, and no team sends a touched ball back over the net.
The programs aim and swing well at a ball that arrives near the robot. In badminton, the process scores of Fable and Astra, the mean of their scores for the steps before the outcome, are both 92, against outcome scores of 62 and 52. Yet the programs keep the robot’s legs fixed during a badminton rally, the hand-written strokes described under Build. In the sprint and hurdles, where no ball has to be reached, Fable and Astra finish nearly every run.
Robots rarely pass to each other, and humanoids almost never
In humanoid soccer, where one program controls three humanoids, only 3 of 291 goals follow an assist. In wheeled soccer, where one program controls 11 wheeled robots, 42 of 265 goals (16%) do. Splitting a team among agents does not help either. In HSL Team, under the rules of the RoboCup Humanoid Soccer League, each of the three robots has its own agent, and teammates coordinate over a shared workspace and a rate-limited text channel. Three agents that each write one humanoid never measurably beat a single agent that writes all three. For Astra the single agent is measurably better (+3.6 goals per game, 95% interval 1.1 to 6.2), and for Fable 5.2, Gemini and Opus 5.5 the interval includes zero. The single agent also sees the poses of all three robots, so this comparison does not separate coordination from the wider view.
The rare case: five passes, one goal. From the Embodied Olympics video: a wheeled-soccer attack in which the team passes five times before it scores. In wheeled soccer, 16% of goals follow an assist; in humanoid soccer, 1%.
Diagnose: agents read the opponent correctly but rarely act on it
Two thirds of reviews analyze the opponent correctly; 4% lead to a change that wins more points
The review windows give the agent what an engineer would want: a pause, the record of the last point, and the chance to change the program. The paper follows each seat-match through four steps. The agent’s reviews notice how the opponent plays, the reviews analyze it correctly, the agent fields a changed program, and that program wins more points. Two LLMs from different providers mark the first three steps, and a step counts only when both agree (Cohen’s κ from 0.70 to 0.85).
In 75% of the 515 Episodic seat-matches, the agent notices how the opponent plays, and in 66% it analyzes the opponent correctly. But only 26% field a changed program, and only 4% field one that wins more points. Memory does not close the gap. In the Continual format, where notes carry over, agents notice the opponent in 87% of 502 seat-matches, and the chain still ends at 4%. Replays show no clear gain from the changes either: 2.8 percentage points, 95% interval −3.0 to +9.2.
From noticing the opponent to winning more points, the chain thins out at the change. Paper Figure 5. (a) Episodic seat-matches in which the agent notices how the opponent plays, analyzes it correctly, fields a changed program, and fields one that wins more points. (b) Wheeled-soccer goals that follow an assist, per agent. (c) Ten consecutive wheeled-soccer matches of Claude Opus 5.5 against GPT-6 Astra: Opus’s goal margin (open markers: tied after regulation), with bars for the matches in which one side changes its play and arrows to the other side’s answer.

What the full chain looks like when it does complete. From the Embodied Olympics video, Claude Opus 5.5 against Kimi 3.1 Pro in badminton: after missing serves, the agent notices that the opponent’s serves repeat the same paths, practices returning that serve in its development world, and returns it in the match. Most seat-matches stop before the change.
Changes made for a known opponent do not win the rematch
The Continual play-offs give agents a second meeting with an opponent they have already played. Before such a rematch, 16 of the 64 preparations change the program for that opponent, and 11 of them test the change on a stand-in rebuilt from the opponent’s record. Yet first-meeting losers who changed their program won 2 of 13 rematches, and those who did not change it won 4 of 19. The counts are small, but nothing in them suggests that preparing for a known opponent pays off.
Over ten straight matches, one agent did adapt to its opponent
Two extra series of ten wheeled-soccer matches show what the official matches rarely do. Fable 5.2 won all 10 of its matches against Astra. Opus 5.5 won 7 and drew 3 against Astra, and in this series Astra adapted. In matches 1 to 3, Opus scored 5 of its 7 goals off set pieces. By match 4, Astra had changed its goalkeeper and its defense of the penalty area, none of Opus’s 31 corners in matches 4 to 10 scored, and Opus’s last 7 margins were 0 or 1 goal. After Opus saved all 5 penalties in the match-4 shoot-out, Astra copied Opus’s kick from the match video and won the next shoot-out. The records do not show which change produced which result, but the series shows that the loop can close when an agent meets the same opponent often enough.
Revise and verify: revisions help some agents, slowly, and agents overrate them
Within a match, revisions add almost nothing against fixed opponents
Replaying each agent’s Initial and Final programs against the same fixed opponents shows how little a match changes the program. Of 232 Episodic seat-matches, the Final program is measurably better than the Initial one in 17 and measurably worse in 12. In the 195 seat-matches with both kinds of replay, the Final program gains 3.9 percentage points over the Initial program against the current opponent’s Initial program, but 0.2 points against the fixed opponents. What a revision gains is mostly specific to the opponent of the day.
Over a season, three of six agents improve, and they improve match by match
The Continual format is the direct test of the loop: the agent keeps its code, weights and notes, and gets the record and video of every match. Over a season, Gemini 4 Argon rises from 16% to 36% of the points against the fixed opponents (+19 points, 95% interval 12 to 27), GPT-6 Astra from 34% to 48% (+14, 9 to 19) and Kimi 3.1 Pro from 6% to 13% (+7, 5 to 9). The gains build up match by match. The other three agents show no measurable change. Gemini gains the most and still ends below Astra and Opus 5.5. Opus starts highest and shows no measurable improvement.
Two caveats bound this result. Season gains combine match experience with additional development time, so they do not isolate learning from experience. And Claude Fable 5.2 did not enter the Continual seasons, so whether the strongest Episodic agent improves over a season is not known.
Three of six agents improve over a season, and replays rarely confirm the gains agents report. Paper Figure 7. (a) Level of each Continual agent’s program after each match of the season (0: the first program), averaged over its own season sports, so not comparable with the Episodic levels above. Bands are 95% bootstrap intervals of the level; gains are tested within paired resamples, so their intervals are narrower than the bands. (b) The 305 of 457 self-assessments that claim an improvement, by the change that replay measures.

Carrying experience into the next match. From the Embodied Olympics video: in a Continual season, GPT-6 Astra reviews its previous match and practices on 72 of its own falls before the next one.
Agents report improvements that replays rarely confirm
The last step of the loop is verification: checking that a change actually helped. Agents rate their own changes more favorably than the replays measure. Of the 305 self-assessments that claim an improvement, 35 coincide with a measured gain, 256 with no measurable change and 14 with a measured loss. After a measured loss, the agent still reports an improvement in 14 of 22 cases. These assessments were labeled by one LLM. Our reading, which the paper does not test: an agent that believes a change worked has less reason to look for a better one, so optimistic self-assessment may be part of why revisions stall.
How far today’s agentic robots are from strong systems
Against strong scripted opponents and RoboCup champion software, the best systems still lose
The strongest agent, Claude Fable 5.2, builds programs that win 43% of their games against L3, the strongest of our scripted opponents. The 43% averages over eight duel and team sports (curling and billiards are turn-based and left out, as are tennis and beach volleyball, where no rallies developed), with a draw counting as half a win. GPT-6 Astra’s programs win 20% and Opus 5.5’s 16%, and the other four agents’ programs at most 4%.
Against the software of the RoboCup Small Size League champions, Fable 5.2, Astra, Opus 5.5 and Gemini each play eight consecutive wheeled-soccer matches, with their workspaces carried over from match to match. All four lose all 32 matches without scoring a goal. Eight matches with a carried-over workspace are the setting in which an engineering loop should make progress, and none of the four agents scores in any of them.
The agents separate into clear tiers, and replays keep the order
The medal boards and a Bradley-Terry rating (on an Elo scale with the average player at 1000) give the same order of agents, with more than 700 points between the first and the last. Replaying the matches with the Final programs on new seeds keeps this order in 20 of 21 pairs in Episodic and in all 15 in Continual; the one exception, Kimi and Gemini, has overlapping intervals. For the question of this post, the spread matters more than the order. Even Gemini, which gains the most over its season, still ends below Astra and Opus 5.5.
The agents separate into clear tiers in both formats. Paper Figure 3. (a) Medals on the 16 medal boards of each format, from the final official standings (tennis and beach volleyball award no medals, as serves decided their points). (b) Bradley-Terry strength on an Elo scale: dots are fitted on official duel and team matches, ticks on match replays. Claude Fable 5.2 entered the Episodic format only.

What’s next
Limitations
All competitions run in simulation under real-hardware actuator limits; real-world deployment remains future work. In the Episodic format, each pair of agents meets once per sport on one hidden seed, so results for a single sport rest on few matches. Replays of the saved programs confirm the overall order but do not repeat the agents’ development. Season gains combine match experience with additional development time, and Claude Fable 5.2 joined after the Continual seasons. Behavioral labels come from LLM annotators, two of which are among the evaluated agents, and still need human validation. The comparison is with fixed opponents and RoboCup champion software, not with human engineers working under the same budgets.
Closing the loop from noticing a failure to fielding a fix that works
Current coding agents build a strong first system but improve it poorly from experience. The break is not in one place. Robots plan actions well but fail to complete them, agents analyze an opponent correctly but rarely turn that analysis into a better program, and agents rate their own changes above what replays measure. For autoresearch on robot policies, the hard part is the second half of the loop: turning what the agent sees in play into a change, and checking that the change helps. Embodied Olympics makes each step of that loop measurable, so progress on it can be tracked as agents improve.
Data and citation
The paper, data and code will be linked here when they are released. Sports, the leaderboard and match replays are on the Embodied Olympics site. Numbers in this post come from the paper’s official results, frozen on 13 November 2026.