Learning from Fleet Experience: A Collective Learning Paradigm for Robots
Tsinghua University · AgiBot · Infinigence AI
01
CEL: learning from a fleet’s own experience
As robots carry out tasks, they collect real-world experience that can help robot foundation models improve during deployment. Larger fleets collect this experience in parallel, potentially accelerating policy improvement. One way to realize this potential is to learn from human corrections. However, this requires experts to monitor execution and provide corrective actions when robots need help. The demand for expert supervision increases as fleets grow. Human supervision thus becomes a bottleneck: robots can gather experience at increasing scale, but their ability to learn from it remains constrained by the availability of experts.
We introduce Collective Experience Learning (CEL), a paradigm in which robots share experience generated during deployment to improve a common policy. The figure above shows how the source of actions used for learning changes: human demonstrations in SFT, expert interventions in (M)HG-DAgger, and successful robot executions in the CEL workflow shown here. After expert-supervised initialization, CEL pools the fleet’s experience and returns updated policies to the robots. One robot’s successful actions can then help improve the policy used by others, without additional expert demonstrations or corrective actions. In our implementation, humans provide outcome labels and reset environments.
For phone packaging, a long, multi-stage industrial task, the policy’s success rate increased from 79.2% to 98.3% in evaluations on two robots of the same model that had not collected training data. Over 120 trials per policy, failures fell from 25 to 2, a 92% reduction. The benefits of scale are also evident: with collection time per robot held fixed, increasing the training fleet from 16 to 96 robots raised success from 86.7% to 98.3%. More robots translated into more reliable task execution.
- Largest training fleet
- 96robots
- Separately trained tasks
- 5real-world tasks
- Phone Packaging
- 98.3% success
- Phone Packaging failures
- 92% reduction vs. initial policy
Putting CEL into practice requires overcoming two challenges. First, how can robots find useful experience in large collections of trajectories with only binary success-or-failure labels? Second, how can a large robot fleet keep collecting experience despite limited communication bandwidth and local faults?
02
FERT: using failures to find what worked
As robots carry out tasks, CEL gathers a growing pool of experience across the fleet. A task can run for several minutes and span thousands of timesteps, yet its final outcome supplies just one label: success (1) or failure (0). The challenge is to turn this sparse feedback into effective training. We developed Failure-guided Experience Retrieval Fine-Tuning (FERT) to organize fleet-generated experience for retrieval and policy fine-tuning. Successful trajectories enter a shared experience buffer. Observations preceding a failure serve as queries to find similar successful segments, matching visual features and joint state over time. The actions in these segments supply the training targets, and the segments are sampled more often during fine-tuning. A success recorded by one robot can therefore help others learn from situations where they struggled.
Learn the task
Human demonstrations teach the pretrained policy how to perform the task.
SFT
Refine with corrections
Human operators take over during multi-robot practice and demonstrate corrective actions.
(M)HG-DAgger
Improve through practice
The fleet learns from its successes, using failures to guide what to revisit.
FERT
Robots execute the policy independently. Each completed trajectory receives a binary task-outcome label. The interactive example follows four illustrative trajectories.
Successful trajectories enter a rolling collective experience buffer. Failure contexts provide queries for retrieval. Robots contribute at different rates while policy optimization continues.
A frozen copy of the policy’s vision tower encodes camera views. Spatial pooling reduces each view’s 16 × 16 visual-token grid to 4 × 4.
Visual features and joint state are normalized separately, then concatenated into a retrieval representation. Successful-trajectory embeddings are cached as experience arrives.
A query contains up to 90 frames of observations, ending 10 frames before the failed episode’s recorded endpoint.
Slope-constrained subsequence dynamic time warping aligns this query with each successful trajectory. FERT retrieves up to 10 candidates and expands each interval by 30 frames on either side.
Retrieved successful intervals increase the sampling weight of overlapping action chunks. Training samples successful actions using these weights, then returns the fine-tuned policy to the fleet.
Pink bars show chunks overlapping retrieved intervals.
03
From folding a T-shirt to packing a phone
We used five tasks to test how learning from shared experience helps robots handle flexible objects, control contact and use tools. Each task has its own policy, trained on dual-arm AgiBot G2 robots.
Phone Packaging
The robot packs a manual, charging cable, charger, phone case and phone, then aligns and closes the lid. The sequence takes roughly two to three minutes, and each placement matters: an item left in the wrong position can make a later step harder.
98.3% success rate118 successful trials out of 12096 training robots
04
Five real-world tasks: how much does collective experience help?
We evaluated FERT on five real-world manipulation tasks: flower arrangement, clothes folding, phone charging, stool assembly and phone packaging. FERT outperformed all baselines and human operators across all five tasks. Multi-robot (M)HG-DAgger raised mean success across the five tasks from 36.3% to 81.0%, providing stronger starting policies for further learning. Without additional expert demonstrations or corrective actions, FERT then raised mean success to 97.3%, a gain of 16.3 percentage points and an 86.0% reduction in mean failure rate. On phone charging, success increased from 60.0% to 97.5%, reducing failures by 93.8%. On phone packaging, learning with 96 robots raised success from 79.2% to 98.3%, reducing failures by 92.0%. These results show that FERT can improve the reliability of initialized policies through collective experience under the evaluated tasks and training configurations. We tested each policy on two AgiBot G2 robots that had not collected training data, with 60 trials per robot and no human intervention. The charts below show success rates over these 120 trials per task, with 95% Wilson confidence intervals.
Explore the task-by-task results
Success by method
| Task | Human | SFT | (M)HG-DAgger | RLT | RECAP | FERT |
|---|
Scaling trends emerge in online robot training
To see how performance changes as more robots contribute, we varied fleet size while keeping collection time per robot fixed. Across the five tasks, FERT generally improves with larger fleets. On phone packaging, success reaches 50.8%, 86.7% and 98.3% with 4, 16 and 96 training robots, respectively. The 4-robot result falls below the starting policy’s 79.2%, while expanding from 16 to 96 robots adds 11.7 percentage points. By collecting autonomous experience with solely binary task-outcome labels, FERT provides a practical route to expanding the experience available for policy improvement. Dashed lines mark the SFT and (M)HG-DAgger starting policies.
Explore results at each fleet size
| Method | Successful trials | 95% Wilson interval |
|---|
Ablation study: What makes the learning loop work?
To understand how each part of the learning loop contributes, we ran an ablation study on phone packaging, testing four design choices with 16 training robots and 120 evaluation trials per configuration. The full configuration reaches 86.7% success. Removing the 30 frames of context on either side of each retrieved segment, while keeping retrieval, lowers it to 57.5%. Both choosing relevant examples and keeping the surrounding movements matter for learning the sequence. We also varied how often robots receive updated policies and how much previous sampling priority carries forward. Among the settings we tested, synchronizing every 500 optimizer steps and retaining 88% of previous priorities each gave the highest success rate.
05
FATS: Learning together without waiting for everyone
Collective experience learning turns a robot fleet into a distributed training system. Keeping that system stable is a challenge: robots may go offline, edge nodes may lose their connections, and some robots may pause for resets or recovery. To keep these local interruptions from holding up the whole learning process, we developed the Fault-tolerant Asynchronous Training System (FATS). Asynchronous collection and fault recovery allow available robots to continue contributing experience while recovered workers rejoin the fleet, helping collective learning continue through interruptions.
HotSwarm: a robot hot-plugging system
To keep training data consistent through interruptions, HotSwarm gives each restarted worker a new identity. When a robot’s control process fails, it restarts the process at the same workstation and rebuilds its connections. The other robots keep collecting. The incomplete episode is discarded, and the replacement rejects late actions addressed to its predecessor. If an inference worker fails, it restarts and reloads the current policy. If training workers fail, they resume from a saved checkpoint. During training recovery, surviving cloud inference workers continue serving their current policy, while new experience waits in the channels until training resumes. Operators handle scene resets and hardware repairs.
SwarmLink: connecting robots and training
Cloud inference and fleet learning share the same network link, so bulk data transfers must leave room for robots waiting for actions. SwarmLink pools incoming observations for available cloud inference workers, which batch requests and return actions to the corresponding robots. Meanwhile, execution records and task outcomes stream to a cloud trajectory collector. The collector combines them with observations and policy outputs already available in the cloud, avoiding a second upload of those observations. It releases an episode to training only when all required fields and the outcome are present. Interaction traffic takes priority over bulk transfers, allowing collection and learning to overlap.
System performance: keeping collection running
In a 96-robot run, the fleet collected 9,834 episodes despite 30 fault incidents affecting 22 robots. The recorded faults cost 1.3% of the fleet’s total robot time. Local recovery also changes how robot faults scale: adding robots increases the number of faults, while each recovery still interrupts only the affected workers. Under the same per-robot fault rates and downtime, the model keeps the fraction of time available for collection steady as the fleet grows.
Cost of fault recovery with fleet size
Higher values mean more time for collecting experience. Orange shows HotSwarm’s local recovery; blue shows whole-system restarts lasting 2–10 minutes. The point at 96 robots is measured, while other fleet sizes are modeled under the same per-robot fault rates and downtime.
Collection continues through faults
Episodes completed per minute in 5-min bins. Marks above the curve show fault events, with height proportional to the number of robots affected, and the shaded interval is the lunch break. Fleet-level collection continued through the recorded local fault events. Outside the lunch break, 65 robots were executing at a time on average, and the fleet completed 24 episodes per minute. For the 14 fault events that occurred at least 10 min from the lunch break and from either end of the run, we compare the episodes completed in the 10 min after the event with those in the 10 min before it. After a median event, the fleet completed 99% as many episodes as before it, and never fewer than 88%.
Fault incidents by cause
Fault incidents by initiating cause and duration. Circles mark incidents in which the robot was restarted, and squares mark incidents in which recovery was blocked. We recorded 30 fault incidents on 22 of the 96 robots. By initiating cause, 19 incidents began with a compute or network fault, e.g., host memory exhaustion or a network interruption, 9 with a hardware fault of the chassis, joints, battery or sensors, and 2 with a robot that was not ready, e.g., powered off or awaiting SDK startup. Compute and network faults were the most widespread, affecting 18 robots with a median duration of 15 min. Hardware faults recurred on fewer robots, with 9 incidents on 6 robots, and lasted a median of 8 min but up to 72 min. Five incidents required restarting the robot, which took 3 to 11 min, except for one restart that spanned the lunch break. In five other incidents, recovery was blocked for 5 to 26 min, and four of these ended together at 15:21.
Replaying the measured traces under other designs
Episodes collected when each robot's measured episodes and pauses are replayed under other designs, relative to the run. A full restart stops all robots for 2, 5 or 10 minutes after each fault event, and synchronous rounds make each policy update wait for one episode from every robot. The two synchronous variants replace the pauses that overlap a fault, or all pauses, with the robot's median pause. In the trace replay, synchronous rounds collect only 33% of the episodes of the run. A round lasts a median of 9.3 min, more than twice a robot's typical cycle of 4.1 min, because it ends only when the robot with the longest pause or episode finishes. Faults explain a small part of this loss. Replacing each pause that overlaps a fault with the robot's median pause raises synchronous collection to 42%, and replacing every pause in this way raises it to 78%. The remaining 22% is lost because episode durations and typical pauses differ across robots and from one episode to the next. The full-restart replay preserves 92%, 86% and 74% of the episodes for restart durations of 2, 5 and 10 min, respectively.
Collection varies across robots
Completed episodes per robot, sorted by count. Robots progress at different rates even without faults. The median episode lasted 3.2 min, but per-robot medians ranged from 0.6 to 4.2 min, and robots completed between 45 and 222 episodes each. Resets and faults add pauses of varying length between episodes.
06
Conclusion
We introduce Collective Experience Learning (CEL), a paradigm in which robots improve a shared policy during deployment using fleet-generated experience after expert-supervised initialization. We realize CEL through Failure-guided Experience Retrieval Fine-Tuning (FERT), which uses successful executions as action supervision and failures to select relevant successful experience. A Fault-tolerant Asynchronous Training System (FATS) supports this learning process by coordinating robot execution, cloud inference and policy optimization despite interruptions to individual robots. Together, these components enable the fleet to supply experience for subsequent policy updates without additional expert demonstrations or corrective actions. Across five real-world manipulation tasks, the resulting policies achieved 93.3–99.2% success on robots excluded from training data collection.