9. Foundations of Robot Learning and Imitation Learning
Chapter 9 of the Seeed Embodied Intelligence Beginner's Course — why the arm needs learning, rule-based vs learning-based control, observation/state/action, action chunks, data distribution, and the three phases of training, inference, and evaluation.
9.1 Why Do Robot Arms Need Learning?
9.1 Why Do Robot Arms Need Learning?
In Stage 2, you can already control the robot arm with the Python SDK: read joint angles, send target positions, open and close the gripper. So it's natural to think — just write a program that makes the arm automatically pick up blocks on the table, right?
So you write such a program. During the first demo in the lab, the program worked great. But the next day, things changed — and you'll find that to handle these changes, more and more "if" statements get added to the program, eventually becoming a giant catalog of special cases that no one can maintain.
This is the fundamental dilemma of traditional programmatic control: the real world changes continuously, while if-else is discrete. You cannot enumerate every possible state of the world.
Enabling robot arms to acquire skills this way is what Robot Learning sets out to solve. And the imitation learning covered in this stage is the most mature and easiest-to-deploy-on-real-hardware approach.
9.2 Rule-Based Control vs. Learning-Based Control
9.2 Rule-Based Control vs. Learning-Based Control
Before continuing, let's put the two control approaches side by side and see them clearly.
Stages 3 and 4 of this course follow the "learning-based control" path. But remember: even in the future, your learning systems will still retain a large number of rules — such as joint limits and safe speed constraints. Learning and rules are complementary.
9.3 Imitation Learning and Behavioral Cloning
9.3 Imitation Learning and Behavioral Cloning
What Is Imitation Learning
What Is Behavioral Cloning
9.4 Observation, State, and Action
9.4 Observation, State, and Action
All data in imitation learning can be categorized into three concepts. These three terms will appear repeatedly in every chapter that follows, so build accurate intuition here.
- Observation: the world the robot "sees."
- State: the robot's "own" condition.
- Action: what the robot "will do."
9.5 Single-Step Actions vs. Action Chunks
9.5 Single-Step Actions vs. Action Chunks
- Single-step action: one decision per frame.
- Action chunk: predict a sequence of actions at once.
Of course, longer action chunks aren't always better. Predicting too far ahead means the environment may change mid-execution (e.g., an object gets bumped), while the arm is still executing "outdated" actions. Real systems use a compromise: execute open-loop for a short segment, then re-observe and re-predict.
9.6 Data Distribution and Model Generalization
9.6 Data Distribution and Model Generalization
This is the most important section of this chapter, and the one beginners most easily overlook, yet it determines success or failure in practice. Remember this one sentence first:
An imitation learning model can only learn what's in the data, and can only work within the range covered by the data.
- All (observation, action) pairs the model saw during training constitute a Data Distribution. During inference, if the images and states the arm encounters fall within this distribution, the model generally performs well; once they go outside the distribution, the model's output loses its basis and behavior becomes unpredictable.
This leads to several very practical implications:
- If you want it to grasp blocks at any position on the table, the data must cover all positions on the table — if you only collect data at the center of the table, the model will only grasp at the center.
- If you want it to grasp objects of different colors, the data must contain different colors — otherwise, a new color is an unfamiliar world to it.
- Lighting, background, and camera position should all stay as consistent as possible with data collection — a model trained on afternoon-collected data may completely fail under evening indoor lighting.
And generalization is the model's ability to apply patterns learned from data to new situations within the distribution that it has not seen individually. For example, if training saw blocks at 100 different positions, and during inference a block appears at the 101st position (still on the tabletop), the model can still grasp it — that's generalization. Generalization isn't magic; it comes from the data's diversity: the richer and more continuous the data coverage, the smaller the "gaps" within the distribution, and the better the generalization.
The upper limit of imitation learning is essentially determined at the moment data is collected. Training merely realizes that upper limit.
9.7 Training, Inference, and Evaluation
9.7 Training, Inference, and Evaluation
Training: Offline Learning
Training happens after data collection is complete; it's an offline process: the arm can be powered off and set aside; all work is done on the GPU.
- Input: collected dataset (time series of images, State, and Action);
- Process: the model repeatedly reads the data, continuously adjusting its internal parameters so that its predicted Actions increasingly match the human demonstrations;
- Output: a trained model file.
Training quality is primarily observed through Loss: as Loss decreases, the model's predicted actions increasingly resemble human demonstrations.
Inference: Online Decision-Making
Inference is the process where the model is deployed on the real robot and works in real time: reading camera and joint states → the model predicts an Action Chunk → sending it to the motors for execution. Inference has real-time requirements — the model must output actions within tens of milliseconds, otherwise the arm will stutter.
Evaluation: Judged by Success Rate
The model is trained, Loss is low — can it actually do the job? Not necessarily.
Low Loss only means the model is "human-like," not that it "can complete the task." The only reliable evaluation method is real-robot testing:
- Set clear task success criteria (e.g., "the block ends up in the box");
- Vary initial conditions (block position, lighting), repeat the test N times;
- Calculate the task success rate — e.g., 14 successes out of 20 tests = 70% success rate.
Failure cases found during evaluation are not endpoints; they are input for the next round of data collection — wherever failures occur, collect additional data there, then retrain. This is the data iteration loop, and it's the daily routine of real robot learning projects.
9.8 Advantages and Limitations of Imitation Learning
9.8 Advantages and Limitations of Imitation Learning
