Skip to main content

reBot-DM Bimanual Collaboration

reBot-RS bimanual collaboration

1. Environment Setup​

Before using two arms, set up the environment by following the single-arm getting-started tutorials:

1.1 Activate the Existing Environment​

After successfully completing the entire single-arm workflow, proceed to install the bimanual environment.

conda activate lerobot

cd ~
git clone https://github.com/xiehuangbao888/lerobot-robot-seeed-b601.git
cd lerobot-robot-seeed-b601
pip install -e .

1.2 Port Assignments Used in This Guide​

DeviceLeftRight
Arm 102 leader/dev/ttyUSB0/dev/ttyUSB1
RS followercan0can1
DM follower, Damiao serial/dev/ttyACM0/dev/ttyACM1

2. Bimanual Calibration​

Keep the same robot.id and teleop.id during calibration and subsequent operations. Both RS and DM examples use bi_follower, and the two leaders use bi_leader. The id identifies calibration files, not the motor model. If you previously calibrated your DM arms with bi_follower_dm, continue using that ID in subsequent commands for those arms.

2.2 Calibrate Two DM Followers: Damiao Serial​

sudo chmod 666 /dev/ttyACM*
lerobot-calibrate \
--robot.type=bi_seeed_b601_follower \
--robot.id=bi_follower \
--robot.left_arm_config.motor_family=dm \
--robot.left_arm_config.port=/dev/ttyACM0 \
--robot.right_arm_config.motor_family=dm \
--robot.right_arm_config.port=/dev/ttyACM1

2.3 Calibrate Two Leaders​

lerobot-calibrate \
--teleop.type=bi_seeed_102_leader \
--teleop.id=bi_leader \
--teleop.left_arm_config.port=/dev/ttyUSB0 \
--teleop.right_arm_config.port=/dev/ttyUSB1

2.4 Calibration Procedure​

warning
  1. Follow the prompts to calibrate the left and right arms in order.
  2. If calibration already exists, press Enter to reuse it, or type c and press Enter to recalibrate.
  3. When prompted to position the arm manually at zero, move the entire current arm to the specified zero pose, fully close the empty gripper, and confirm.
  4. Follower calibration resets the zero positions of all motors in the entire follower arm, not just the gripper. Do not confirm zeroing at an arbitrary pose.

Exiting calibration does not automatically reset the arms. Calibration files are stored under ~/.cache/huggingface/lerobot/calibration/ by default, with subdirectories for the underlying device types and _left or _right appended to each side’s ID. Recheck calibration after replacing a physical arm or changing motor zero positions.

3. Bimanual Teleoperation​

3.2 Teleoperate Two DM Arms: Damiao Serial​

lerobot-teleoperate \
--robot.type=bi_seeed_b601_follower \
--robot.id=bi_follower \
--robot.left_arm_config.motor_family=dm \
--robot.left_arm_config.port=/dev/ttyACM0 \
--robot.right_arm_config.motor_family=dm \
--robot.right_arm_config.port=/dev/ttyACM1 \
--teleop.type=bi_seeed_102_leader \
--teleop.id=bi_leader \
--teleop.left_arm_config.port=/dev/ttyUSB0 \
--teleop.right_arm_config.port=/dev/ttyUSB1

4. Data Collection​

Verify that teleoperation works before running the recording commands. The examples record two 20-second episodes to check the data pipeline. Replace dataset.single_task with the task you actually demonstrate.

4.2 Two DM Arms: Damiao Serial and Three-Camera Recording​

lerobot-record \
--robot.type=bi_seeed_b601_follower \
--robot.id=bi_follower \
--robot.left_arm_config.motor_family=dm \
--robot.left_arm_config.port=/dev/ttyACM0 \
--robot.right_arm_config.motor_family=dm \
--robot.right_arm_config.port=/dev/ttyACM1 \
--robot.cameras='{
top: {
type: opencv,
index_or_path: "/dev/video0",
width: 640,
height: 480,
fps: 30,
fourcc: "MJPG"
},
left_wrist: {
type: opencv,
index_or_path: "/dev/video2",
width: 640,
height: 480,
fps: 30,
fourcc: "MJPG"
},
right_wrist: {
type: opencv,
index_or_path: "/dev/video4",
width: 640,
height: 480,
fps: 30,
fourcc: "MJPG"
}
}' \
--teleop.type=bi_seeed_102_leader \
--teleop.id=bi_leader \
--teleop.left_arm_config.port=/dev/ttyUSB0 \
--teleop.right_arm_config.port=/dev/ttyUSB1 \
--dataset.repo_id=seeed_rebot/test \
--dataset.single_task="pick the cube" \
--dataset.fps=30 \
--dataset.num_episodes=2 \
--dataset.episode_time_s=20 \
--dataset.reset_time_s=10 \
--dataset.push_to_hub=false \
--display_data=false

5. Training and Evaluation​

note

This section follows the DM LeRobot guide, retaining its training methods and adapting evaluation, optional teleoperation, and the asynchronous inference client to the DM bimanual configuration above. Use a model trained on your bimanual dataset and keep calibration IDs, camera names, resolution, and frame rate consistent with recording. Replace dataset names, task descriptions, and model paths with your actual values. Training examples retain the source guide’s wandb options; for the bimanual environment above, use trackio.enable as supported by your installed version rather than mixing unsupported logging options.

Refer to the official tutorial ACT

Training

To train a policy to control your robot, use the python -m lerobot.scripts.train script. Some parameters are required. Here is an example command:

lerobot-train \
--dataset.repo_id=${HF_USER}/rebot_test \
--policy.type=act \
--output_dir=outputs/train/act_rebot_test \
--job_name=act_rebot_test \
--policy.device=cuda \
--wandb.enable=false \
--steps=300000

If you want to train on a local dataset, make sure the repo_id matches the name used during data collection and add --policy.push_to_hub=false.

lerobot-train \
--dataset.repo_id=seeed_rebot_b601_dm/test \
--policy.type=act \
--output_dir=outputs/train/act_rebot_test \
--job_name=act_rebot_test \
--policy.device=cuda \
--wandb.enable=false \
--policy.push_to_hub=false \
--steps=300000
tip

If you are using an RTX 50 series GPU, you need to add --dataset.video_backend=pyav to bypass missing APIs in the preview version of torchvision. The training command becomes:

lerobot-train \
--dataset.repo_id=seeed_rebot_b601_dm/test \
--dataset.video_backend=pyav \
--policy.type=act \
--output_dir=outputs/train/act_rebot_test \
--policy.device=cuda \
--wandb.enable=false \
--policy.push_to_hub=false \
--steps=300000

Command Explanation

  • Dataset specification: We provide the dataset via the parameter --dataset.repo_id=${HF_USER}/rebot_test.
  • Training steps: We modify the number of training steps using --steps=300000. The algorithm defaults to 800000 steps; adjust based on your task difficulty. You can set it higher if unsure, since checkpoints are generated during training and evaluation can resume from any checkpoint.
  • Policy type: We provide the policy with policy.type=act. Similarly, you can switch between policies such as [act, diffusion, pi0, pi0fast, sac, smolvla]. This will load the configuration from configuration_act.py. Importantly, this policy will automatically adapt to your robot's motor states, motor actions, and the number of cameras, as this information is already stored in your dataset.
  • Device selection: We provide policy.device=cuda because we are training on an Nvidia GPU, but you can use policy.device=mps for training on Apple Silicon.
  • Visualization tool: We provide wandb.enable=true to visualize training charts using Weights and Biases. This is optional, but if you use it, ensure you have logged in by running wandb login.

Evaluation

You can use the record function from lerobot/record.py but with a policy checkpoint as input. For instance, run this command to record 10 evaluation episodes:

lerobot-record \
--robot.type=bi_seeed_b601_follower \
--robot.left_arm_config.motor_family=dm \
--robot.left_arm_config.port=/dev/ttyACM0 \
--robot.right_arm_config.motor_family=dm \
--robot.right_arm_config.port=/dev/ttyACM1 \
--robot.cameras='{
top: {
type: opencv,
index_or_path: "/dev/video0",
width: 640,
height: 480,
fps: 30,
fourcc: "MJPG"
},
left_wrist: {
type: opencv,
index_or_path: "/dev/video2",
width: 640,
height: 480,
fps: 30,
fourcc: "MJPG"
},
right_wrist: {
type: opencv,
index_or_path: "/dev/video4",
width: 640,
height: 480,
fps: 30,
fourcc: "MJPG"
}
}' \
--robot.id=bi_follower \
--display_data=false \
--dataset.repo_id=seeed/eval_test123 \
--dataset.single_task="Put lego brick into the transparent box" \
--policy.path=outputs/train/act_rebot_test/checkpoints/last/pretrained_model
  1. The --policy.path parameter indicates the path to the weight file of your policy training results (e.g., outputs/train/act_rebot_test/checkpoints/last/pretrained_model). If you upload the model training result weight file to Hub, you can also use the model repository (e.g., ${HF_USER}/act_rebot_test).
  2. The dataset name dataset.repo_id starts with eval_. This operation will separately record videos and data during evaluation, which will be saved in the folder starting with eval_, such as seeed/eval_test123.
  3. If you encounter File exists: 'home/xxxx/.cache/huggingface/lerobot/xxxxx/seeed/eval_xxxx' during the evaluation phase, please delete the folder starting with eval_ first and then run the program again.
  4. When encountering mean is infinity. You should either initialize with stats as an argument or use a pretrained model, please note that keywords like front and side in the --robot.cameras parameter must be strictly consistent with those used when collecting the dataset.

To resume training from a checkpoint, here is an example command to resume from the last checkpoint of the act_rebot_test policy:

lerobot-train \
--config_path=outputs/train/act_rebot_test/checkpoints/last/pretrained_model/train_config.json \
--resume=true
Loading Comments...