Skip to main content
SO-ARM × LeRobot × GR00T × Jetson · Advanced Course

SO-ARM101 Sim-to-Real VLA

The complete Seeed × NVIDIA Sim-to-Real curriculum, migrated to the SO-ARM101: build the real robot lab, teleoperate it to collect your own data, fine-tune Isaac GR00T N1.7, deploy on the physical follower and on Jetson — with optional Isaac Sim episodes and Cosmos augmentation.

Course Overview​

Overview

From an empty bench to a Jetson-deployed policy

This course mirrors the reBot DLI curriculum article for article and adapts it to the SO-ARM101 — the same embodiment NVIDIA's SO-101 workshop uses:

  1. Overview — the workflow, LeRobot, Sim-to-Real, and the SO-ARM101 hardware (Chapters 1–5);
  2. Build Your Robot Lab — install the environment, write servo IDs (kit only — pre-built arms skip this), calibrate both arms, and build the studio: lightbox, brackets, prop positions, cameras (Chapters 6–8);
  3. Get Your Own VLA Model — collect real data, fine-tune Isaac GR00T N1.7, evaluate in the real environment, and optionally scale up with Isaac Sim episodes and Cosmos augmentation (Chapters 9–13);
  4. Deploy on Jetson — flash the edge device, export target-specific TensorRT engines, and serve the policy to the follower (Chapters 14–18);
  5. Reference — every repository and document the course uses (Chapter 19).
SO-ARM101 kit

Course Structure​

DLI modules

Overview → Build Your Robot Lab → Get Your Own VLA Model → Jetson

ModuleChaptersFocus
1 · Overview1–5Course overview, LeRobot, Sim-to-Real, SO-ARM101, summary
2 · Build Your Robot Lab6–8Module intro, prepare the arms (IDs + calibration), workspace
3 · Get Your Own VLA Model9–11Collect real data, train GR00T N1.7, evaluate in the real environment
3+ · Optional12–13Collect data from Isaac Sim, dataset augmentation with Cosmos
4 · Deploy on Jetson14–18Jetson setup, hardware, GR00T env, assets, TensorRT inference
5 · Reference19Repositories, products, documentation

The real-robot path is the main line of the course. Isaac Sim appears only as an optional accelerator, the way it is used in the Seeed × NVIDIA Sim-to-Real article.

Prerequisites​

Before you start

What you need

RequirementDetails
Workstation GPUNVIDIA RTX with ≥ 8 GB VRAM for Chapters 9–12; a 24 GB+ GPU (RTX 5090 / RTX 6000 class) for GR00T fine-tuning
OSUbuntu 22.04 or later
SoftwareIsaac Sim 6.1 (Chapter 12 only), LeRobot environment (Chapter 7)
RobotSO-ARM101 leader and follower — kit or pre-built; only the follower is strictly required
CamerasTwo USB cameras (wrist + external)
Edge device (Stage 4)Jetson AGX Orin or AGX Thor with JetPack 7.2
Disk~100 GB free across machines

If you plan to let an AI assistant install the environment for you, copy the setup skill file from the project repository — Chapter 6 links to it.

What You Will Build​

Outcome

Deliverables after the nineteen chapters

  • A calibrated SO-ARM101 leader/follower pair — assembled or factory-pre-built, servo IDs written where needed;
  • A recording studio: lightbox, rigid camera brackets, and a taped-down vial rack with marked prop positions;
  • A real-world LeRobot Dataset v3 with dual-camera video streams;
  • A fine-tuned Isaac GR00T N1.7 policy trained on your data, with optional sim-enhanced and Cosmos-augmented co-training sets;
  • A closed-loop policy running on the physical follower via NVIDIA's real-robot evaluation stack;
  • A target-specific TensorRT engine bundle and policy server on Jetson AGX Orin or Thor.

Course Chapters​

Structure

Nineteen chapters across five modules

#ChapterModule
1Course Overview1 · Overview
2LeRobot Background and the Community1 · Overview
3What is Sim-to-Real?1 · Overview
4Introduction of the SO-ARM1011 · Overview
5Summary1 · Overview
6Build Your Robot Lab (Intro)2 · Build Your Robot Lab
7Prepare the Robot Arm2 · Build Your Robot Lab
8Building the Workspace2 · Build Your Robot Lab
9Collect Data from the Real Environment3 · Get Your Own VLA Model
10Train the GR00T N1.7 Policy3 · Get Your Own VLA Model
11Evaluate in the Real Environment3 · Get Your Own VLA Model
12Collect Data from Isaac Sim (Optional)3 · Optional
13Datasets Augment (Optional)3 · Optional
14Setup Your Jetson Edge Device4 · Deploy on Jetson
15Connect the Robot Hardware4 · Deploy on Jetson
16Configure the GR00T Software Environment4 · Deploy on Jetson
17Prepare the Dataset and Model Assets4 · Deploy on Jetson
18Run GR00T N1.7 TensorRT Inference4 · Deploy on Jetson
19Reference5 · Reference

Start with Chapter 1, or jump straight to Chapter 7 if the software environment is already installed. If you only need the optional simulation part, go directly to Chapter 12.

Loading Comments...