Ajustar GR00T N1.5 para el Brazo LeRobot SO-101 e Implementar en Jetson AGX Thor
Introducción
Este wiki explica cómo ajustar NVIDIA Isaac GR00T N1.5 para el brazo LeRobot SO-101 e implementarlo en NVIDIA Jetson Thor. Cubre:
- Preparación de hardware para LeRobot SO-101 y Jetson AGX Thor
- Configuración del entorno de software para GR00T N1.5 en Jetson Thor
- Uso de la plataforma de entrenamiento LeRobot: recolección de datos, formateo de conjuntos de datos y ajuste fino para el brazo SO-101
- Flujos de trabajo de ejemplo para implementar la política GR00T N1.5 entrenada (LeRobot + SO-101) en Jetson Thor
- Consejos de solución de problemas y errores comunes
Se proporcionan instrucciones detalladas paso a paso y documentación de referencia para ayudarte a ir desde la configuración inicial hasta la implementación completa.
Comenzando con el Kit de Desarrollo NVIDIA Jetson Thor
Descripción General de Jetson Thor
El Kit de Desarrollo NVIDIA® Jetson AGX Thor™ es una potencia para IA física y robótica humanoide. Está construido alrededor de la GPU NVIDIA Blackwell e incluye 128 GB de memoria de alta velocidad, entregando hasta 2,070 FP4 TFLOPS de cómputo de IA mientras opera dentro de un rango de potencia de 40-130 W (uso común hasta 130 W).
Puedes obtener el Kit de Desarrollo Jetson AGX Thor de Seeed Studio aquí: Seeed – Kit de Desarrollo NVIDIA Jetson AGX Thor™
Los elementos de hardware incluidos en la caja son una unidad Thor y un adaptador de corriente:
Flashear la Imagen del Sistema Thor
A partir del 10 de septiembre de 2025, la última versión disponible de imagen del sistema para Thor es 38.2. Si necesitas volver a flashear el sistema para Thor, por favor sigue las instrucciones en esta sección. Elementos Requeridos para Flashear:
- Un dispositivo host con más de 25 GB de espacio disponible en disco (compatible con OS Ubuntu o Windows)
- Una unidad USB con capacidad de al menos 16 GB
- Un monitor y cable de pantalla DP/HDMI
- Un entorno de suministro de energía capaz de entregar más de 240 W
- Un teclado con interfaz USB
Primero, descarga la imagen del sistema en formato ISO para Thor desde el sitio web oficial de NVIDIA. Haz clic aquí para visitar la página de descarga:
Instala Balena Etcher en la máquina host. Para descargar el instalador, haz clic aquí y selecciona la versión apropiada basada en tu sistema operativo host:
Después de descargar el archivo de imagen ISO e instalar exitosamente Balena Etcher, inserta la unidad USB en la máquina host. Luego, ejecuta Balena Etcher para crear una unidad USB de arranque para flashear Thor:
Este proceso formateará la unidad USB. Por favor asegúrate de hacer una copia de seguridad de cualquier dato importante de antemano.
Selecciona el archivo de imagen ISO descargado desde tu almacenamiento local, luego elige el dispositivo objetivo—es decir, tu unidad USB.
¡Asegúrate de verificar cuidadosamente el nombre del dispositivo objetivo y el directorio de montaje! Haz clic en Flash! y espera a que el proceso se complete. Una vez terminado, la unidad USB para flashear el sistema en Thor está lista:
A continuación, inserta la unidad USB preparada, teclado, cable de pantalla (DP/HDMI) y fuente de alimentación (Type-C) en la placa Thor para comenzar el proceso de flasheo.
Enciende el Thor y entra a la interfaz de arranque. Selecciona Boot Manager, luego elige la unidad USB que fue insertada en Thor (basado en el nombre de tu unidad USB). Presiona Esc para regresar al menú anterior, y selecciona Continue:
Después de una breve pantalla negra, aparecerá la siguiente interfaz. Selecciona Jetson Thor options y presiona Enter. Luego, elige la opción
Flash Jetson AGX Thor Developer Kit on NVMe 0.2.0-r38.2 para flashear la imagen del sistema en la unidad de estado sólido NVMe de Thor:
Se mostrará una gran cantidad de información de registro en la pantalla. Espera aproximadamente 15 minutos. Una vez que esta etapa esté completa, el dispositivo se reiniciará automáticamente y procederá a la siguiente interfaz. Espera hasta que la barra de Update Progress alcance el 100%, lo que indica que el proceso de flasheo se ha completado exitosamente:
Después del flasheo, puedes proceder con la configuración inicial del sistema:
Esta imagen del sistema no incluye CUDA, TensorRT u otros componentes SDK de JetPack.
Configuración Básica del Entorno de Desarrollo en Thor
Esta sección proporciona ejemplos de cómo instalar dependencias de software comúnmente utilizadas en Thor para propósitos de desarrollo. Estas dependencias tienen como objetivo facilitar el desarrollo posterior.
Ten en cuenta que las dependencias listadas son solo para referencia—por favor instala paquetes adicionales según los requisitos de sus proyectos individuales.
Instalación de Dependencias de Desarrollo Esenciales
Instalación del SDK JetPack
Abre la terminal y ejecuta el siguiente comando. Después de una breve espera, CUDA, TensorRT y otros componentes SDK serán instalados:
sudo apt update
sudo apt install nvidia-jetpack
Instalación del Navegador
Firefox ha sido probado y verificado para ejecutarse de manera estable en Ubuntu 24.04:
sudo apt update
sudo apt install firefox
Instalación de Jtop
Para instalar jtop, consulta las siguientes instrucciones.
sudo apt update
sudo apt install python3
sudo apt install python3-pip
sudo pip3 install -U pip
sudo pip3 install jetson-stats
Instalación de Miniconda
Miniconda se utiliza para aislar entornos de desarrollo. Para instalar miniconda, consulta las siguientes instrucciones：
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
source ~/.bashrc
conda --version
Instalación de la Versión GPU de PyTorch
Compilar la versión GPU de PyTorch desde el código fuente en Thor puede resultar en problemas de compatibilidad. Para conveniencia, proporcionamos un archivo .whl precompilado para ayudar a los desarrolladores a configurar rápidamente un entorno de desarrollo habilitado para PyTorch en Thor.
Aquí se proporciona un archivo wheel precompilado para instalar PyTorch 2.9 en Thor. Este archivo fue compilado en un entorno Python 3.10 + CUDA 13.
Haz clic aquí para descargar el archivo .whl.
Otros archivos .whl de dependencias precompiladas para thor se pueden encontrar aquí.
Si el archivo wheel esperado no está disponible, el desarrollador necesitará construir las dependencias requeridas por sí mismo para completar la configuración del entorno de desarrollo.
Instalación de Dependencias Adicionales
Este documento proporciona una imagen Docker de referencia para ayudar a los desarrolladores a adaptarse rápidamente al entorno de desarrollo Jetson AGX Thor. Esta imagen es solo para referencia, y los desarrolladores son libres de elegir si usarla basándose en sus necesidades específicas.
Dado que el tamaño de la imagen excede los 40 GB, descargarla desde el servidor es relativamente lento.
Esta imagen Docker puede ejecutar de manera estable la inferencia GR00T N1.5. La salida del comando pip list en la imagen es la siguiente:
pip.list
Package Version Location Installer Editable project location
------------------------- ----------------- ---------------- -------------------------------------- -------------------------
absl-py 2.3.1 /opt/venv/lib/python3.12/site-packages pip
accelerate 1.2.1 /opt/venv/lib/python3.12/site-packages pip
aiohappyeyeballs 2.6.1 /opt/venv/lib/python3.12/site-packages pip
aiohttp 3.12.15 /opt/venv/lib/python3.12/site-packages pip
aiosignal 1.4.0 /opt/venv/lib/python3.12/site-packages pip
albucore 0.0.17 /opt/venv/lib/python3.12/site-packages pip
albumentations 1.4.18 /opt/venv/lib/python3.12/site-packages pip
annotated-types 0.7.0 /opt/venv/lib/python3.12/site-packages pip
antlr4-python3-runtime 4.9.3 /opt/venv/lib/python3.12/site-packages pip
anyio 4.10.0 /opt/venv/lib/python3.12/site-packages pip
argon2-cffi 25.1.0 /opt/venv/lib/python3.12/site-packages pip
argon2-cffi-bindings 25.1.0 /opt/venv/lib/python3.12/site-packages pip
arrow 1.3.0 /opt/venv/lib/python3.12/site-packages pip
asttokens 3.0.0 /opt/venv/lib/python3.12/site-packages pip
astunparse 1.6.3 /opt/venv/lib/python3.12/site-packages pip
async-lru 2.0.5 /opt/venv/lib/python3.12/site-packages pip
attrs 23.2.0 /usr/lib/python3/dist-packages debian
av 12.3.0 /opt/venv/lib/python3.12/site-packages pip
babel 2.17.0 /opt/venv/lib/python3.12/site-packages pip
beautifulsoup4 4.13.5 /opt/venv/lib/python3.12/site-packages pip
bleach 6.2.0 /opt/venv/lib/python3.12/site-packages pip
blessings 1.7 /opt/venv/lib/python3.12/site-packages pip
blinker 1.9.0 /opt/venv/lib/python3.12/site-packages pip
certifi 2025.8.3 /opt/venv/lib/python3.12/site-packages pip
cffi 2.0.0 /opt/venv/lib/python3.12/site-packages pip
charset-normalizer 3.4.3 /opt/venv/lib/python3.12/site-packages pip
click 8.2.1 /opt/venv/lib/python3.12/site-packages pip
cloudpickle 3.1.1 /opt/venv/lib/python3.12/site-packages pip
cmake 3.31.6 /opt/venv/lib/python3.12/site-packages pip
comm 0.2.3 /opt/venv/lib/python3.12/site-packages pip
contourpy 1.3.3 /opt/venv/lib/python3.12/site-packages pip
cramjam 2.11.0 /opt/venv/lib/python3.12/site-packages pip
cryptography 41.0.7 /usr/lib/python3/dist-packages debian
cuda-bindings 13.0.0 /opt/venv/lib/python3.12/site-packages pip
cuda-core 0.3.2 /opt/venv/lib/python3.12/site-packages pip
cuda-pathfinder 1.2.2 /opt/venv/lib/python3.12/site-packages pip
cycler 0.12.1 /opt/venv/lib/python3.12/site-packages pip
Cython 3.1.3 /opt/venv/lib/python3.12/site-packages pip
datasets 4.0.0 /opt/venv/lib/python3.12/site-packages pip
dbus-python 1.3.2 /usr/lib/python3/dist-packages
debugpy 1.8.16 /opt/venv/lib/python3.12/site-packages pip
decorator 5.2.1 /opt/venv/lib/python3.12/site-packages pip
decord 0.6.0 /opt/venv/lib/python3.12/site-packages pip
deepdiff 8.6.1 /opt/venv/lib/python3.12/site-packages pip
defusedxml 0.7.1 /opt/venv/lib/python3.12/site-packages pip
devscripts 2.23.7 /usr/lib/python3/dist-packages
diffusers 0.36.0.dev0 /opt/venv/lib/python3.12/site-packages pip
dill 0.3.8 /opt/venv/lib/python3.12/site-packages pip
distro 1.9.0 /usr/lib/python3/dist-packages debian
dm-tree 0.1.8 /opt/venv/lib/python3.12/site-packages pip
docker-pycreds 0.4.0 /opt/venv/lib/python3.12/site-packages pip
docstring_parser 0.17.0 /opt/venv/lib/python3.12/site-packages pip
docutils 0.22 /opt/venv/lib/python3.12/site-packages pip
draccus 0.11.5 /opt/venv/lib/python3.12/site-packages pip
einops 0.8.1 /opt/venv/lib/python3.12/site-packages pip
eval_type_backport 0.2.2 /opt/venv/lib/python3.12/site-packages pip
evdev 1.9.2 /opt/venv/lib/python3.12/site-packages pip
executing 2.2.1 /opt/venv/lib/python3.12/site-packages pip
Farama-Notifications 0.0.4 /opt/venv/lib/python3.12/site-packages pip
fastjsonschema 2.21.2 /opt/venv/lib/python3.12/site-packages pip
fastparquet 2024.11.0 /opt/venv/lib/python3.12/site-packages pip
feetech-servo-sdk 1.0.0 /opt/venv/lib/python3.12/site-packages pip
filelock 3.19.1 /opt/venv/lib/python3.12/site-packages pip
flash_attn 2.8.4 /opt/venv/lib/python3.12/site-packages pip
Flask 3.1.2 /opt/venv/lib/python3.12/site-packages pip
flatbuffers 25.2.10 /opt/venv/lib/python3.12/site-packages pip
fonttools 4.59.2 /opt/venv/lib/python3.12/site-packages pip
fqdn 1.5.1 /opt/venv/lib/python3.12/site-packages pip
frozenlist 1.7.0 /opt/venv/lib/python3.12/site-packages pip
fsspec 2025.3.0 /opt/venv/lib/python3.12/site-packages pip
gast 0.6.0 /opt/venv/lib/python3.12/site-packages pip
gitdb 4.0.12 /opt/venv/lib/python3.12/site-packages pip
GitPython 3.1.45 /opt/venv/lib/python3.12/site-packages pip
google-pasta 0.2.0 /opt/venv/lib/python3.12/site-packages pip
grpcio 1.74.0 /opt/venv/lib/python3.12/site-packages pip
gymnasium 1.0.0 /opt/venv/lib/python3.12/site-packages pip
h11 0.16.0 /opt/venv/lib/python3.12/site-packages pip
h5py 3.12.1 /opt/venv/lib/python3.12/site-packages pip
hf-xet 1.1.9 /opt/venv/lib/python3.12/site-packages pip
httpcore 1.0.9 /opt/venv/lib/python3.12/site-packages pip
httplib2 0.20.4 /usr/lib/python3/dist-packages
httpx 0.28.1 /opt/venv/lib/python3.12/site-packages pip
huggingface-hub 0.34.4 /opt/venv/lib/python3.12/site-packages pip
hydra-core 1.3.2 /opt/venv/lib/python3.12/site-packages pip
id 1.5.0 /opt/venv/lib/python3.12/site-packages pip
idna 3.10 /opt/venv/lib/python3.12/site-packages pip
imageio 2.34.2 /opt/venv/lib/python3.12/site-packages pip
importlib_metadata 8.7.0 /opt/venv/lib/python3.12/site-packages pip
iniconfig 2.1.0 /opt/venv/lib/python3.12/site-packages pip
inquirerpy 0.3.4 /opt/venv/lib/python3.12/site-packages pip
iopath 0.1.9 /opt/venv/lib/python3.12/site-packages pip
ipykernel 6.30.1 /opt/venv/lib/python3.12/site-packages pip
ipython 9.5.0 /opt/venv/lib/python3.12/site-packages pip
ipython_pygments_lexers 1.1.1 /opt/venv/lib/python3.12/site-packages pip
ipywidgets 8.1.7 /opt/venv/lib/python3.12/site-packages pip
isoduration 20.11.0 /opt/venv/lib/python3.12/site-packages pip
itsdangerous 2.2.0 /opt/venv/lib/python3.12/site-packages pip
jaraco.classes 3.4.0 /opt/venv/lib/python3.12/site-packages pip
jaraco.context 6.0.1 /opt/venv/lib/python3.12/site-packages pip
jaraco.functools 4.3.0 /opt/venv/lib/python3.12/site-packages pip
jedi 0.19.2 /opt/venv/lib/python3.12/site-packages pip
jeepney 0.9.0 /opt/venv/lib/python3.12/site-packages pip
Jinja2 3.1.6 /opt/venv/lib/python3.12/site-packages pip
json5 0.12.1 /opt/venv/lib/python3.12/site-packages pip
jsonlines 4.0.0 /opt/venv/lib/python3.12/site-packages pip
jsonpointer 3.0.0 /opt/venv/lib/python3.12/site-packages pip
jsonschema 4.25.1 /opt/venv/lib/python3.12/site-packages pip
jsonschema-specifications 2025.9.1 /opt/venv/lib/python3.12/site-packages pip
jupyter 1.1.1 /opt/venv/lib/python3.12/site-packages pip
jupyter_client 8.6.3 /opt/venv/lib/python3.12/site-packages pip
jupyter-console 6.6.3 /opt/venv/lib/python3.12/site-packages pip
jupyter_core 5.8.1 /opt/venv/lib/python3.12/site-packages pip
jupyter-events 0.12.0 /opt/venv/lib/python3.12/site-packages pip
jupyter-lsp 2.3.0 /opt/venv/lib/python3.12/site-packages pip
jupyter_server 2.17.0 /opt/venv/lib/python3.12/site-packages pip
jupyter_server_terminals 0.5.3 /opt/venv/lib/python3.12/site-packages pip
jupyterlab 4.4.7 /opt/venv/lib/python3.12/site-packages pip
jupyterlab_myst 2.4.2 /opt/venv/lib/python3.12/site-packages pip
jupyterlab_pygments 0.3.0 /opt/venv/lib/python3.12/site-packages pip
jupyterlab_server 2.27.3 /opt/venv/lib/python3.12/site-packages pip
jupyterlab_widgets 3.0.15 /opt/venv/lib/python3.12/site-packages pip
keras 3.11.3 /opt/venv/lib/python3.12/site-packages pip
keyring 25.6.0 /opt/venv/lib/python3.12/site-packages pip
kiwisolver 1.4.9 /opt/venv/lib/python3.12/site-packages pip
kornia 0.7.4 /opt/venv/lib/python3.12/site-packages pip
kornia_rs 0.1.9 /opt/venv/lib/python3.12/site-packages pip
lark 1.2.2 /opt/venv/lib/python3.12/site-packages pip
launchpadlib 1.11.0 /usr/lib/python3/dist-packages
lazr.restfulclient 0.14.6 /usr/lib/python3/dist-packages
lazr.uri 1.0.6 /usr/lib/python3/dist-packages
lazy_loader 0.4 /opt/venv/lib/python3.12/site-packages pip
lerobot 0.3.4 /opt/lerobot /opt/venv/lib/python3.12/site-packages pip
libclang 18.1.1 /opt/venv/lib/python3.12/site-packages pip
llvmlite 0.44.0 /opt/venv/lib/python3.12/site-packages pip
lockfile 0.12.2 /opt/venv/lib/python3.12/site-packages pip
Mako 1.3.10 /opt/venv/lib/python3.12/site-packages pip
Markdown 3.9 /opt/venv/lib/python3.12/site-packages pip
markdown-it-py 4.0.0 /opt/venv/lib/python3.12/site-packages pip
MarkupSafe 3.0.2 /opt/venv/lib/python3.12/site-packages pip
matplotlib 3.10.0 /opt/venv/lib/python3.12/site-packages pip
matplotlib-inline 0.1.7 /opt/venv/lib/python3.12/site-packages pip
mdurl 0.1.2 /opt/venv/lib/python3.12/site-packages pip
mergedeep 1.3.4 /opt/venv/lib/python3.12/site-packages pip
mistune 3.1.4 /opt/venv/lib/python3.12/site-packages pip
ml_dtypes 0.5.3 /opt/venv/lib/python3.12/site-packages pip
more-itertools 10.8.0 /opt/venv/lib/python3.12/site-packages pip
mpmath 1.3.0 /opt/venv/lib/python3.12/site-packages pip
msgpack 1.1.1 /opt/venv/lib/python3.12/site-packages pip
multidict 6.6.4 /opt/venv/lib/python3.12/site-packages pip
multiprocess 0.70.16 /opt/venv/lib/python3.12/site-packages pip
mypy_extensions 1.1.0 /opt/venv/lib/python3.12/site-packages pip
namex 0.1.0 /opt/venv/lib/python3.12/site-packages pip
nbclient 0.10.2 /opt/venv/lib/python3.12/site-packages pip
nbconvert 7.16.6 /opt/venv/lib/python3.12/site-packages pip
nbformat 5.10.4 /opt/venv/lib/python3.12/site-packages pip
nest-asyncio 1.6.0 /opt/venv/lib/python3.12/site-packages pip
networkx 3.5 /opt/venv/lib/python3.12/site-packages pip
nh3 0.3.0 /opt/venv/lib/python3.12/site-packages pip
ninja 1.13.0 /opt/venv/lib/python3.12/site-packages pip
notebook 7.4.5 /opt/venv/lib/python3.12/site-packages pip
notebook_shim 0.2.4 /opt/venv/lib/python3.12/site-packages pip
numba 0.61.2 /opt/venv/lib/python3.12/site-packages pip
numpy 2.3.3 /opt/venv/lib/python3.12/site-packages pip
numpydantic 1.6.7 /opt/venv/lib/python3.12/site-packages pip
nvtx 0.2.13 /opt/venv/lib/python3.12/site-packages pip
oauthlib 3.2.2 /usr/lib/python3/dist-packages
omegaconf 2.3.0 /opt/venv/lib/python3.12/site-packages pip
onnx 1.17.0 /opt/venv/lib/python3.12/site-packages pip
opencv-contrib-python 4.13.0 /opt/venv/lib/python3.12/site-packages pip
opencv-python 4.11.0.86 /opt/venv/lib/python3.12/site-packages pip
opencv-python-headless 4.11.0.86 /opt/venv/lib/python3.12/site-packages pip
opt_einsum 3.4.0 /opt/venv/lib/python3.12/site-packages pip
optimum 1.27.0 /opt/venv/lib/python3.12/site-packages pip
optree 0.17.0 /opt/venv/lib/python3.12/site-packages pip
orderly-set 5.5.0 /opt/venv/lib/python3.12/site-packages pip
packaging 25.0 /opt/venv/lib/python3.12/site-packages pip
pandas 2.2.3 /opt/venv/lib/python3.12/site-packages pip
pandocfilters 1.5.1 /opt/venv/lib/python3.12/site-packages pip
parso 0.8.5 /opt/venv/lib/python3.12/site-packages pip
peft 0.17.0 /opt/venv/lib/python3.12/site-packages pip
pettingzoo 1.25.0 /opt/venv/lib/python3.12/site-packages pip
pexpect 4.9.0 /opt/venv/lib/python3.12/site-packages pip
pfzy 0.3.4 /opt/venv/lib/python3.12/site-packages pip
pid 3.0.4 /opt/venv/lib/python3.12/site-packages pip
pillow 11.3.0 /opt/venv/lib/python3.12/site-packages pip
pip 25.2 /opt/venv/lib/python3.12/site-packages pip
pkginfo 1.12.1.2 /opt/venv/lib/python3.12/site-packages pip
platformdirs 4.4.0 /opt/venv/lib/python3.12/site-packages pip
pluggy 1.6.0 /opt/venv/lib/python3.12/site-packages pip
portalocker 3.2.0 /opt/venv/lib/python3.12/site-packages pip
prometheus_client 0.22.1 /opt/venv/lib/python3.12/site-packages pip
prompt_toolkit 3.0.52 /opt/venv/lib/python3.12/site-packages pip
propcache 0.3.2 /opt/venv/lib/python3.12/site-packages pip
protobuf 6.32.0 /opt/venv/lib/python3.12/site-packages pip
psutil 7.0.0 /opt/venv/lib/python3.12/site-packages pip
ptyprocess 0.7.0 /opt/venv/lib/python3.12/site-packages pip
pure_eval 0.2.3 /opt/venv/lib/python3.12/site-packages pip
pyarrow 21.0.0 /opt/venv/lib/python3.12/site-packages pip
pycparser 2.23 /opt/venv/lib/python3.12/site-packages pip
pycuda 2025.1.1 /opt/venv/lib/python3.12/site-packages pip
pydantic 2.10.6 /opt/venv/lib/python3.12/site-packages pip
pydantic_core 2.27.2 /opt/venv/lib/python3.12/site-packages pip
Pygments 2.19.2 /opt/venv/lib/python3.12/site-packages pip
PyGObject 3.48.2 /usr/lib/python3/dist-packages
PyJWT 2.7.0 /usr/lib/python3/dist-packages debian
pynput 1.8.1 /opt/venv/lib/python3.12/site-packages pip
pyparsing 3.1.1 /usr/lib/python3/dist-packages debian
pyrsistent 0.20.0 /usr/lib/python3/dist-packages
pyserial 3.5 /opt/venv/lib/python3.12/site-packages pip
pytest 8.4.2 /opt/venv/lib/python3.12/site-packages pip
python-apt 2.7.7+ubuntu5 /usr/lib/python3/dist-packages
python-daemon 3.1.2 /opt/venv/lib/python3.12/site-packages pip
python-dateutil 2.9.0.post0 /opt/venv/lib/python3.12/site-packages pip
python-json-logger 3.3.0 /opt/venv/lib/python3.12/site-packages pip
python-xlib 0.33 /opt/venv/lib/python3.12/site-packages pip
pytools 2025.2.4 /opt/venv/lib/python3.12/site-packages pip
pytorch3d 0.7.8 /opt/venv/lib/python3.12/site-packages pip
pytz 2025.2 /opt/venv/lib/python3.12/site-packages pip
PyYAML 6.0.2 /opt/venv/lib/python3.12/site-packages pip
pyyaml-include 1.4.1 /opt/venv/lib/python3.12/site-packages pip
pyzmq 27.1.0 /opt/venv/lib/python3.12/site-packages pip
ray 2.40.0 /opt/venv/lib/python3.12/site-packages pip
readme_renderer 44.0 /opt/venv/lib/python3.12/site-packages pip
referencing 0.36.2 /opt/venv/lib/python3.12/site-packages pip
regex 2025.9.1 /opt/venv/lib/python3.12/site-packages pip
requests 2.32.3 /opt/venv/lib/python3.12/site-packages pip
requests-toolbelt 1.0.0 /opt/venv/lib/python3.12/site-packages pip
rerun-sdk 0.24.1 /opt/venv/lib/python3.12/site-packages pip
rfc3339-validator 0.1.4 /opt/venv/lib/python3.12/site-packages pip
rfc3986 2.0.0 /opt/venv/lib/python3.12/site-packages pip
rfc3986-validator 0.1.1 /opt/venv/lib/python3.12/site-packages pip
rfc3987-syntax 1.1.0 /opt/venv/lib/python3.12/site-packages pip
rich 14.1.0 /opt/venv/lib/python3.12/site-packages pip
rpds-py 0.27.1 /opt/venv/lib/python3.12/site-packages pip
safetensors 0.6.2 /opt/venv/lib/python3.12/site-packages pip
scikit-build 0.18.1 /opt/venv/lib/python3.12/site-packages pip
scikit-image 0.25.2 /opt/venv/lib/python3.12/site-packages pip
scipy 1.16.1 /opt/venv/lib/python3.12/site-packages pip
SecretStorage 3.4.0 /opt/venv/lib/python3.12/site-packages pip
semantic-version 2.10.0 /opt/venv/lib/python3.12/site-packages pip
Send2Trash 1.8.3 /opt/venv/lib/python3.12/site-packages pip
sentencepiece 0.2.1 /opt/venv/lib/python3.12/site-packages pip
sentry-sdk 2.37.1 /opt/venv/lib/python3.12/site-packages pip
service 0.6.0 /opt/venv/lib/python3.12/site-packages pip
setproctitle 1.3.7 /opt/venv/lib/python3.12/site-packages pip
setuptools 80.9.0 /opt/venv/lib/python3.12/site-packages pip
setuptools-rust 1.12.0 /opt/venv/lib/python3.12/site-packages pip
shtab 1.7.2 /opt/venv/lib/python3.12/site-packages pip
siphash24 1.8 /opt/venv/lib/python3.12/site-packages pip
six 1.16.0 /usr/lib/python3/dist-packages
smmap 5.0.2 /opt/venv/lib/python3.12/site-packages pip
sniffio 1.3.1 /opt/venv/lib/python3.12/site-packages pip
soupsieve 2.8 /opt/venv/lib/python3.12/site-packages pip
stack-data 0.6.3 /opt/venv/lib/python3.12/site-packages pip
sympy 1.14.0 /opt/venv/lib/python3.12/site-packages pip
tensorboard 2.20.0 /opt/venv/lib/python3.12/site-packages pip
tensorboard-data-server 0.7.2 /opt/venv/lib/python3.12/site-packages pip
tensorflow 2.20.0 /opt/venv/lib/python3.12/site-packages pip
tensorrt 10.13.2.6 /opt/venv/lib/python3.12/site-packages pip
termcolor 3.1.0 /opt/venv/lib/python3.12/site-packages pip
terminado 0.18.1 /opt/venv/lib/python3.12/site-packages pip
tf_keras 2.20.1 /opt/venv/lib/python3.12/site-packages pip
tianshou 0.5.1 /opt/venv/lib/python3.12/site-packages pip
tifffile 2025.9.9 /opt/venv/lib/python3.12/site-packages pip
timm 1.0.14 /opt/venv/lib/python3.12/site-packages pip
tinycss2 1.4.0 /opt/venv/lib/python3.12/site-packages pip
tokenizers 0.21.4 /opt/venv/lib/python3.12/site-packages pip
toml 0.10.2 /opt/venv/lib/python3.12/site-packages pip
torch 2.9.0 /opt/venv/lib/python3.12/site-packages pip
torchvision 0.24.0 /opt/venv/lib/python3.12/site-packages pip
tornado 6.5.2 /opt/venv/lib/python3.12/site-packages pip
tqdm 4.67.1 /opt/venv/lib/python3.12/site-packages pip
traitlets 5.14.3 /opt/venv/lib/python3.12/site-packages pip
transformers 4.51.3 /opt/venv/lib/python3.12/site-packages pip
triton 3.5.0+gite3b8f77a /opt/venv/lib/python3.12/site-packages pip
twine 6.2.0 /opt/venv/lib/python3.12/site-packages pip
typeguard 4.4.2 /opt/venv/lib/python3.12/site-packages pip
types-python-dateutil 2.9.0.20250822 /opt/venv/lib/python3.12/site-packages pip
typing_extensions 4.12.2 /opt/venv/lib/python3.12/site-packages pip
typing-inspect 0.9.0 /opt/venv/lib/python3.12/site-packages pip
tyro 0.9.17 /opt/venv/lib/python3.12/site-packages pip
tzdata 2025.2 /opt/venv/lib/python3.12/site-packages pip
uri-template 1.3.0 /opt/venv/lib/python3.12/site-packages pip
urllib3 2.5.0 /opt/venv/lib/python3.12/site-packages pip
uv 0.8.16 /opt/venv/lib/python3.12/site-packages pip
wadllib 1.3.6 /usr/lib/python3/dist-packages
wandb 0.18.0 /opt/venv/lib/python3.12/site-packages pip
wcwidth 0.2.13 /opt/venv/lib/python3.12/site-packages pip
webcolors 24.11.1 /opt/venv/lib/python3.12/site-packages pip
webencodings 0.5.1 /opt/venv/lib/python3.12/site-packages pip
websocket-client 1.8.0 /opt/venv/lib/python3.12/site-packages pip
Werkzeug 3.1.3 /opt/venv/lib/python3.12/site-packages pip
wheel 0.45.1 /opt/venv/lib/python3.12/site-packages pip
widgetsnbextension 4.0.14 /opt/venv/lib/python3.12/site-packages pip
wrapt 1.17.3 /opt/venv/lib/python3.12/site-packages pip
xxhash 3.5.0 /opt/venv/lib/python3.12/site-packages pip
yarl 1.20.1 /opt/venv/lib/python3.12/site-packages pip
zipp 3.23.0 /opt/venv/lib/python3.12/site-packages pip
La imagen se puede descargar directamente desde Docker Hub e incluye dependencias comúnmente utilizadas como PyTorch, TensorRT y FlashAttention:
docker pull johnnync/lerobot:r38.2.aarch64-cu130-24.04
Para ejecutar Docker en Thor, consulte el siguiente comando. Reemplace your_docker_img:tag con el nombre y etiqueta de su imagen Docker, o use el ID de la imagen:
sudo docker run --rm -it \
--network=host \
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics \
--runtime nvidia \
--privileged \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v /etc/X11:/etc/X11 \
--device /dev/nvhost-vic \
-v /dev:/dev \
your_docker_img:tag
Recolección de Datos Usando el SO-ARM
Para un tutorial detallado sobre la recolección de datos, consulte el siguiente enlace:
https://wiki.seeedstudio.com/es/lerobot_so100m_new/#record-the-dataset
Este enlace contiene un tutorial completo para el SO-ARM, que cubre configuración, ensamblaje, calibración, recolección de datos, entrenamiento e inferencia.
La recolección de datos para el brazo robótico lerobot se puede realizar ya sea en una PC o directamente en un dispositivo Jetson.
- Método 1: Recolectar datos usando Jetson
- Método 2: Recolectar datos usando una PC Ubuntu
Los procedimientos de recolección de datos son esencialmente los mismos para ambos métodos.
¡Jetson puede no ser capaz de transmitir simultáneamente dos cámaras USB con los mismos requisitos de ancho de banda. Si conecta dos cámaras USB, deben estar conectadas a diferentes chips de hub USB!
Para transmitir dos cámaras USB simultáneamente en Thor, después de conectar una cámara al puerto USB-A, también necesitará usar un hub USB externo conectado a través del puerto USB Type-C. Esto asegura que la segunda cámara USB esté montada en un controlador de hub USB diferente dentro de Thor. Un ejemplo de un hub USB Type-C compatible se muestra a continuación:
Configuración del Entorno Lerobot
El proceso de configuración del entorno de desarrollo para Lerobot se puede encontrar en la subsección del siguiente enlace:
https://wiki.seeedstudio.com/es/lerobot_so100m_new/#install-lerobot
Configurar los motores
Los motores en cada articulación del SO-ARM necesitan ser configurados antes del ensamblaje. Los pasos de configuración se pueden encontrar en la subsección del siguiente enlace: https://wiki.seeedstudio.com/es/lerobot_so100m_new/#configure-the-motors
Ensamblaje
El proceso de instalación de los brazos maestro y seguidor del SO-ARM se puede encontrar en la subsección del siguiente enlace:
https://wiki.seeedstudio.com/es/lerobot_so100m_new/#assembly
Calibración del SO-ARM
Después de que el SO-ARM haya sido completamente ensamblado, se requiere calibración. Consulte la subsección del siguiente enlace para el procedimiento de calibración:
https://wiki.seeedstudio.com/es/lerobot_so100m_new/#calibrate
Al calibrar los brazos robóticos, por favor NO conecte ninguna cámara USB, ya que esto puede causar conflictos de puertos o asignaciones incorrectas de puertos.
¡Después de ejecutar el script de calibración, mueva manualmente cada articulación del brazo robótico para asegurar que alcance su rango completo de movimiento! No hacerlo puede resultar en una discrepancia entre las poses de los brazos líder y seguidor durante la teleoperación.
Camera Add
It is generally recommended to install one camera on the wrist/gripper of the robotic arm, and another camera on the desktop surface, to ensure proper coverage of the arm’s posture. The specific installation approach depends on your application scenario; the example shown below is for reference only.
For details, please refer to the subsection of the following link:
https://wiki.seeedstudio.com/lerobot_so100m_new/#add-cameras
Data Collection
After completing both the camera installation and the robotic arm calibration, the dataset collection procedure can be found in the subsection of the following link:
https://wiki.seeedstudio.com/lerobot_so100m_new/#record-the-dataset
Visualizing Collected Data
To visualize the data collected on the SO-ARM, please refer to the subsection of the following link:
https://wiki.seeedstudio.com/lerobot_so100m_new/#visualize-the-dataset
Policy Training
Based on the collected data, the procedure for training a Policy can be found in the subsection of the following link:
https://wiki.seeedstudio.com/lerobot_so100m_new/#train-a-policy
In the following sections, we will introduce a cloud-based training platform and demonstrate basic usage. You may choose to complete training more efficiently on the remote server.
If you choose to train the policy in the cloud, make sure that your dataset is uploaded to the cloud server in advance, or downloaded from the Hugging Face Hub. However, due to potential network issues when downloading directly from the Hugging Face Hub, it is strongly recommended to manually upload the dataset to your cloud server.
Use NVIDIA Brev for training policies
NVIDIA Brev provides streamlined access to NVIDIA GPU instances on popular cloud platforms, automatic environment setup, and flexible deployment options, enabling developers to start experimenting instantly.
Platform access URL: https://login.brev.nvidia.com/signin
You will need to register an account to use the platform. This section introduces the basic usage of the training platform.
Enabling the Cloud-Based Training Platform
Step-by-Step Instructions
- Create a Cloud Server Instance -step 1
- Create a Cloud Server Instance -step 2
- Create a Cloud Server Instance -step 3
Launch the Web-Based Jupyter Notebook：
Training the Model and Exporting from the Server
Developers can train models directly within the notebook terminal. Below is an example for training an act model and exporting it from the server afterward.
Install Conda on the server:
mkdir -p ~/miniconda3
cd miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
source ~/miniconda3/bin/activate
conda init --all
Install lerobot projectt:
conda create -y -n lerobot python=3.10 && conda activate lerobot
git clone https://github.com/Seeed-Projects/lerobot.git ~/lerobot
cd ~/lerobot && pip install -e ".[feetech]"
Train the ACT model:
python -m lerobot.scripts.train \
--dataset.repo_id=seeedstudio123/test \
--policy.type=act \
--output_dir=outputs/train/act_so101_test \
--job_name=act_so101_test \
--policy.device=cuda \
--wandb.enable=false \
--policy.push_to_hub=false\
--steps=300000
If you want to train or fine-tune Gr00t on a server, you can refer to this link.
After training is completed, you may want to download the model to your local machine. But he Jupyter Notebook sidebar does not support direct navigation into model training folders.
Solution is that: First compress the target folder into a .zip or .tar.gz archive, then download the archive file through the notebook interface. Refer to the figure below for an example.
Isaac GR00T N1.5 Inference on Thor
Jetson AGX Thor, as a powerful edge computing and deployment platform, provides sufficient resources to support large-scale model inference. In this section, building upon the previously introduced content, we demonstrate how to run inference for GR00T N1.5 on Thor.
GR00T N1.5 is an open-source baseline system released by NVIDIA Research in the field of robot learning. It aims to provide a unified framework for embodied AI training and inference, particularly focusing on imitation learning and policy learning driven by large-scale models.
Preparation
The pre-trained models of GR00T N1.5 are available via Hugging Faces. You can download them from the following link:
https://huggingface.co/nvidia/GR00T-N1.5-3B/tree/main
All dependencies required for GR00T inference have been pre-configured in a dedicated Docker image.
Use the following command to start the container:
sudo docker run --rm -it \
--network=host \
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics \
--runtime nvidia \
--privileged \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v /etc/X11:/etc/X11 \
--device /dev/nvhost-vic \
-v /dev:/dev \
lerobot:r38.2.aarch64-cu130-24.04
Git clone the source code of Gr00t,and install it:
git clone https://github.com/NVIDIA/Isaac-GR00T.git
cd Isaac-GR00T
pip install --upgrade setuptools
pip install -e .[thor]
Gr00t is fully compatible with the datasets collected using the lerobot framework. Refer to the previous "Data Collection" section to prepare your dataset for fine-tuning the Gr00t model.
Model Fine-Tuning
The fine-tuning process can be executed either on the provided cloud training platform or directly inside the Docker container on Thor.
If you have not downloaded any pretrained GR00T model and do not plan to use a custom version, you may use the following command to fine-tune based on the Hugging Face weights:
python scripts/gr00t_finetune.py \
--dataset-path ./demo_data/so101-table-cleanup/ \
--num-gpus 1 \
--output-dir ./so101-checkpoints \
--max-steps 10000 \
--data-config so100_dualcam \
--video-backend torchvision_av
This script will automatically download the pretrained GR00T model from Hugging Face and begin the fine-tuning process.
If you wish to use a locally stored pretrained GR00T model, modify the command as follows:
python scripts/gr00t_finetune.py \
--dataset-path ./demo_data/so101-table-cleanup/ \
--num-gpus 1 \
--output-dir ./so101-checkpoints \
--max-steps 10000 \
--data-config so100_dualcam \
--video-backend torchvision_av \
--base-model-path ./pretrained/GR00T-N1.5-3
--dataset-path is the file path of collection data from SO-ARM.
The default fine-tuning settings require ~25G of VRAM. If you don't have that much VRAM, try adding the --no-tune_diffusion_model flag to the gr00t_finetune.py script.
Running Inference with GR00T N1.5
To achieve optimal performance, it is recommended to replicate the real-world setup used during data collection as closely as possible when deploying fine-tuned GR00T models. This is because the model's generalization capabilities are limited.
Inside the Docker container, open a terminal and launch the GR00T inference service:
python scripts/inference_service.py --server \
--model_path ./so101-checkpoints \
--embodiment-tag new_embodiment \
--data-config so100_dualcam \
--denoising-steps 4
Upon successful launch, the terminal should display logs similar to:
Open another terminal, and use the following command to enter the same container from a different shell:
sudo docker exec -it <container id> /bin/bahs
Then, in this second shell, start the inference client:
python examples/eval_lerobot.py \
--robot.type=so100_follower \
--robot.port=/dev/ttyACM0 \
--robot.id=my_awesome_follower_arm \
--robot.cameras="{ wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, front: {type: opencv, index_or_path: 1, width: 640, height: 480, fps: 30}}" \
--policy_host=0.0.0.0 \
--lang_instruction="Grab pens and place into pen holder."
When the client process starts successfully, the following output should appear:
During the first run of the client process, you must calibrate the arm's servos. The calibration process is the same as described earlier.
Make sure each joint moves to its full range of motion to ensure proper teleoperation and inference behavior.
Once everything is set up, GR00T N1.5 can be successfully deployed on Jetson AGX Thor. The system and hardware environment are now verified to support full inference:
FAQ
Q1: The Brev CLI tool doesn't work on the cloud training platform?
This is often due to network issues. You may install and log in to Brev CLI on your local Ubuntu host, then attempt to connect to your cloud instance using SSH from your local terminal.
Q2: How do I upload data to the training platform?
Use the following command: scp <local-file-path> <brev-instance-name>:<remote-file-path>,例如scp -r ./record_2_cameras/ gr00t-trainer:/home/ubuntu/Datasets
References
- https://developer.nvidia.com/embedded/jetpack
- https://huggingface.co/blog/nvidia/gr00t-n1-5-so101-tuning
Tech Support & Product Discussion
Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.