Skip to main content

Control SO-Arm by OpenClaw on Jetson Thor

Introduction​

This wiki explains how to combine OpenClaw and LeRobot on Jetson Thor to control a SO-Arm with a local AI agent.

NVIDIA Jetson AGX Thor is a high-performance edge AI platform designed for robotics and physical AI workloads, providing strong on-device compute for perception, planning, and control.

SO-Arm is an open-source low-cost robotic arm platform (SO-ARM100/SO-ARM101) that is widely used for embodied AI experiments, teleoperation, and manipulation task development.

OpenClaw is an AI agent framework that can orchestrate local tools and models. In this project, OpenClaw is used as the high-level control interface, while LeRobot provides the low-level motor communication and calibration utilities for SO-Arm.

note

In this guide, OpenClaw handles agent planning and task orchestration, while SO-Arm motion execution is handled by LeRobot.

Table of Contents​

  1. Hardware Preparation
  2. System Setup on Jetson Thor
  3. Install Ollama and Run a Local LLM
  4. Install OpenClaw on Jetson Thor
  5. Connect and Calibrate SO-Arm
  6. Run Control Demo
  7. References

Hardware Preparation​

Device List​

  • 1x NVIDIA® Jetson AGX Thor™ Developer Kit
  • 1x SO-ARM101 Low-Cost AI Arm
NVIDIA® Jetson AGX Thor™ Developer Kit SO-ARM101 Low-Cost AI Arm
Get One Now 🖱️
Get One Now 🖱️

Wiring and Connection​

  • Connect the SO-Arm controller board to Thor through USB.
  • Connect the matching DC power adapter to the SO-Arm controller board.
  • Power on Thor, then power on the arm controller board.

Power-On Checklist​

  • Thor boots normally and network is available.
  • SO-Arm controller board LEDs are on.
  • Serial device appears after USB connection.
ls /dev/ttyACM*

If a serial node is detected in terminal output, the hardware connection is correct.

System Setup on Jetson Thor​

Update System Packages​

sudo apt update
sudo apt install -y nvidia-jetpack git curl ffmpeg python3-pip
python3 -m pip install -U pip

Install Core Dependencies​

Install Miniconda (recommended):

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh
chmod +x Miniconda3-latest-Linux-aarch64.sh
./Miniconda3-latest-Linux-aarch64.sh
source ~/.bashrc

Create LeRobot environment:

conda create -y -n lerobot python=3.12
conda activate lerobot
git clone https://github.com/huggingface/lerobot.git ~/lerobot
cd ~/lerobot
pip install -e . 
pip uninstall torch torchvision
pip install torch torchvision --index-url https://pypi.jetson-ai-lab.io

Install Pinocchio in the LeRobot environment:

conda install pinocchio -c conda-forge

Verify CUDA and Peripheral Devices​

python -c "import torch; print(torch.cuda.is_available())"
lerobot-find-port

Expected result:

  • torch.cuda.is_available() prints True
  • arm serial ports are detected (for example /dev/ttyACM0)

Install Ollama and Run a Local LLM​

Install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Pull a model:

ollama pull qwen3-vl:9b
info

This guide uses qwen3-vl:9b as an example. You can replace it with another Ollama model based on your performance and memory constraints.

Install OpenClaw on Jetson Thor​

Install OpenClaw​

curl -fsSL https://openclaw.ai/install.sh | bash

Configure Runtime Parameters​

Edit ~/.openclaw/openclaw.json and set Ollama as the default model provider:

{
  "agents": {
    "defaults": {
      "models": {
        "ollama": {}
      },
      "model": {
        "primary": "ollama/qwen3-vl:9b"
      }
    }
  },
  "models": {
    "providers": {
      "ollama": {
        "baseUrl": "http://127.0.0.1:11434/v1",
        "apiKey": "ollama-local",
        "api": "openai-completions",
        "models": [
          {
            "id": "qwen3-vl:9b",
            "name": "Qwen3 VL 9B",
            "reasoning": false,
            "input": [
              "text"
            ],
            "cost": {
              "input": 0,
              "output": 0,
              "cacheRead": 0,
              "cacheWrite": 0
            },
            "contextWindow": 128000,
            "maxTokens": 8192
          }
        ]
      }
    }
  }
}
note

Optional: you can also directly use the script provided by Ollama to quickly set up the OpenClaw configuration file.

ollama launch openclaw --model qwen3.5

More Configure​

Install the SO-Arm control skill:

Prepare robot description file:

  • Download SO-ARM101 URDF
  • Move it to ~/.openclaw/workspace/skills/soarm-control/references

Restart OpenClaw gateway:

openclaw gateway restart

Open WebUI:

http://127.0.0.1:18789/

Connect and Calibrate SO-Arm​

Serial Port Permission and Detection​

conda activate lerobot
lerobot-find-port
sudo chmod 666 /dev/ttyACM*

Initial Calibration​

Calibrate follower arm:

lerobot-calibrate \
  --robot.type=so101_follower \
  --robot.port=/dev/ttyACM0 \
  --robot.id=openclaw_soarm

Calibration files are saved under:

~/.cache/huggingface/lerobot/calibration/

note

For a full calibration walkthrough, refer to: SO-Arm in LeRobot - Calibrate

Run Control Demo​

Launch OpenClaw Service​

Ensure OpenClaw and LeRobot environments are ready:

openclaw gateway restart
conda activate lerobot

Execute Basic Motion Task​

In OpenClaw WebUI, enter robot control instructions. OpenClaw will parse your prompt and call the installed soarm-control skill to drive the arm to the target position.

The video demonstrates three commands:

  1. Move the robotic arm’s end effector upward by 20 cm.
  2. Then move forward by 20 cm, keeping the height unchanged.
  3. Return to the initial position.

OpenClaw’s responses to these three commands all match the expected behavior.

References​

Tech Support & Product Discussion​

Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.

Loading Comments...