Skip to main content

B601-RS with Isaac Sim

reBot Arm B601-RS Isaac Sim
Get One Now

Introduction​

reBot-Isaacsim is an NVIDIA Isaac Sim simulation project designed specifically for the reBotArm. It leverages Isaac Sim's high-fidelity physics engine to accurately replicate the kinematic characteristics and gripper coordination logic of the robot arm in a virtual environment, providing an independent simulation-only environment for control algorithm development, trajectory planning verification, and communication protocol testing.

Environmental requirements​

  • Operating System: Ubuntu 22.04 LTS / 24.04 LTS (recommended) or Windows 11 (requires WSL2)
  • GPU: NVIDIA RTX series graphics card (recommended RTX 3070 or above), VRAM ≥ 8GB
  • Driver: NVIDIA official driver ≥ 535.x, supports CUDA 12.x
  • Memory: ≥ 32GB RAM (Isaac Sim scenes and physics simulations use a lot of memory)
  • Storage: ≥ 100GB SSD available space (for Isaac Sim installation, cache, and USD assets)
info

The computers used in this wiki are equipped with NVIDIA RTX 4080 GPUs and run the Ubuntu 22.04 LTS operating system.

Install Isaac Sim​

Official links and resources:

https://docs.isaacsim.omniverse.nvidia.com/6.0.0/installation/quick-install.html

https://docs.isaacsim.omniverse.nvidia.com/6.0.0/installation/download.html#isaac-sim-latest-release

Select an Isaac Sim installation method, then continue with the project download.

🔧 Method 1: Precompiled Binary Installation​

💡 Suitable for most users, no need to compile, ready to use out of the box.

1

Download and unzip​

On the NVIDIA Isaac Sim 6.0.0 download page, select Linux (x86_64) in the Isaac Sim row and download the precompiled package isaac-sim-standalone-6.0.0-linux-x86_64.zip to ~/Downloads. The following commands are for Linux x86_64.

mkdir -p ~/isaacsim
cd ~/Downloads
unzip isaac-sim-standalone-6.0.0-linux-x86_64.zip -d ~/isaacsim
cd ~/isaacsim
./post_install.sh
2

Set up environment variables​

Add the following to ~/.bashrc or ~/.zshrc:

export ISAACSIM_PATH="${HOME}/isaacsim"
export ISAACSIM_PYTHON_EXE="${ISAACSIM_PATH}/python.sh"
export ISAACSIM_ROOT="${HOME}/isaacsim"

Then run source ~/.bashrc to make it take effect.

3

Start Verification​

${ISAACSIM_PATH}/isaac-sim.sh

The first launch will cache shaders, which may take 5-10 minutes, so please be patient while the GUI appears.

Download project​

1

Clone the project and submodules​

This repository uses the upstream reBotArm_control_py Git submodule. Clone from the directory where you want to store the project:

git clone --recurse-submodules https://github.com/Seeed-Projects/reBot-Isaacsim.git
cd reBot-Isaacsim

If you already cloned the repository and third_party/reBotArm_control_py is empty, run this from the project root:

git submodule update --init --recursive
2

Install sender dependencies​

Run this from the project root. Both run_sender.sh and uv run use the root uv workspace:

uv sync
3

Switch the hardware config to RS​

This repository's Isaac Sim asset is RS (usd/RS-rebot-dev-arm), while upstream rebotarm.yaml defaults to DM. Both RebotArm() and load_robot_model() use this file, so gravity compensation, joint-reader, IK, and Traj require the RS configuration; otherwise, the motor protocol will not match and Pinocchio will load the DM URDF. The joint test does not require this step. This modifies the submodule working tree locally; do not commit the change:

python reBotArm_Isaacsim/set_hw_rs.py

On success it prints .../config/rebotarm.yaml -> rebotarm_rs.yaml. The shell remains in the project root.

Overview of Functional Components​

This project provides various senders to meet different use scenarios:

ComponentDescription
gravity_joint_senderGravity Compensation + Handle Mode: For modified robot arms (gripper removed, handle attached), hand-guided; compensation comes from upstream GravityCompensation, this repo only mirrors joint angles to Isaac Sim
gravity_gripper_assistGravity Compensation and Gripper Assist: Experimental example with a model-only check
isaacsim_ik_senderInverse Kinematics (IK) Mode: Input the end-effector pose, use the IK solver to get joint angles, and send them to Isaac Sim
isaacsim_traj_senderTrajectory Planning (Traj) Mode: Adds joint-space trajectory planning (MIN_JERK time profile) on top of IK to achieve smooth motion control
isaacsim_joint_test_senderJoint Test Mode: Sends preset joint angle trajectories without a real robot to verify if Isaac Sim's receiver and communication are working properly
joint_reader_senderReal-to-Sim Mapping Mode: Read-only joint angles and map them to Isaac Sim, ideal for use with other control projects (e.g., synchronize the real robot performing other tasks to Isaac Sim for visualization)

Directory Structure​

reBot-Isaacsim/
├── pyproject.toml                           # uv workspace configuration
├── README.md
├── README_EN.md
├── README_ES.md
├── reBotArm_Isaacsim/                       # Main example directory
│   ├── gravity_joint_sender.py              # Gravity-comp handle mode (upstream GravityCompensation + UDP)
│   ├── gravity_gripper_assist.py             # Gravity compensation with gripper assist
│   ├── isaacsim_ik_sender.py                # Inverse kinematics mode (IK control)
│   ├── isaacsim_traj_sender.py              # Trajectory planning mode (IK + joint-space trajectory)
│   ├── isaacsim_joint_test_sender.py        # Joint test mode (preset trajectory, no hardware needed)
│   ├── joint_reader_sender.py                # Real-to-Sim mapping mode (read-only joints, sync visualization)
│   ├── isaacsim_joint_receiver.py           # Isaac Sim receiver (joint-angle sync)
│   ├── live_sync.py                         # Startup instruction script
│   ├── set_hw_rs.py                         # Point submodule hardware YAML at RS (local; do not commit)
│   ├── run_sender.sh                        # Launch the sender
│   └── run_isaacsim_receiver.sh             # Launch the Isaac Sim receiver
├── .gitmodules
├── third_party/
│   └── reBotArm_control_py/                 # git submodule: upstream control library
└── usd/
    └── RS-rebot-dev-arm/
        └── RS-rebot-dev-arm.usda            # Isaac Sim robot asset

Launch (Dual-Terminal Mode)​

Two separate terminals are required. Terminal 1 runs the Isaac Sim receiver, while Terminal 2 runs the appropriate sender depending on the desired functionality. In each terminal, enter the reBot-Isaacsim project root cloned above before running the commands below.

1

Terminal 1 — Launch the Isaac Sim Receiver (Shared by All Modes)​

Step 1

cd reBotArm_Isaacsim
./run_isaacsim_receiver.sh

Expected behavior:

  • Launch the Isaac Sim GUI
  • Load the ground plane and robot arm USD assets
  • Listen for UDP packets on 127.0.0.1:5005
  • Wait for a sender to connect
2

Terminal 2 — Launch the Appropriate Sender​

Step 2

For simulation mirroring, start the receiver first, then the sender. The model-only check for the experimental gripper assist below does not require the receiver.

tip

Some modes require a physical robot arm. Configure the USB-to-CAN adapter as shown below.

# Check the CAN interface status
ip link show can0

# Bring up the CAN interface with a bitrate of 1000000
sudo ip link set can0 up type can bitrate 1000000 restart-ms 100

Choose one of the following modes. These are independent options and do not need to be run in sequence.

1

Joint Test Mode (isaacsim_joint_test_sender)​

No physical hardware is required. A predefined joint trajectory is continuously sent to verify communication with the Isaac Sim receiver.

cd reBotArm_Isaacsim
uv run python isaacsim_joint_test_sender.py

The sender continuously interpolates between several predefined joint configurations and transmits them to Isaac Sim. It does not read the hardware YAML, so neither set_hw_rs.py nor CAN is required.

2

Inverse Kinematics Mode (isaacsim_ik_sender)​

Enter an end-effector pose (position/orientation). The IK solver computes the joint configuration and drives the robot arm in Isaac Sim. load_robot_model() reads the submodule rebotarm.yaml, so switch to RS first:

cd reBotArm_Isaacsim
python set_hw_rs.py
uv run python isaacsim_ik_sender.py

Input format (one command per line):

x y z                       # Position (meters), keep current orientation
x y z r p y                 # Position + orientation (meters/degrees)
q j1 j2 j3 j4 j5 j6         # Send joint angles directly (degrees)
gripper <0~1>               # Update gripper only
3

Trajectory Planning Mode (isaacsim_traj_sender)​

Adds joint-space trajectory planning (MIN_JERK) on top of IK for smooth robot motion. It also uses load_robot_model() against that YAML, so switch to RS first:

cd reBotArm_Isaacsim
python set_hw_rs.py
uv run python isaacsim_traj_sender.py

Input format (one command per line):

x y z                       # Position (meters)
x y z r p y                 # Position + orientation (meters/degrees)
q j1 j2 j3 j4 j5 j6         # Send joint angles directly (degrees)
gripper <0~1>               # Update gripper only
speed <scale>               # Adjust trajectory duration scaling
resync                      # Re-read the current joint state from Isaac Sim
4

Gravity Compensation Handle Mode (gravity_joint_sender)​

Designed for modified robot arms (gripper removed and handle installed). The robot can be manually guided while Isaac Sim follows the motion.

cd reBotArm_Isaacsim
python set_hw_rs.py
./run_sender.sh

Expected behavior:

  • set_hw_rs.py points the submodule rebotarm.yaml at rebotarm_rs.yaml so motors and the gravity model share one YAML (local change; do not commit)
  • The physical arm connects and starts upstream GravityCompensation (same MIT + g(q) feed-forward as example/9)
  • The arm can be moved freely by hand
  • This script only forwards joint angles to Isaac Sim over UDP at 60 Hz
  • Do not also run upstream example/9 at the same time; the two processes would contend for CAN
5

Gravity Compensation and Gripper Assist (gravity_gripper_assist)​

Manually move an RS arm and its attached gripper while mirroring them in Isaac Sim.

Run the model check from the project root; it does not connect to CAN:

cd reBotArm_Isaacsim
python set_hw_rs.py
uv run python gravity_gripper_assist.py --check

Start the Isaac Sim receiver, ensure can0 is available, secure and support the arm, and stop other programs using CAN. The gripper defaults to 0 rad closed and 5 rad open; these must match the arm's zero calibration.

Start the physical arm from the same terminal; this command controls the motors:

uv run python gravity_gripper_assist.py

If the endpoints differ, set the measured values in radians with --closed-rad and --open-rad. Enter assist, open, close, or hold to control the gripper; enter quit or press Ctrl+C to stop.

6

Real-to-Sim Mapping Mode (joint_reader_sender)​

Reads joint angles only and mirrors the physical robot state into Isaac Sim. This mode is intended for visualization while the real robot is controlled by another application. RebotArm() reads the submodule rebotarm.yaml, so switch to RS first:

cd reBotArm_Isaacsim
python set_hw_rs.py
uv run python joint_reader_sender.py

Expected behavior:

  • set_hw_rs.py switches the motor config to RS (local change; do not commit)
  • Read joint angles only (passive feedback mode), without sending any control commands
  • Continuously transmit joint angles via UDP at 60 Hz
  • Visualize the physical robot in Isaac Sim while it is controlled by another project

Communication Protocol​

UDP JSON over port 127.0.0.1:5005.

Sender Payload (per frame):

{
  "sequence": 123,
  "timestamp": 1718000000.123,
  "joint_positions": [0.0, 0.1, 0.2, -0.1, 0.0, -0.02],
  "gripper_position": 0.05
}
FieldTypeDescription
sequenceintIncrementing frame sequence number
timestampfloatUnix timestamp (seconds)
joint_positionsfloat[6]First six joint positions (rad)
gripper_positionfloatGripper finger position target (m); each sender computes it with its own mapping (see below)

Gripper control pipeline:

The receiver applies the received gripper_position directly as the position target of both prismatic finger joints, clipped per finger to [0, upper limit] (USD upper limits: 0.05 m on both fingers; the fingers are driven 1:1 by one motor through a single pinion). There is no extra scaling on the receiver side. The senders map their input to gripper_position as follows:

SenderMapping to gripper_position (m)
gravity_joint_sendergripper_q × 0.03 (GRIPPER_POSITION_SCALE = 0.03)
joint_reader_sendergripper_q × 0.007 (GRIPPER_POSITION_SCALE = 0.007)
isaacsim_traj_senderratio × 0.045 (gripper <0~1> input, clipped to 0.045 m)
isaacsim_ik_senderraw ratio ∈ [0, 1] sent as meters, so any ratio ≥ a finger's upper limit fully opens that finger

Configuration Parameters​

Sender (gravity_joint_sender.py)​

ParameterDefaultDescription
Hardware YAMLset_hw_rs.py → rebotarm_rs.yamlRebotArm() reads submodule config/rebotarm.yaml; motors and Pinocchio share it
ARM_JOINT_COUNT6Number of arm joints
DEFAULT_PORT5005UDP port
DEFAULT_SEND_HZ60.0Transmission frequency (Hz)
GRIPPER_POSITION_SCALE0.03Conversion factor from gripper angle to position
position_alpha0.2Low-pass filter coefficient

Receiver (isaacsim_joint_receiver.py)​

ParameterDefaultDescription
ARM_JOINT_COUNT6Number of arm joints
DEFAULT_PORT5005UDP port
DEFAULT_RENDER_HZ120.0Simulation rendering frequency (Hz)
ROBOT_PRIM_PATH/World/reBotArmRobot Prim path in Isaac Sim
ASSET_RELATIVE_PATHusd/RS-rebot-dev-arm/RS-rebot-dev-arm.usdaRelative path to the USD asset

Troubleshooting​

OSError: [Errno 98] Address already in use​

Port 5005 is already occupied. Find and terminate the process using the port:

# Find the process using port 5005
sudo lsof -i :5005

# Terminate the process (replace PID with the actual process ID)
kill <PID>

Isaac Sim Asset Not Found​

Verify that the USD asset exists and that REPO_ROOT is configured correctly:

ls usd/RS-rebot-dev-arm/RS-rebot-dev-arm.usda

CAN Bus Not Ready​

Make sure the CAN interface is up and configured with the correct bitrate:

can_restart can0

# Verify the bitrate
ip -details link show can0 | grep bitrate

Joint Angles Not Synchronizing​

  • Verify that both the sender and receiver are using port 5005.
  • Check that the sender log continuously outputs [send].
  • Check that the receiver log continuously outputs [recv].
  • Try isaacsim_joint_test_sender.py to rule out hardware-related issues.

Components and Python Environments​

ComponentPython EnvironmentLaunch Script
Sender (Physical Robot)reBotArm_control_py uv environmentrun_sender.sh
Sender (Test Mode)reBotArm_control_py uv environmentisaacsim_joint_test_sender.py
ReceiverIsaac Sim official Python (python.sh)run_isaacsim_receiver.sh

Tech Support & Product Discussion​

Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.

Loading Comments...