reBot Arm B601-RS Digital Twin & Control Stack

Get One Now

Three.js visualization · ROS 2 · MuJoCo simulation · DeepSeek/MCP · Leader teleoperation

The reBot Arm B601-RS Digital Twin & Control Stack integrates a Three.js web console, a ROS 2 real-robot driver, an RS-specific MuJoCo dynamics scene, vision detection, trajectory and inverse kinematics, a natural-language Agent using DeepSeek by default, and Arm102 Leader teleoperation. This guide follows the current project through installation, simulation, web control, Agent setup, and leader/follower operation.

The real robot uses /rebot_arm_rs; simulation uses /rebot_arm_rs_sim. Users of older versions should first run git pull --ff-only in the project directory to update the repository, then update custom scripts, ROS interface prefixes, and Agent configuration and restart the controller, rosbridge, and web console. Changing the web target selector does not configure the backend namespace.

Star on GitHub4.4k

Project Features

1. RS-specific arm model

   The web console and ROS 2 use ReBot_Arm_RS.urdf and its matching STL meshes. The web server reads the model from the rebotarm_ros2_RS workspace first and keeps an offline fallback copy.

2. RobStride + SocketCAN real-robot link

   The real robot uses the Linux SocketCAN interface can0 at 1 Mbps by default. The controller communicates with RobStride motors through reBotArm_control_py.

3. MIT 125 Hz online control

   The web console updates targets at up to 60 Hz, while the real-robot controller generates an online reference limited by velocity, acceleration, and jerk at 125 Hz and sends MIT q/dq/kp/kd/tau commands. Web target updates do not block the motor control loop.

4. Real-robot and simulation namespace isolation

   The real robot uses /rebot_arm_rs by default and simulation uses /rebot_arm_rs_sim. The same web console can switch control targets, and you can run the real robot and Fake Driver at the same time to compare interfaces.

5. RS-specific MuJoCo scene

   MuJoCo supports physics and kinematic modes. The default physics mode includes gravity, contact, tabletop objects, an overhead camera, color detection, and physical grasping validation.

6. Feedback-driven web animation

   In real-robot mode, the solid model shows actual feedback and the translucent model shows the control target. The web console applies adaptive low-pass filtering, a feedback dead zone, and display interpolation to rosbridge measurements so that local animation is not mistaken for the real arm pose.

7. Safety state machine

   The system arbitrates trajectories, gravity compensation, safe homing, and continuous web commands. Disabling from a non-zero pose runs safe homing first and verifies it; if verification fails, the arm remains enabled to avoid dropping suddenly.

8. Visual grasping and LLM/MCP Agent

   The simulation scene provides red, blue, and yellow objects with overhead detection, IK, grasping, lift validation, and placing support. The natural-language entry point defaults to DeepSeek deepseek-flash; MCP tools can also be called independently.

9. Arm102 Leader teleoperation

   The leader connects to USB/UART on the ROS host. The web console handles scanning, preview, unlocking, calibration, and sessions; the ROS controller handles mapping, speed limits, control ownership, and timeout holds. Relative following is the default, with optional gripper following and rebasing after a pause.

Wiring and Network Notes

RS Real-Robot CAN Wiring

1. Secure the arm and clear its range of motion.
2. Connect the RobStride motor CAN bus, power supply, and USB-CAN adapter.
3. Verify that CAN_H, CAN_L, and GND are wired correctly and configure the termination resistor according to the hardware requirements.
4. After power-on, confirm that the target CAN adapter appears in Ubuntu.

Configure can0:

sudo ip link set can0 down 2>/dev/null || true
sudo ip link set can0 type can bitrate 1000000
sudo ip link set can0 up
ip -details link show can0

Observe bus frames in read-only mode:

candump can0

Continuous CAN frames mean the interface is receiving data. Press Ctrl+C to exit candump.

If the interface enters BUS-OFF, first check power, grounding, baud rate, termination resistors, motor IDs, and CAN wiring. Do not mask hardware faults by repeatedly restarting the controller.

Web and ROS 2 Networking

The web console communicates with ROS 2 over the rosbridge WebSocket, using port 9090 by default:

Web console: http://<Ubuntu-IP>:3002
rosbridge:   ws://<Ubuntu-IP>:9090
MCP Server:  http://<Ubuntu-IP>:8081/mcp
Text Agent:  http://<Ubuntu-IP>:8082

You can use localhost when the web console and ROS 2 run on the same machine. When accessing from another computer on the LAN, do not enter the browser computer's own localhost; use the IP of the Ubuntu host running the ROS 2 services instead.

The project startup script sets the ROS 2 discovery range to LOCALHOST by default, so Wi-Fi roaming or IP changes do not split same-machine nodes into different DDS networks. This does not affect access to the HTTP/WebSocket ports from other devices. To let another computer join the ROS graph directly, set the same value in the relevant terminals:

export REBOTARM_ROS_DISCOVERY_RANGE=SUBNET

Environment Requirements

Item	Recommended requirement
Backend OS	Ubuntu 24.04
ROS 2	Jazzy
Compatible reference environment	Ubuntu 22.04 + ROS 2 Humble (real-robot use requires your own regression testing)
Python	3.12 (Jazzy) or 3.10 (Humble)
Node.js	18 or newer
Browser	Current stable Chrome, Chromium, Edge, or Firefox
CAN tools	iproute2, can-utils
Real-robot CAN	can0, 1 Mbps
MuJoCo	Installed by the project Python environment

Installation Steps

1

Step 1: Get the project

Step 1

git clone https://github.com/Yang-Ci/ReBot_Arm_DigitalTwin_RS.git ~/ReBot_Arm_DigitalTwin_RS
cd ~/ReBot_Arm_DigitalTwin_RS

The commands below use ~/ReBot_Arm_DigitalTwin_RS as the project directory. If your directory is different, replace the paths accordingly.

2

Step 2: Read-only environment check

Step 2

cd ~/ReBot_Arm_DigitalTwin_RS
./setup.sh --check

This command only checks the system and does not modify the environment. Checks include:

• Ubuntu, Python, and Node.js versions;
• ROS 2, rosbridge, MoveIt, and build tools;
• SocketCAN tools and can0;
• the Python virtual environment and key modules;
• ROS 2 workspace build results;
• the web package.json and .env.

3

Step 3: One-click install and build

Step 3

./setup.sh --yes
./rebotarm doctor

The installation script installs missing ROS 2, Node.js, SocketCAN, and build dependencies, creates rebotarm_ros2_RS/.venv, installs the Python dependencies for the RS real robot, MuJoCo, and Agent, runs rosdep, and builds the ROS 2 workspace with:

colcon build --symlink-install

The control SDK and MuJoCo models are tracked by the main repository as ordinary files. They are not Git submodules and do not contain nested Git repositories. Existing .env files, SDKs, and models are not reset by the installation script.

If the system dependencies are already complete, you only need to prepare and rebuild the ROS workspace:

./scripts/setup_rs_workspace.sh

After modifying Python packages, this is usually enough:

source scripts/rs_env.sh
cd rebotarm_ros2_RS
colcon build --symlink-install

4

Step 4: Configure web environment variables

Step 4

On first install, reBotArm_simulator-RS/.env is created from .env.example. Key fields:

PORT=3002
HTTPS=0
REBOTARM_TEXT_AGENT_URL=http://localhost:8082
REBOTARM_MCP_URL=http://localhost:8081/mcp
MOTORBRIDGE_WS_URL=ws://127.0.0.1:9002
MOTORBRIDGE_WS_TOKEN=

If the web console and ROS 2/Agent are not on the same machine, change the proxy addresses to the backend machine's IP. API keys and tokens should only be stored in environment variables or the uncommitted local .env.

5

Step 5 (Optional): Configure and start the default DeepSeek Agent

Step 5 (Optional)

To use the web natural-language assistant, follow “Launching the Project” below to start full simulation in terminal 1 (./rebotarm start rs_sim) and the web console in terminal 2 (./rebotarm start web). Then configure the API key and start the Agent in a third terminal:

cd ~/ReBot_Arm_DigitalTwin_RS
export DEEPSEEK_API_KEY='your-key'
./scripts/start_rs_text_agent.sh

The default model is DeepSeek deepseek-flash; no model override is needed. If you previously configured another model, check old REBOTARM_LLM_* environment variables so they do not override the defaults. Configure the API key only on the server and keep it out of the repository.

After [text-agent-http] listening on http://0.0.0.0:8082/ appears, click “Connect AI assistant” in the web console and send “Query the arm status” to verify the connection. See “DeepSeek / MCP Text Control” below for model switching, health checks, and troubleshooting.

Launching the Project

Pure web demo

Start only the Node.js web server, without rosbridge or ROS 2:

cd ~/ReBot_Arm_DigitalTwin_RS/reBotArm_simulator-RS
npm start

Open the browser at:

http://localhost:3002

This mode is suitable for developing the 3D model, UI, pose presets, and front-end features. Without the ROS 2 backend, the web console cannot control the real robot or the full MuJoCo scene.

Full RS MuJoCo simulation

Terminal 1 starts Fake Driver, MuJoCo, camera, detection, MCP Agent, and rosbridge:

cd ~/ReBot_Arm_DigitalTwin_RS
./rebotarm start rs_sim

Terminal 2 starts the web console:

cd ~/ReBot_Arm_DigitalTwin_RS
./rebotarm start web

Open http://localhost:3002, select the control target "RS Simulation (/rebot_arm_rs_sim)", and enter ws://localhost:9090 as the ROS WebSocket.

Common simulation options:

# No native MuJoCo window; suitable for automated tests
REBOTARM_MUJOCO_VIEWER=false ./rebotarm start rs_sim

# Kinematic following only; does not validate contact, gravity, or physical grasping
REBOTARM_MUJOCO_MODE=kinematic ./rebotarm start rs_sim

# Do not start the MCP Agent
REBOTARM_START_AGENT=false ./rebotarm start rs_sim

# After confirming there are no active tasks, clean up leftover simulation processes, then start
./rebotarm start rs_sim --force

Visual grasping must use the default physics mode. The kinematic mode directly synchronizes joint positions and cannot be used to evaluate control gains, contact stability, or grasping force.

RS real robot control

First configure and check SocketCAN:

sudo ip link set can0 down 2>/dev/null || true
sudo ip link set can0 type can bitrate 1000000
sudo ip link set can0 up
ip -details -statistics link show can0

After confirming that the emergency stop works and the workspace is clear, start the real-robot controller in terminal 1:

cd ~/ReBot_Arm_DigitalTwin_RS
REBOTARM_RS_HARDWARE_CONFIRM=I_UNDERSTAND_RS_WILL_MOVE \
  ./rebotarm start rs

Start the web console and rosbridge in terminal 2:

./rebotarm start web

Open http://localhost:3002 and select "RS Real Robot (/rebot_arm_rs)". For the first test, follow this order:

1. Check that the web console reports connected, the arm status is normal, and there are no motor errors.
2. Click enable.
3. Set the speed to 0.2-0.4 rad/s.
4. Move one joint by a small amount at a time and confirm direction, limits, and feedback.
5. Then test TCP dragging, trajectories, or gravity compensation.
6. When finished, perform safe homing and disable, then press Ctrl+C in the controller terminal.

Do not use Ctrl+Z to pause the real-robot controller, and do not treat a web button as a physical emergency stop. If an old controller instance is left behind, the real-robot startup script asks the old instance to exit safely and cleans up the residual resources confirmed to belong to that controller.

Dual real/Fake debugging

export REBOTARM_RS_HARDWARE_CONFIRM=I_UNDERSTAND_RS_WILL_MOVE
./scripts/start_rs_dual.sh

This script starts:

• the real robot /rebot_arm_rs;
• the Fake Driver /rebot_arm_rs_sim;
• rosbridge 9090.

It is used to compare interfaces, joint directions, and status. It does not start the full MuJoCo stack, vision, Agent, or web server. Confirm the namespace selected in the web console again before sending commands.

Leader Teleoperation

Connection and Installation

The Arm102 / 102-LD Leader connects by USB serial to the Ubuntu host running ROS 2. If ROS runs in a VM, pass the leader USB device through to that VM. A COM port on the browser computer cannot replace a serial port on the ROS host.

Arm102 Leader (servo IDs 0–6, UART 1 Mbps)
  -> ROS host USB serial / leader_worker.py
  -> LeaderTeleop: calibration, mapping, sessions, and status
  -> RS controller: control ownership, speed limits, and timeout holds
  -> Fake Driver / MuJoCo, or SocketCAN can0 / RS follower

Complete the base installation first. Stop the old controller, simulation, and rosbridge, then install the Leader dependencies and rebuild the interfaces:

cd ~/ReBot_Arm_DigitalTwin_RS
bash scripts/setup_rs_leader.sh

The script reuses the ROS Python virtual environment, installs motorbridge-smart-servo==0.0.4 and pyserial==3.5, and rebuilds rebotarm_msgs and rebotarmcontroller. The full LeRobot training environment is unnecessary for this feature. Restart the controller, rosbridge, and web console, then hard-refresh the browser.

Check in a new terminal:

cd ~/ReBot_Arm_DigitalTwin_RS
source scripts/rs_env.sh
python -c 'from motorbridge_smart_servo import FashionStarServo; from serial.tools import list_ports; print([(p.device, p.description) for p in list_ports.comports()])'
ros2 interface show rebotarm_msgs/srv/LeaderControl

Use /dev/ttyUSB0, /dev/ttyACM0, or a stable /dev/serial/by-id/... path. For a permissions error, run the following command and log in again before retrying:

sudo usermod -aG dialout "$USER"

Verify the Workflow in Simulation

Run in two separate terminals:

# Terminal 1
./rebotarm start rs_sim

# Terminal 2
./rebotarm start web

Select ROS control mode and “RS simulation (/rebot_arm_rs_sim)”, then connect to rosbridge. In the “Leader teleoperation” card, clear any old port, click “Scan leader”, and select mock; you can also enter mock directly in the port field. Follow the connection, unlocking, calibration, and following steps below, then test pause, resume, and stop.

Only the Fake Driver permits mock; real RS hardware rejects it. Mock mode verifies the UI, mapping, and session workflow. It does not validate the real serial link, joint directions, or hardware safety.

Connect, Calibrate, and Begin Following

For hardware tests, launch /rebot_arm_rs and the web console using the real-robot instructions. Enable the follower, confirm it is IDLE, and enable the ROS control lock. Stop trajectories, hand following, and gravity compensation; return the gripper to position hold. Do not run a separate LeRobot teleoperation process accessing the same can0.

1. Scan and connect. Scan serial ports on the ROS host in the “Leader teleoperation” card. A complete leader responds with all seven servo IDs 0–6. Select the device and click “Connect”, or enter its serial path manually.
2. Check the read-only preview. The Arm102 / 102-LD model, seven angles, sample rate, and delay appear while the follower holds its current pose. Drag to rotate; use the wheel or a two-finger gesture to zoom. “Reset view” changes only the displayed view.
3. Unlock and calibrate leader zero. Support the leader, click “Unlock leader”, place it at the Arm102 calibration pose, and close its gripper. Check “Leader is at reference zero with gripper closed”, then click “Confirm and set leader zero”. Move each axis slightly and verify the J1–J6 directions and J7 finger-ring opening against the preview.
4. Set initial parameters. Set target speed to 0.1 rad/s, clear “Follow gripper”, and leave “Absolute follow” unchecked. The current hardware Leader speed range is 0.05–1.0 rad/s, independent of the ordinary web joint speed setting.
5. Start relative following. Click “Start following”. Relative mode establishes offsets from the two arms' starting poses; the follower should not jump to the leader's absolute pose on the first frame. Start with small J6 movements, then check J1–J5 directions, feedback, and limits individually.
6. Test pause and resume. Move the leader after clicking “Pause and hold”; the follower should hold. In relative mode, “Resume” rebases the arms at their current poses rather than catching up with movements made during the pause.
7. Release control before testing the gripper. Click “Stop and release control” and confirm that ordinary sliders work again. After validating all six axes, start another session with gripper following enabled and check its opening direction and endpoints with small movements.

Leader calibration and follower zero

“Confirm and set leader zero” calibrates the Arm102 Leader. The RS set_zero service rewrites follower motor zero points. Do not use follower zeroing to correct the leader preview or following offset. Absolute following requires the arms to be aligned first; the controller rejects excessive pose differences. Keep the default relative mode for initial use.

Pause, Stop, and Connection Recovery

• Pause and hold: the follower holds its current pose while the session retains control. Relative mode rebases when resumed.
• Stop and release control: following ends and the follower holds its current pose. Sliders, trajectories, gravity compensation, safe homing, and disable become available after release. Neither pause nor stop directly removes motor torque.
• Stale samples, leader USB removal, or browser heartbeat timeout: the controller stops advancing targets and holds; the UI reports a fault or stale data. Reconnection does not resume following automatically. Check the connection, recalibrate, and explicitly start again.
• Another page owns the session: stop and release control on the original page. If it closed unexpectedly, wait for the heartbeat timeout to release the session.

Initial hardware acceptance should include USB removal, browser disconnection, leader movement while paused, and slider recovery after stopping. The project includes mock and control-policy tests; the real Leader → RS hardware link still requires on-site acceptance.

Following Rate and Speed Limits

Leader UART sampling targets 60 Hz; the actual rate depends on serial read time. Each valid new sample updates the target. The hardware MIT loop follows the latest target at 125 Hz with speed limits and cached gravity feedforward, without adding the ordinary web target's second-order smoothing or jerk filter. The follower can still lag when the leader moves faster than the selected speed limit.

Leader following uses separate leader_mit_kp / leader_mit_kd gains and v_des=0 for the six axes. Pause or stop restores ordinary hold gains. The gripper reference speed is min(6 × teleoperation speed, leader_gripper_velocity_limit), currently capped at 3 rad/s by configuration. Joint limits, feedback timeouts, and the session watchdog remain active. See “Key Control Parameters” below for the configuration location.

Inspect Leader Status

source scripts/rs_env.sh
ros2 topic echo /rebot_arm_rs/leader/status --once

For simulation, use /rebot_arm_rs_sim/leader/status. Check state, calibrated, following, paused, sample_hz, sample_age, and angles_deg. Displayed leader angles are in degrees; targets_rad contains the six follower joint targets in radians.

Unified Command Line

Command	Description
./rebotarm doctor	Read-only environment check, equivalent to ./setup.sh --check
./rebotarm start web	Start or reuse rosbridge and start the web server
./rebotarm start rs_sim	Start the full RS MuJoCo simulation stack
./rebotarm start rs	Start the RS real-robot controller; the hardware confirmation variable is required
./rebotarm status	View registered processes, ports, and can0
./rebotarm stop	Stop the web/rosbridge child processes registered by start web

Simulation and real-robot processes running in the foreground should be ended normally with Ctrl+C in their own terminals.

Project Architecture

ReBot_Arm_DigitalTwin_RS/
|-- setup.sh                              One-click check, install, and build
|-- rebotarm                              Unified start, status, and stop entry
|-- requirements-rs-hardware.txt          RS real-robot Python dependencies
|-- requirements-rs-mujoco.txt            MuJoCo/Agent Python dependencies
|-- scripts/
|   |-- rs_env.sh                         Load ROS, venv, and workspace
|   |-- setup_rs_workspace.sh             Prepare and build the ROS 2 workspace
|   |-- start_rs_sim.sh                   Full RS simulation stack
|   |-- start_rs_hardware.sh              RS real-robot controller
|   |-- start_rs_dual.sh                  Real-robot/Fake dual-namespace startup
|   `-- start_rs_text_agent.sh            Text Agent HTTP service
|-- rebotarm_ros2_RS/
|   |-- src/rebotarmcontroller/           Real-robot controller and Fake Driver
|   |-- src/rebotarm_msgs/                Custom msg/srv/action
|   |-- src/rebotarm_bringup/             Config, URDF, meshes, and launch
|   |-- src/rebotarm_mujoco_rs/           RS MuJoCo, camera, detection, and tasks
|   |-- src/rebotarm_agent/               MCP Server and Text Agent
|   |-- src/rebotarm_moveit_config/       MoveIt configuration
|   |-- src/rebotarm_moveit_demos/        Trajectory and picking/placing examples
|   `-- third_party/reBotArm_control_py/  Plain-source control SDK
`-- reBotArm_simulator-RS/
    |-- server.js                           Static server and Agent proxy
    |-- public/index.html                   Web entry point
    |-- public/js/rebot-sim.js              Three.js, IK, scene, and animation
    |-- public/js/ros/rebot-ros-client.js   rosbridge client
    |-- public/js/ros/rebot-ros-ui.js       ROS UI, control, and feedback logic
    |-- public/js/rebot-llm.js              LLM chat UI
    |-- description/                         Standalone fallback model for the web
    `-- split_meshes/grouped_gripper/        Optional web gripper meshes

System Data Flow

Web Commands to RS Real Robot

User drags a joint slider or TCP handle
  -> Three.js / browser DLS IK
  -> 30 ms slider damping, 1 deg input dead zone, max 60 Hz throttle
  -> rosbridge WebSocket
  -> JointMitCmd ROS topic
  -> command state arbitration
  -> 125 Hz velocity/acceleration/jerk-limited online reference
  -> MIT q/dq/kp/kd/tau
  -> reBotArm_control_py
  -> MotorBridge + SocketCAN can0
  -> RobStride motors

A new web target only updates the endpoint of the online trajectory. Even if the browser temporarily stops sending the next frame, the 125 Hz control loop continues generating a continuous reference. The command QoS depth is 1, so a new target overwrites the old unprocessed target and avoids replaying a backlog of positions after dragging stops.

RS Real-Robot Feedback to Web

RobStride encoder/status
  -> SocketCAN can0
  -> SDK status cache
  -> 20 Hz real-robot sync query
  -> 60 Hz ROS JointState publication
  -> rosbridge subscription throttle
  -> adaptive low-pass + feedback dead zone
  -> 32-120 ms display interpolation
  -> Three.js solid feedback model

The translucent model represents the control target. The target shadow disappears automatically when the joint and gripper errors are small enough. Display filtering only affects the web picture and does not change the targets sent to the motors.

MuJoCo Simulation Chain

Web console or Agent
  -> ROS 2 commands, services, or actions
  -> Fake RS Driver (100 Hz)
  -> MuJoCo Sync (250 Hz)
  -> RS MJCF dynamics and collision
  -> joint/object states, overhead camera, and color detection
  -> Web console and MCP Agent

Web Control Notes

Control Target and Namespace

Mode	Namespace	Web model source
RS real robot	/rebot_arm_rs	Real-robot /joint_states feedback
RS simulation	/rebot_arm_rs_sim	Uses the actual MuJoCo state first

After switching modes, the web console recreates the rosbridge client and interface paths according to the target namespace. Simulation mirrors the actual MuJoCo state by default; hardware mode shows measured feedback and a translucent target model.

Use these read-only commands to check the available targets:

source scripts/rs_env.sh
ros2 topic list | grep /rebot_arm_rs
ros2 service list | grep /rebot_arm_rs

Check the complete prefix: /rebot_arm_rs is hardware and /rebot_arm_rs_sim is simulation. The presence of rs alone does not identify the target.

Joints and Gripper

• J1-J6 are controlled in radians, with limits consistent with the RS URDF.
• The real-robot web joint speed limit ranges from 0.05–1.50 rad/s; start at 0.2–0.4 rad/s. This panel is hidden in simulation. Leader teleoperation has a separate speed setting; begin at 0.1 rad/s.
• J1-J6 use 30 ms slider damping and a 1 deg input dead zone by default; the final position is committed forcefully when you release the slider.
• J7/the gripper is shown as opening width on the web, with a range of 0-71.5 mm.
• The gripper is converted to the RS motor range 0-5 rad when publishing, and ROS feedback is converted back to millimeters.
• J7 does not go through the J1-J6 radian damping or input dead zone; only the latest target is sent per browser render frame.

TCP Dragging and IK

The web console uses damped least squares (DLS) to solve TCP targets. The RS version adapts the damping according to the singularity level, then sends the joint solution through the same MIT online control chain.

The web Three.js scene uses Y-up and ROS uses Z-up. The coordinate conversion logic is in rebot-sim.js; when extending pose features, reuse the existing conversion instead of exchanging form values directly.

Pose input uses meters: X is forward, Y is left, and Z is up. If a target is unreachable, first increase Z or reduce the horizontal distance, then check orientation constraints and joint limits.

Teaching and Trajectory Playback

The web console can record joint trajectories and call:

/<namespace>/follow_joint_trajectory

The RS real-robot action server uses continuous monotone cubic Hermite position/velocity references and automatically extends trajectory segments that are too short, keeping ordinary real-robot trajectory speeds below about 0.60 rad/s. Callers must wait for the action result or real-robot feedback and should not finish the playback animation early according to the original request duration.

Enable, Safe Homing, and Disable

• The real-robot controller needs to be enabled after every startup.
• safe_home smoothly returns to zero and validates angle and velocity.
• Clicking disable from a non-zero pose enters SAFE_HOMING first.
• If homing validation fails, the motors stay enabled and a failure result is returned.
• set_zero rewrites the motor zero point and is not an ordinary homing button. Only call it when the mechanical structure is actually at the calibrated pose.

Gravity Compensation

RS gravity compensation starts from the measured pose and smoothly transitions gains. Repeated starts are idempotent; stopping holds the last measured position. The gripper defaults to low-resistance movement assistance and returns to position hold when compensation stops. Remove held objects and keep fingers clear before starting; stop compensation before trajectories, grasping, or Leader following.

Web joint, TCP, trajectory, and gripper commands are rejected during gravity compensation. Related services:

/<namespace>/gravity_compensation/start
/<namespace>/gravity_compensation/stop
/<namespace>/gravity_compensation/status

ROS 2 Interfaces

The examples below use the real-robot /rebot_arm_rs. For simulation, replace the prefix with /rebot_arm_rs_sim.

Status Topics

Topic	Type	Description
/rebot_arm_rs/joint_states	sensor_msgs/msg/JointState	Unified six-axis and gripper status
/rebot_arm_rs/joints/<name>/state	rebotarm_msgs/msg/JointMotorState	Single-motor position, velocity, torque, and status code
/rebot_arm_rs/gripper/state	rebotarm_msgs/msg/JointMotorState	Raw gripper motor status
/rebot_arm_rs/control_target	sensor_msgs/msg/JointState	Final target from the web console or action layer
/rebot_arm_rs/control_reference	sensor_msgs/msg/JointState	ROS sample of the 125 Hz online trajectory reference; effort carries the reference acceleration
/rebot_arm_rs/arm_status	rebotarm_msgs/msg/ArmStatus	Enable state, mode, state machine, and error codes
/rebot_arm_rs/leader/status	rebotarm_msgs/msg/LeaderStatus	Leader connection, calibration, session, angles, and sample status

Low-Level Command Topics

Topic	Type	Description
/rebot_arm_rs/joints/<joint1..joint6>/cmd/mit	rebotarm_msgs/msg/JointMitCmd	Default joint command from the web console for the real robot
/rebot_arm_rs/joints/<joint1..joint6>/cmd/pos_vel	rebotarm_msgs/msg/JointPosVelCmd	Position-velocity command
/rebot_arm_rs/gripper/cmd/mit	rebotarm_msgs/msg/JointMitCmd	RS gripper MIT command
/rebot_arm_rs/gripper/cmd/pos_vel	rebotarm_msgs/msg/JointPosVelCmd	Gripper position-velocity command
/rebot_arm_rs/mujoco/target_pose	geometry_msgs/msg/PoseStamped	TCP target pose

Before publishing low-level commands yourself, check arm_status.state_machine. Do not preempt control during GRAVITY_COMP, SAFE_HOMING, trajectory execution, or an active Leader session.

Services

Service	Type	Description
/rebot_arm_rs/leader/control	rebotarm_msgs/srv/LeaderControl	Scan, connect, unlock, calibrate, follow, and manage sessions
/rebot_arm_rs/enable	std_srvs/srv/Trigger	Enable the arm
/rebot_arm_rs/disable	std_srvs/srv/Trigger	Safely disable; homes first when necessary
/rebot_arm_rs/safe_home	std_srvs/srv/Trigger	Safe homing with validation
/rebot_arm_rs/set_zero	rebotarm_msgs/srv/SetZero	Write the motor zero point; use with caution
/rebot_arm_rs/gravity_compensation/start	std_srvs/srv/Trigger	Start gravity compensation
/rebot_arm_rs/gravity_compensation/stop	std_srvs/srv/Trigger	Stop gravity compensation
/rebot_arm_rs/gravity_compensation/status	std_srvs/srv/Trigger	Query gravity compensation status
/rebot_arm_rs/move_to_pose_ik	rebotarm_msgs/srv/MoveToPoseIK	Pose IK solving
/rebot_arm_rs/gripper/set	rebotarm_msgs/srv/SetGripper	Gripper position control
/rebot_arm_rs/gripper/open	rebotarm_msgs/srv/GripperCommand	Open the gripper
/rebot_arm_rs/gripper/close	rebotarm_msgs/srv/GripperCommand	Close the gripper

Example:

source scripts/rs_env.sh
ros2 service call /rebot_arm_rs/gravity_compensation/status std_srvs/srv/Trigger '{}'
ros2 service call /rebot_arm_rs/safe_home std_srvs/srv/Trigger '{}'

Actions

Action	Type	Description
/rebot_arm_rs/follow_joint_trajectory	control_msgs/action/FollowJointTrajectory	Joint trajectory
/rebot_arm_rs/gripper/command	control_msgs/action/GripperCommand	Gripper action
/rebot_arm_rs/move_to_pose	rebotarm_msgs/action/MoveToPose	Cartesian pose motion

MuJoCo and Vision Interfaces

Interface	Type/Use
/rebot_arm_rs_sim/mujoco/joint_states	Actual MuJoCo joint states
/rebot_arm_rs_sim/mujoco/object_states	Scene object states as JSON
/rebot_arm_rs_sim/mujoco/overhead_rgb/image_raw	Overhead RGB camera
/rebot_arm_rs_sim/vision/color_blocks/detections	Color object detection JSON
/rebot_arm_rs_sim/mujoco/reset	Reset the simulation
/rebot_arm_rs_sim/mujoco/record/start	Start recording
/rebot_arm_rs_sim/mujoco/record/stop	Stop recording
/rebot_arm_rs_sim/mujoco/record/replay	Replay a recording
/rebot_arm_rs_sim/mujoco/record/clear	Clear recordings

Visual Grasping

The full simulation includes a red cube, a blue cuboid, and a yellow cylinder by default. The web console can select a target color or choose automatically, then perform alignment, pre-grasp, descending, closing, lifting, and placing.

Grasp success is not only about trajectory completion. The system also checks whether the MuJoCo object was actually lifted, preventing a false success report when the gripper closes empty or the trajectory completes without picking up the object.

Troubleshoot visual grasping:

source scripts/rs_env.sh
ros2 topic echo /rebot_arm_rs_sim/vision/color_blocks/detections --once
ros2 topic echo /rebot_arm_rs_sim/mujoco/object_states --once
ros2 service list | grep /rebot_arm_rs_sim/

If the gripper closes but does not lift the object:

• confirm that physics mode is used;
• check whether the camera and color detection are publishing continuously;
• check the target width, orientation, and grasping height;
• review the contact stability and physical lift validation results in the logs;
• avoid starting multiple grasping or IK requests at the same time.

DeepSeek / MCP Text Control

Roles of the Three Services

Web rebot-llm.js
  -> Node.js /api/llm/chat
  -> Text Agent (:8082, default DeepSeek deepseek-flash)
  -> MCP Server (:8081/mcp, bound to a target namespace)
  -> ROS 2 service/action/topic

The web server, MCP Server, and Text Agent are separate processes. Full rs_sim starts the MCP Server by default. “Connect AI assistant” only checks and connects to the Text Agent; it cannot start the Python service on Ubuntu. MCP clients can call the tools directly without an LLM.

Start the Default DeepSeek Agent

Start full simulation and the web console, then run in a third terminal:

cd ~/ReBot_Arm_DigitalTwin_RS
export DEEPSEEK_API_KEY='your-key'
./scripts/start_rs_text_agent.sh

Current source defaults:

Setting	Default / usage
LLM endpoint	https://api.deepseek.com
Model	deepseek-flash
DeepSeek key	DEEPSEEK_API_KEY
MCP tools endpoint	http://127.0.0.1:8081/mcp
Text Agent / Dashboard	http://localhost:8082
Default target namespace	/rebot_arm_rs_sim

Previously exported REBOTARM_LLM_BASE_URL, REBOTARM_LLM_MODEL, or REBOTARM_LLM_API_KEY values override the defaults. Clear old values or explicitly set the intended configuration before using default DeepSeek. Keep real keys in server-side environment variables or uncommitted local configuration.

After [text-agent-http] listening on http://0.0.0.0:8082/ appears, check:

curl http://127.0.0.1:8082/health

Expected response:

{"ok": true, "service": "rebotarm-text-agent"}

This confirms HTTP reachability. Click “Connect AI assistant” and send a read-only query to verify model authentication and the MCP link. Then try motion commands in simulation:

Query the arm status
Detect the colored blocks on the table
Open the gripper
Move to X=0.30 Y=0 Z=0.30
Grasp the red block
Return safely to zero

MCP pick_color holds the object after a successful grasp. The web “Grasp into storage” button runs a separate workflow with different placement behavior.

Change the Model or Backend Address

To use another OpenAI-compatible service, set all three values in the Text Agent terminal and restart it:

export REBOTARM_LLM_BASE_URL='https://<provider-host>/v1'
export REBOTARM_LLM_MODEL='<model-id>'
export REBOTARM_LLM_API_KEY='<api-key>'
./scripts/start_rs_text_agent.sh

Replace the placeholders with values from your provider. REBOTARM_MCP_URL selects the MCP endpoint used by the Text Agent.

If the web server and Agent run on different machines, update REBOTARM_TEXT_AGENT_URL and REBOTARM_MCP_URL in reBotArm_simulator-RS/.env to the backend addresses, then restart the web server. If only the browser is remote and all three services remain on the same Ubuntu host, keep the proxy addresses local.

Check MCP Namespace and Motion Permissions

The simulation script starts MCP with arm_namespace:=rebot_arm_rs_sim motion_mode:=allow. A manually launched MCP Server defaults to motion_mode:=locked. Switching the web target does not restart MCP or change its namespace. When a startup script reuses an existing service on 8081, check that service's target too.

For real-robot inspection, stop the previous MCP service and launch it separately with motion locked:

source scripts/rs_env.sh
ros2 launch rebotarm_agent rebotarm_mcp.launch.py \
  arm_namespace:=rebot_arm_rs motion_mode:=locked

The Text Agent selects its tools service by MCP URL; the actual target namespace comes from that MCP service's launch arguments. Restart the Text Agent against the newly launched service:

REBOTARM_MCP_URL=http://127.0.0.1:8081/mcp ./scripts/start_rs_text_agent.sh

Verify read-only status first, then follow the backend motion-permission workflow. Natural-language control cannot bypass the safety state machine. Stop and release an active Leader session before calling Agent motion tools.

Key Control Parameters

The real-robot default configuration is located at:

rebotarm_ros2_RS/src/rebotarm_bringup/config/rebotarm_hardware.yaml

Current RS key parameters:

rate: 125
control:
  arm_control_mode: mit
  mit_kp: [80, 150, 150, 50, 50, 50]
  mit_kd: [5, 10, 10, 5, 4, 4]
  leader_mit_kp: [50, 50, 50, 30, 50, 50]
  leader_mit_kd: [3, 5, 5, 3, 4, 4]
  leader_gripper_velocity_limit: 3.0
  stream_acceleration_limit: 4.0
  stream_jerk_limit: 30.0
  stream_natural_frequency: 8.0
gravity_compensation:
  kp: 1.5
  kd: 1.0

Other default frequencies:

Layer	Default frequency	Description
Web joint target	Up to 60 Hz	Sliders and TCP IK continuously update the target
Real-robot sync feedback query	20 Hz	Refresh the RobStride measurement cache
ROS real-robot state	60 Hz	Publish feedback, target, and reference from the cache
RS MIT control	125 Hz	Online smoothing and motor commands
Fake Driver	100 Hz	Simulation control and state
MuJoCo Sync	250 Hz	Default dynamics synchronization
MuJoCo object state	30 Hz	Scene and Agent
Overhead camera	8 Hz	RGB image
Color detection	10 Hz	Detection results

Before adjusting gains, record all of the following at the same time:

/rebot_arm_rs/control_target
/rebot_arm_rs/control_reference
/rebot_arm_rs/joint_states

If the target is discontinuous, check the web input layer; if the reference is discontinuous, check the online trajectory; if the reference is continuous but feedback overshoots, check MIT gains, load, friction, current, and temperature rise; only check feedback filtering and rosbridge arrival intervals when the web animation alone is jittery.

Secondary Development Guide

Modify Web Joint Limits or Pose Presets

File:

reBotArm_simulator-RS/public/js/rebot-sim.js

Joint definitions are in jointDefs and presets are in presets. When modifying joint limits, verify the RS URDF, SDK configuration, and real mechanical limits together; do not only change the display range on the web.

Modify Web Command Frequency, Filtering, and Damping

File:

reBotArm_simulator-RS/public/js/ros/rebot-ros-ui.js

This file handles web command throttling, real-robot feedback filtering, the feedback dead zone, interpolation, the target shadow, control locks, and vision logic. Do not compensate for web animation latency by increasing the real-robot MIT gains.

Add Custom ROS Interfaces

The client wrapper is located at:

reBotArm_simulator-RS/public/js/ros/rebot-ros-client.js

Add Topic, Service, or Action paths in the UI layer and call the client's subscribe(), callService(), or sendActionGoal(). Interfaces should always be generated from the current namespace to avoid sending simulation commands to the real robot.

Modify the RS URDF or Mesh

ROS 2 primary models:

rebotarm_ros2_RS/src/rebotarm_bringup/description/urdf/ReBot_Arm_RS.urdf
rebotarm_ros2_RS/src/rebotarm_bringup/description/meshes_rs/

The web server reads the models above first. If the web directory is copied and run alone, it uses:

reBotArm_simulator-RS/description/urdf/ReBot_Arm_RS.urdf
reBotArm_simulator-RS/description/meshes_rs/

Keep both copies in sync when modifying them and check Linux filename case.

Modify MuJoCo

File/directory	Role
rebotarm_ros2_RS/src/rebotarm_mujoco_rs/models/	RS MJCF and STL
rebotarm_mujoco_rs/mujoco_sync.py	Sync frequency, dynamics, and PD
rebotarm_mujoco_rs/scene_camera.py	Overhead camera
rebotarm_mujoco_rs/scene_detector.py	Color detection
rebotarm_mujoco_rs/task_server.py	Tasks, recording, and replay
rebotarm_mujoco_rs/launch/mujoco_rs.launch.py	Launch parameters

Modify Leader Mapping and Control

File	Purpose
reBotArm_simulator-RS/public/js/ros/rebot-leader-ui.js	Namespaced service calls, status subscription, and browser heartbeat
reBotArm_simulator-RS/public/js/ros/leader-model.js	Arm102 model display and angle mapping
rebotarm_ros2_RS/src/rebotarmcontroller/rebotarmcontroller/leader_worker.py	UART sampling and calibration in a separate process
rebotarm_ros2_RS/src/rebotarmcontroller/rebotarmcontroller/leader_teleop.py	ROS services, status, and leader/follower mapping integration
rebotarm_ros2_RS/src/rebotarmcontroller/rebotarmcontroller/leader_policy.py	Mapping, sample validity, and session policy
rebotarm_ros2_RS/src/rebotarmcontroller/rebotarmcontroller/hardware_manager.py	Hardware ownership, MIT following, speed limits, and hold

Leader model assets live in reBotArm_simulator-RS/public/models/leader-arm102/. Model display directions and follower control mapping are maintained separately; do not change hardware direction parameters to fix the preview. Validate mock mode, timeouts, pause/resume, and control release before low-speed, single-axis hardware acceptance.

Extend MCP Tools

The MCP Server is located at:

rebotarm_ros2_RS/src/rebotarm_agent/rebotarm_agent/rebotarm_mcp_server.py

When adding a tool:

1. define a clear input schema and units;
2. separate read-only tools from motion tools;
3. validate namespace, state, and motion permission in motion tools;
4. call the backend through ROS 2 Service/Action/Topic instead of bypassing the controller directly;
5. rebuild the workspace and test error, timeout, and cancellation paths in simulation.

Verification Commands

source scripts/rs_env.sh
python3 -m pytest rebotarm_ros2_RS/src/rebotarmcontroller/test -q

bash -n setup.sh rebotarm scripts/*.sh

node --check reBotArm_simulator-RS/server.js
node --check reBotArm_simulator-RS/public/js/rebot-sim.js
node --check reBotArm_simulator-RS/public/js/ros/rebot-ros-client.js
node --check reBotArm_simulator-RS/public/js/ros/rebot-ros-ui.js
node --check reBotArm_simulator-RS/public/js/rebot-llm.js
node --check reBotArm_simulator-RS/public/js/ros/rebot-leader-ui.js
node --check reBotArm_simulator-RS/public/js/ros/leader-model.js
node reBotArm_simulator-RS/scripts/ros-namespaces.test.js

Runtime checks:

./rebotarm status
ros2 node list
ros2 topic list | grep /rebot_arm_rs
ros2 service list | grep /rebot_arm_rs
ros2 action list | grep /rebot_arm_rs

FAQ

1. The web page stays on the model loading screen

Open the browser developer tools Network panel and check whether these requests return 200:

/api/urdf
/api/description/meshes/*.STL

Common causes:

• the RS URDF or STL does not exist;
• the web directory was moved alone and the fallback model is incomplete;
• the package:// mapping in the URDF is incorrect;
• Linux filename case does not match;
• the load order of Three.js, STLLoader, or URDFLoader is wrong.

2. ROS shows "disconnected"

ss -ltnp | grep 9090
ros2 node list
./rebotarm status

Confirm that rosbridge listens on 0.0.0.0:9090 and that the WebSocket address starts with ws://. Remote browsers should enter the Ubuntu IP, not the browser computer's localhost.

3. The web console connects, but sliders cannot control the real robot

Check in order:

1. The web console is on the RS real robot /rebot_arm_rs.
2. The ROS WebSocket is connected.
3. The web control lock is enabled.
4. The arm is enabled.
5. arm_status is not in GRAVITY_COMP, TRAJ_RUNNING, or SAFE_HOMING.
6. There are no CAN or motor errors in the real-robot controller.

4. The web model jitters or the target shadow persists

• Confirm that only one controller and one state publisher exist for the same namespace.
• Check whether /joint_states has multiple publishers.
• Record control_target, control_reference, and joint_states at the same time.
• Check CAN error counters and feedback arrival intervals.
• Do not run trajectory playback and manual continuous dragging at the same time.

A long-lived target shadow usually means actual feedback has not reached the target; this is not necessarily a web rendering problem.

5. No CAN feedback, or BUS-OFF

ip -details -statistics link show can0
candump can0

Check:

• whether the baud rate is 1 Mbps;
• whether CAN_H/CAN_L are reversed;
• termination resistors at both ends;
• motor power and common ground;
• motor IDs and SDK configuration;
• USB-CAN driver and interface name.

Stop the real-robot test immediately if the error counter keeps increasing.

6. The arm does not power off immediately after clicking disable

If the arm is not near the zero pose, the controller performs safe homing first and validates it. This is expected behavior. If homing fails, the system keeps the arm enabled to prevent falling from a non-zero pose. Use the physical emergency stop in an emergency.

7. Gravity compensation fails to start or cannot control

• Confirm that the arm is enabled.
• Query /gravity_compensation/status.
• Confirm that no trajectory or safe homing is running.
• Other motion commands are rejected during gravity compensation.
• Support any load that may fall when switching modes.

8. Visual grasping does not work

Confirm that the full rs_sim is running in physics mode, then check:

ros2 topic echo /rebot_arm_rs_sim/mujoco/overhead_rgb/image_raw --once
ros2 topic echo /rebot_arm_rs_sim/vision/color_blocks/detections --once
ros2 topic echo /rebot_arm_rs_sim/mujoco/object_states --once

If the button shows that a task is queued, wait for the current serial action to finish and do not start multiple grasping requests repeatedly.

9. The AI Assistant Cannot Connect or Reports a Missing Key

Set DEEPSEEK_API_KEY in the third terminal and start the Text Agent, then check:

ss -ltnp | grep 8082
curl http://127.0.0.1:8082/health
ss -ltnp | grep 8081

ECONNREFUSED 127.0.0.1:8082 means the web server cannot reach the Text Agent. Check REBOTARM_TEXT_AGENT_URL in reBotArm_simulator-RS/.env; use the Agent host address for a deployment across machines and restart the web server. If health checks pass but chat fails, inspect the key, old REBOTARM_LLM_* overrides, model, and MCP logs. The web /api/llm/health check does not replace an actual model call.

10. The page still shows an old version after modifying the front end

The RS web console includes Service Worker/PWA support. First use Ctrl+Shift+R for a hard refresh; if it still does not update, clear site data or unregister the Service Worker in browser developer tools, then reload the page.

11. setup.sh or colcon build fails

• Confirm that the Ubuntu, ROS 2, and Python versions match.
• Confirm that rosdep is initialized and can update.
• Run ./setup.sh --check to see the missing items.
• Confirm that numpy, scipy, mujoco, pinocchio, motorbridge, and fastmcp can be imported in rebotarm_ros2_RS/.venv.
• Back up user files before deleting or resetting them, and do not solve build problems with destructive commands.

12. Leader Scanning, Calibration, or Following Fails

Symptom	Check
Waiting for ROS leader service	Rebuild rebotarm_msgs / rebotarmcontroller, restart the controller and rosbridge, check the namespace, and hard-refresh the page
No serial port / probe timed out	USB connection to the ROS host or VM passthrough, power, dialout permissions, and other processes occupying the port
Missing servo IDs	All seven IDs 0–6 should respond; check wiring, power, and 1 Mbps baud rate
uncalibrated / samples stale	Reconnect for preview, unlock and calibrate again; inspect Hz and delay rather than relaxing timeouts
follower is busy	Stop trajectories, hand following, and gravity compensation; restore enabled IDLE and gripper position hold
absolute poses differ	Use default relative mode first; absolute mode requires aligned poses
browser heartbeat timed out	Check page closure, network loss, or background throttling; recalibrate and explicitly start again
Leader model fails to load	Check local public/models/leader-arm102/urdf/leader.urdf and meshes/ assets, then hard-refresh

Quick Command Table

# Check and install
./setup.sh --check
./setup.sh --yes
./rebotarm doctor

# Full simulation
./rebotarm start rs_sim
./rebotarm start web

# Natural-language Agent
export DEEPSEEK_API_KEY='your-key'
./scripts/start_rs_text_agent.sh

# Optional Leader dependencies and interfaces (stop old controller and rosbridge first)
bash scripts/setup_rs_leader.sh

# RS real robot
sudo ip link set can0 type can bitrate 1000000
sudo ip link set can0 up
REBOTARM_RS_HARDWARE_CONFIRM=I_UNDERSTAND_RS_WILL_MOVE ./rebotarm start rs

# Status and stop
./rebotarm status
./rebotarm stop

References

• ROS 2 Jazzy documentation: https://docs.ros.org/en/jazzy/
• rosbridge_suite: https://github.com/RobotWebTools/rosbridge_suite
• Three.js: https://threejs.org/docs/
• URDFLoader: https://github.com/gkjohnson/urdf-loaders
• MuJoCo: https://mujoco.readthedocs.io/
• Model Context Protocol: https://modelcontextprotocol.io/
• Current B601-RS project
• B601-RS project documentation: README.md, USER_MANUAL_ZH.md, DEVELOPER_GUIDE_ZH.md, DATA_FLOW_RS_ZH.md, LEADER_TELEOP_TEST_ZH.md