Skip to main content

reBot Arm B601-RS Visual Grasping Demo

reBot Arm B601-RS
Get One Now

License: MITPython VersionPlatformCameraYOLO

Depth Perception · Object Detection · Hand-Eye Calibration · Autonomous Grasping · Fully Open Source

reBot Arm B601-RS visual grasping demo

How do the two grasping methods differ?​

This page covers two visual grasping demos with different implementations. Compare their frameworks, calibration requirements, and startup flows, then choose a tutorial below.

Method 01

Python SDK + YOLO / OBB

Use YOLO, an RGB-D depth camera, and the Python SDK for the complete workflow from environment setup and camera integration to hand-eye calibration and grasp debugging.

Grasping logic
Estimate grasp poses from detection boxes or OBB minimum-area rectangles; optionally use GraspNet for 6D pose estimation.
Hand-eye calibration
Complete Eye-in-Hand calibration with an ArUco marker before the first full grasping run.
Startup
Run Python scripts to verify detection, pose estimation, and robotic grasping step by step.
Method 02

ROS 2 + YOLOE

Use ROS 2 and YOLOE to start the arm, Gemini 2 / D405 / D435i camera, and grasping nodes in separate terminals for picking and placing.

Grasping logic
The visual grasping node detects, grasps, and places objects. Text class names specify the grasp and place targets.
Hand-eye calibration
No calibration board is required; adjust offset parameters to compensate for small mounting and printed-part tolerances.
Startup
Start the arm, camera, and grasping node in separate terminals, then trigger picking and placing.

Choose a grasping method​

Switch tabs to view the tutorial for your chosen method and follow its installation and running steps.

Visual Grasping Method 1​

This route uses Python SDK + RGB-D camera + YOLO / OBB without a ROS 2 workspace. The default route suits ordinary tabletop objects; enable the optional GraspNet module below when you need more complex 6D grasping.

1. Project Features​

  1. Direct grasp pose estimation from YOLO + OBB The pipeline uses detection boxes or OBB minimum-area rectangles directly and takes the short axis as the gripper opening direction, avoiding complex 3D point-cloud processing.
  2. GraspNet-Baseline 6D grasp pose estimation (optional) The project also supports GraspNet-Baseline (graspnet/graspnet-baseline) for 6D grasp pose estimation from RGB-D point clouds, with YOLO bounding boxes used to select target candidates for more complex grasping experiments.
  3. Robotic arm and gripper driver integration The main grasping script is based on the robotic arm and end-pose controller from reBotArm_control_py, with a lightweight grasp helper for gripper opening, force-controlled grasping, and TCP pose reading.
  4. Open Source and Extensible All source code is open, and users can customize control algorithms and effects based on their own needs.

2. Specifications​

The hardware for this tutorial is provided by Seeed Studio

ParameterSpecification
Robot Arm ModelreBot Arm B601-RS
Degrees of Freedom6-DOF + Gripper
Camera ModelOrbbec Gemini 2 / RealSense D435i / D405
Detection MethodYOLO + OBB Minimum-Area Rectangle
Communication MethodCAN Bus via USB2CAN adapter; USB 3.0 camera connection
Operating Voltage48V DC
Host PlatformUbuntu 22.04+ PC
Recommended Python VersionPython 3.10

3. Bill of Materials (BOM)​

ComponentQuantityIncluded
reBot Arm B601-RS Robotic Arm1✅
Gripper1✅
USB2CAN Serial Bridge1✅
Power Adapter (48V)1✅
USB-C / Communication Cable1✅
RGB-D Depth Camera1✅
Camera Connector / Mounting Bracket1✅

Wiring​

  1. Connect the depth camera to the host via USB 3.0.
  2. Connect the USB2CAN adapter to the arm CAN bus.
  3. Make sure the 48V power supply, camera, and robotic arm are all connected securely.
  4. If you encounter permission issues, run:
sudo chmod a+rw /dev/bus/usb/*/*
sudo chmod 666 /dev/ttyUSB0

4. Environment Requirements​

ItemRequirement
Operating SystemUbuntu 22.04+
Python3.10

5. Install the Basic Environment​

Follow these steps in order. Steps 0–3 are required for the default route; choose the camera SDK in step 4 to match your hardware. GraspNet installation and running instructions are in the separate advanced module below.

0

Step 0. Complete the basic robotic arm preparation first​

Before starting this tutorial, please finish the content in reBot Arm B601-RS Quick Start, including robotic arm assembly, zero-point initialization, motor ID configuration, and basic connectivity checks.

1

Step 1. Clone the repository​

Prefer the official Seeed-Projects repository:

git clone https://github.com/Seeed-Projects/reBot-DevArm-Grasp.git rebot_grasp
cd rebot_grasp
2

Step 2. Create the environment and choose PyTorch for your hardware​

Complete these three phases in order: create the base environment, install PyTorch for your hardware, and install the project dependencies.

Create the base environment​
conda env create -f environment.yml -n rebotarm
conda activate rebotarm
Expected result

The terminal prompt shows rebotarm, indicating that the environment is active. PyTorch is not installed yet; continue with the next two phases.

To use a different environment name, replace rebotarm in both commands above. For an existing environment, activate it and run conda env update -n rebotarm -f environment.yml to update the base dependencies. This command does not remove existing pip packages.

Install PyTorch for your hardware​

Both tabs use the same compatible package versions; only the download index differs. The current version file specifies PyTorch 2.9.1, torchvision 0.24.1, and torchaudio 2.9.1 for the Python 3.10, Ubuntu x86_64 environment used in this tutorial.

Use this option for computers with Intel / AMD integrated graphics, or to verify the basic workflow on CPU first.

python -m pip install -r requirements-pytorch.txt --index-url https://download.pytorch.org/whl/cpu
python -c "import torch; print(torch.__version__, torch.__file__); print('CUDA available:', torch.cuda.is_available())"
Expected result

The PyTorch path points inside the active Conda environment. CUDA available: False is expected.

Keep the following setting in config/default.yaml:

yolo:
  device: "cpu"
Supported workflow

The basic YOLO / OBB workflow can run on CPU. Inference speed depends on the model and your computer. This project's GraspNet CUDA operators require an NVIDIA GPU, so skip the GraspNet advanced module below.

Install and verify the project dependencies​

After completing either hardware option, run the following from the project root:

python -m pip install -r requirements.txt -c requirements-pytorch.txt
python -m pip check
python -c "import torch, torchvision, torchaudio, cv2; from ultralytics import YOLO; from calibration.aruco_pose import ArUcoDetector; ArUcoDetector(); print('Vision dependencies OK:', cv2.__version__)"
Expected result

pip check prints No broken requirements found., and the verification command prints Vision dependencies OK without import errors. Complete these checks before installing the robotic arm and camera SDKs.

Installation failures

-c requirements-pytorch.txt constrains the compatible PyTorch package versions. Keep this argument and do not add --upgrade. If pip fails, review the final error and retry the corresponding phase. Activating a Conda environment does not mean that all dependencies were installed successfully.

Install only the standard OpenCV package specified in requirements.txt. The project's ArUco APIs are available in version 4.7; do not install additional OpenCV wheels alongside it. CLIP is installed from the Ultralytics Git repository, so Git must be available on your system. Loading YOLOE for the first time may also download model weights.

3

Step 3. Install the robotic arm SDK​

git clone https://github.com/Seeed-Projects/reBotArm_control_py.git sdk/reBotArm_control_py
cd sdk/reBotArm_control_py
pip install -e .
cd ../..

If running pip install -e . gives 'Multiple top-level packages discovered in a flat-layout', please add an explicit package discovery configuration in reBotArm_control_py's pyproject.toml, and then run pip install -e . again:

[tool.setuptools.packages.find]
include = ["reBotArm_control_py*"]

B601 DM and RS configurations are selected through the SDK configuration files. For B601-RS, confirm the following in sdk/reBotArm_control_py/config/rebotarm.yaml:

hardware_yaml: rebotarm_rs.yaml

The visual grasping programs read this SDK configuration and automatically select the matching arm control mode and gripper parameters.

4

Step 4. Install the depth camera SDK​

This project supports RGB-D depth cameras such as Orbbec Gemini 2 and RealSense D435i / D405. Install the SDK that matches your camera; if your environment can already import the camera driver, you can skip this step.

The Orbbec Gemini 2 depth camera depends on pyorbbecsdk, the Python wrapper for Orbbec SDK v2. Prefer installing the prebuilt Python package first.

On Linux / Ubuntu, configure udev rules before using the camera for the first time, whichever installation method you choose. The rules allow ordinary users to access the USB device and are installed system-wide, so each machine only needs this setup once. Skip the corresponding commands if the rules are already configured.

pip install pyorbbecsdk2

Configure udev rules (Linux / Ubuntu)

pyorbbecsdk2 2.1.0 and later include an environment setup script. Keep your conda environment active and run the script from the installed package. It requests sudo permission and installs and reloads the udev rules automatically.

orbbec_sdk_dir="$(python -c 'import os, pyorbbecsdk; print(os.path.dirname(pyorbbecsdk.__file__))')"
python "$orbbec_sdk_dir/shared/setup_env.py"

If shared/setup_env.py cannot be found, run pip install --upgrade "pyorbbecsdk2>=2.1.0", then repeat the commands above. See the official Orbbec installation guide for details.

If both installation methods fail, refer to the official Orbbec documentation below.

Verify installation

python -c "import pyorbbecsdk; print('pyorbbecsdk OK')"
Expected result

The matching verification command prints pyorbbecsdk OK or pyrealsense2 OK without a ModuleNotFoundError.

SDK Resource Summary

ResourceLink
Gemini 2 Product Pagehttps://www.orbbec.com.cn/index/Product/info.html?cate=38&id=51
Development Resourceshttps://www.orbbec.com.cn/index/Download2025/info.html?cate=121&id=1
Orbbec SDK v2https://github.com/orbbec/OrbbecSDK_v2
SDK v2 API Documentationhttps://orbbec.github.io/docs/OrbbecSDKv2_API_User_Guide/
pyorbbecsdkhttps://github.com/orbbec/pyorbbecsdk
pyorbbecsdk Documentationhttps://orbbec.github.io/pyorbbecsdk/index.html
ROS2 Wrapperhttps://github.com/orbbec/OrbbecSDK_ROS2/tree/v2-main
RealSense SDKhttps://github.com/realsenseai/librealsense

6. Directory Structure​

rebot_grasp/
├── config/
│   ├── default.yaml              # Main configuration file
│   └── calibration/
│       └── <camera_type>/
│           ├── intrinsics.npz    # Camera intrinsics
│           └── hand_eye.npz      # Hand-eye calibration results
├── drivers/
│   ├── camera/
│   │   ├── base.py               # Camera abstract base class
│   │   ├── orbbec_gemini2.py     # Gemini 2 driver
│   │   └── realsense.py          # RealSense driver (alternative)
│   └── robot/
│       └── grasp_driver.py       # Lightweight grasp helper based on arm SDK
├── calibration/
│   ├── aruco_pose.py             # ArUco pose estimation
│   └── hand_eye.py               # Hand-eye calibration solver
├── utils/
│   ├── ordinary_grasp.py         # OBB grasp pose estimation and visualization
│   └── transforms.py             # Coordinate transformation utilities
├── scripts/
│   ├── main.py                   # Main grasping program
│   ├── set.py                    # Grasp and place program
│   ├── ordinary_grasp_pipeline.py
│   ├── object_detection.py
│   └── collect_handeye_eih.py
├── sdk/
│   ├── pyorbbecsdk/              # Orbbec SDK Python wrapper (source installation)
│   └── reBotArm_control_py/      # reBot Arm SDK
├── requirements-pytorch.txt      # Compatible versions installed from the CPU / CUDA index
├── requirements.txt              # Project dependencies installed after PyTorch
└── environment.yml               # Base Conda environment

7. Hand-Eye Calibration​

Before running the full grasping pipeline, complete the Eye-in-Hand hand-eye calibration first.

Before running the calibration script, bring up and verify the CAN interface:

sudo ip link set can0 down 2>/dev/null
sudo ip link set can0 type can bitrate 1000000
sudo ip link set can0 up
ip -details link show can0

Before running it, make sure the following ArUco size parameter in config/default.yaml matches the actual printed marker:

calibration:
  aruco:
    marker_length_m: 0.1

After confirming the configuration, run automatic calibration:

python scripts/collect_handeye_eih.py

In automatic mode, the arm traverses 50 preset poses and records a sample whenever the ArUco marker is detected stably. Even if you interrupt the process with c or q, the script still tries to compute the calibration result from the collected samples.

If you want to move the robotic arm manually during collection, use manual mode:

python scripts/collect_handeye_eih.py --manual

In manual mode, the arm enters gravity-compensation mode. Move the end effector to a proper viewing angle, press Enter to capture, and press c or q to finish and compute the result.

tip

If you find that the robotic arm's grasping accuracy cannot meet your requirements after calibration, you can set the X (front-back), Y (left-right), Z (up-down) parameters in config/default.yaml under calibration.hand_eye_compensation_m to provide positional compensation.

The calibration result is saved to:

config/calibration/<camera_type>/hand_eye.npz

Recommended sample count is at least 5 samples, with 15 or more recommended.

8. Running and Debugging​

Verify the scenarios in this order: the first two do not connect to the robotic arm and help check the camera, detection, and grasp estimation. The next two connect to and control the robotic arm.

1

1. Verify object detection only​

python scripts/object_detection.py

If you need to change the detection model or classes, modify config/default.yaml:

yolo:
  model_name: "yoloe-26l-seg.pt"
  device: "cpu"
  use_world: true
  custom_classes:
    - "yellow banana"
    - "water bottle"
    - "cup"

This step is useful to confirm:

  • The camera opens correctly
  • The YOLO model loads correctly
  • YOLO object detection works as expected
2

2. Verify grasp estimation only​

python scripts/ordinary_grasp_pipeline.py

If you need to adjust the grasp inference frequency or the pre-grasp retreat distance, modify:

grasp_pipeline:
  infer_every_live: 3
  grasp:
    depth_quantile: 0.5
    pregrasp_offset_m: 0.080
    insertion_depth_m: 0.015
    min_base_z_m: 0.00

This script does not connect to the robotic arm. It is only used to verify:

  • Whether the OBB or minimum-area rectangle is reasonable
  • Whether the grasp point lies near the target center area
  • Whether the short-axis direction matches the expected gripper opening direction

Key controls:

  • Left mouse button: inspect depth at the selected pixel
  • G: print the current best grasp pose
  • Q / Esc: exit
3

3. Run the main grasping program​

python scripts/main.py

To validate the target pose while skipping the grasp motion:

python scripts/main.py --dry-run

Verify the pose and workspace with --dry-run before executing a real grasp. In this mode, the current script still initializes the arm, moves to the ready pose, and performs cleanup and homing on exit. For vision-only verification, use the first two scripts, which do not connect to the arm.

Main program flow:

  1. Initialize the RGB-D camera and confirm the image stream is available.
  2. Enable the robotic arm and gripper.
  3. Move to the ready pose. If you want to change the startup ready pose, modify config/default.yaml:
robot:
  ready_pose:
    x: 0.3
    y: 0.0
    z: 0.3
    roll: 0.0
    pitch: 0.7
    duration: 3.0
  1. Detect tabletop targets in real time.
  2. Estimate the grasp pose from the short axis.
  3. Press G to capture the current frame and execute grasping.

Runtime keys:

  • G: grasp the current best target
  • R: resume live preview
  • Q / Esc: exit
4

4. scripts/set.py — Grasp and Place Program​

Function: Grasp the banana and place it in the box.

python scripts/set.py

Completed flow:

  1. Camera and arm initialization, move to ready position
  2. Real-time camera preview + YOLO object detection and instance segmentation
  3. Press G to freeze frame, compute arm target pose via hand-eye transformation
  4. Arm moves to grasp banana and lift
  5. Arm places banana in the box and returns to initial pose
  6. Press Q to exit system, arm returns to zero position

9. GraspNet Advanced Module (Optional)​

The default Python SDK / OBB route does not require GraspNet. Follow these steps only when you need 6D grasp pose estimation from RGB-D point clouds.

1

Step 1. Install GraspNet and CUDA operators​

You do not need GraspNet for scripts/main.py or scripts/ordinary_grasp_pipeline.py. Configure it only when you want to run scripts/graspnet_camera_demo.py or scripts/grasp.py, which require GraspNet, CUDA-enabled PyTorch, the PointNet2/knn CUDA operators, and a pretrained checkpoint.

The GraspNet pointnet2 / knn extensions require a CUDA compiler. Before starting, confirm that nvcc is available in the current environment and compare its CUDA version with the version used to build PyTorch:

nvcc --version
python -c "import torch; print(torch.__version__, torch.version.cuda)"

PyTorch is installed separately in Step 2 of the basic installation. GraspNet requires an NVIDIA GPU and working CUDA support; skip this section if you followed the CPU option. Complete the checks below before configuring the compiler.

Step 1: Confirm your GPU

nvidia-smi

The driver must recognize your NVIDIA GPU. Installing the CUDA Toolkit alone does not enable Intel / AMD integrated graphics to run this project's CUDA operators.

Step 2: Confirm PyTorch CUDA support

python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"

If torch.cuda.is_available() prints True, continue to the next step. If PyTorch is missing, torch.version.cuda is None, or CUDA is unavailable, return to Step 2 of the basic installation and complete the NVIDIA installation and verification. When switching between CPU and CUDA builds, remove the old build before reinstalling.

Step 3: Install the CUDA compiler matching your PyTorch

pointnet2 / knn are local CUDA extensions: the CUDA version reported by nvcc --version must match torch.version.cuda when building, otherwise the build fails with The detected CUDA version (...) mismatches the version that was used to compile PyTorch (...). Check both versions first:

nvcc --version
python -c "import torch; print(torch.version.cuda)"

Take torch.version.cuda showing 12.8 as an example, and install the matching compiler inside the conda environment:

conda install -c nvidia cuda-nvcc=12.8 cuda-cudart-dev=12.8

which nvcc       # should point to $CONDA_PREFIX/bin/nvcc, not /usr/bin/nvcc
nvcc --version   # should match torch.version.cuda

cuda-nvcc provides the compiler and cuda-cudart-dev provides the CUDA headers under targets/x86_64-linux/include; both are required. Once the conda environment is activated, $CONDA_PREFIX/bin precedes the system paths, so the conda nvcc automatically shadows the old system one (for example CUDA 11.5 from the Ubuntu repositories) — there is no need to uninstall the system CUDA.

warning

Do not downgrade PyTorch to match an old system nvcc: old CUDA releases may not support your GPU architecture (CUDA 11.5 supports at most sm_86; RTX 40 series requires CUDA 11.8+ and RTX 50 series requires CUDA 12.8+), and old PyTorch builds may be incompatible with other project dependencies such as recent ultralytics releases.

Step 4: Verify

python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"
nvcc --version

When both CUDA versions match and torch.cuda.is_available() prints True, the PyTorch setup is complete and you can continue building the CUDA operators.

cd sdk
git clone https://github.com/graspnet/graspnet-baseline.git
cd graspnet-baseline

# Install PyTorch for your CUDA version first, then install GraspNet runtime dependencies
pip install open3d tensorboard Pillow tqdm

# Configure CUDA build paths before building the local operators.
# Note: use the conda cuda-toolkit targets directory for headers/libraries;
# do not mix in the pip nvidia/cu13 directory (a version mismatch with conda nvcc causes build errors).
export CUDA_HOME=$CONDA_PREFIX
export TORCH_CUDA_ARCH_LIST="$(python -c 'import torch; major, minor = torch.cuda.get_device_capability(); print(f"{major}.{minor}")')"
export CPATH=$CONDA_PREFIX/targets/x86_64-linux/include:$CPATH
export CPLUS_INCLUDE_PATH=$CONDA_PREFIX/targets/x86_64-linux/include:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=$CONDA_PREFIX/targets/x86_64-linux/lib:$CONDA_PREFIX/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=$CONDA_PREFIX/targets/x86_64-linux/lib:$CONDA_PREFIX/lib:$LD_LIBRARY_PATH

# Build CUDA operators
cd pointnet2
pip install . --no-build-isolation
cd ../knn
pip install . --no-build-isolation
cd ..

# Install GraspNet API
git clone https://github.com/graspnet/graspnetAPI.git
cd graspnetAPI
sed -i "s/'sklearn'/'scikit-learn'/" setup.py
pip install .
cd ../../..
tip

Note: If you follow the official graspnet-baseline repository documentation and use python setup.py install, CUDA / PyTorch related errors may occur. We recommend using pip install . --no-build-isolation so the extension is built against the PyTorch and CUDA configuration already installed in the active conda environment.

tip

If building fails with fatal error: cusparse.h: No such file or directory, run find $CONDA_PREFIX -name cusparse.h and add the directory containing cusparse.h to CPATH / CPLUS_INCLUDE_PATH (with conda cuda-toolkit this is usually $CONDA_PREFIX/targets/x86_64-linux/include). If you get header-related errors such as '__NV_IS_DEVICE' was not declared in this scope, they are usually caused by mixing the pip nvidia/cu13 headers with the conda nvcc version — switch to the conda targets paths above and rebuild.

tip

In addition, older GraspNet API dependencies may still use the deprecated sklearn package name. The sed command replaces it with the currently recommended scikit-learn package name to avoid The 'sklearn' PyPI package is deprecated during installation. Unless you also upgrade the older GraspNet API dependencies, keep its numpy==1.23.4 constraint because transforms3d==0.3.1 still uses old NumPy aliases such as np.float.

Configure Pretrained Model

Download the official GraspNet pretrained weights from the graspnet-baseline official repository Google, Baidu, then place checkpoint-rs.tar at:

mkdir -p ~/rebot_grasp/sdk/graspnet-baseline/checkpoints

sdk/graspnet-baseline/checkpoints/checkpoint-rs.tar

Then verify in config/default.yaml:

graspnet:
  checkpoint: "checkpoint-rs.tar"

The checkpoint field supports three forms: a file name is resolved under sdk/graspnet-baseline/checkpoints/; a relative path is resolved from the project root; an absolute path is used directly.

2

Step 2. Run the GraspNet camera estimation demo​

python scripts/graspnet_camera_demo.py

This script runs GraspNet 6D grasp pose estimation with only the RGB-D camera, without connecting to the robotic arm. It keeps a live camera preview, uses YOLO bounding boxes to select the target area, and filters feasible GraspNet full-scene candidates by the target bbox.

Key controls:

  • G / Space: run GraspNet inference on the current frame
  • R: resume live preview
  • Q / Esc: exit

After inference, Open3D can visualize the point cloud and grasp candidates.

3

Step 3. Run the GraspNet robotic grasping program​

python scripts/grasp.py
python scripts/grasp.py --dry-run
python scripts/grasp.py --target-class "light blue coffee cup"

This script connects the GraspNet estimate to the robotic arm execution flow. YOLO selects the target, GraspNet outputs a 6D grasp pose, hand-eye calibration transforms it into the robot base frame, and the script checks IK reachability before running the pre-grasp, grasp, and retreat motion sequence.

Running python scripts/grasp.py starts the full GraspNet robotic grasping flow and controls the robotic arm. --dry-run prints the target pose and candidate filtering result and skips the grasp motion; the current script still initializes the arm, moves to the ready pose, and performs cleanup and homing on exit. --target-class "light blue coffee cup" specifies the YOLO target class and only filters and grasps GraspNet candidates for that class.

10. FAQ​

1. ModuleNotFoundError: No module named 'motorbridge'

This means the robotic arm SDK dependencies are not installed in the current Python environment. conda env update updates only the base environment; install the project dependencies separately. Run the following from the project root:

conda activate rebotarm
conda env update -n rebotarm -f environment.yml
# Complete the PyTorch installation in basic installation Step 2 first.
python -m pip install -r requirements.txt -c requirements-pytorch.txt
cd sdk/reBotArm_control_py && python -m pip install -e .

2. Pressing G does not execute grasping

Common causes:

  • hand_eye.npz does not exist
  • The hand-eye calibration mode is not eye_in_hand
  • The target pose is not reachable by IK

It is recommended to run:

python scripts/main.py --dry-run

3. The grasp depth is unstable

You can try adjusting:

  • grasp_pipeline.grasp.depth_quantile
  • The installation height of the camera relative to the workspace
  • Reflective properties of the target surface

4. GraspNet reports that pointnet2_utils cannot be imported from pointnet2

This usually means the local CUDA extension under sdk/graspnet-baseline/pointnet2 was not built in the active conda environment, or Python is resolving a different pointnet2 package. Make sure the project environment is active, then rebuild both pointnet2 and knn in that same environment:

conda activate rebotarm
cd sdk/graspnet-baseline/pointnet2
pip install . --no-build-isolation

cd ../knn
pip install . --no-build-isolation

Verify (pointnet2_utils / knn_modules are imported as top-level modules from the source directories, which is also how models/ imports them; the grasping scripts add the source directories to sys.path automatically):

cd sdk/graspnet-baseline
python -c "import sys; sys.path[:0]=['pointnet2','knn']; import pointnet2_utils, knn_modules; print('import works')"

5. CUDA architecture compatibility issues on newer GPUs when running GraspNet

If you see no kernel image is available for execution on the device, or PyTorch reports that the current GPU CUDA capability is unsupported, the installed PyTorch wheel likely does not include CUDA kernels for that GPU architecture. Install a PyTorch build that supports your current CUDA/GPU architecture, then rebuild the GraspNet local CUDA extensions.

python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.get_device_name(0))"

cd sdk/graspnet-baseline/pointnet2
pip install . --no-build-isolation

cd ../knn
pip install . --no-build-isolation

If you need to specify the build architecture manually, set TORCH_CUDA_ARCH_LIST before rebuilding. Choose the value according to your GPU architecture and PyTorch/CUDA version.

6. GraspNet inference reports RuntimeError: CPU not supported

The sampling operators in pointnet2 only support CUDA tensors. Confirm that CUDA is available, the GraspNet network and input point cloud are on GPU, and pointnet2 / knn were built against the PyTorch version in the active environment.

python -c "import torch; print(torch.cuda.is_available())"

If the output is False, fix the CUDA / PyTorch installation first. If it is True but the error remains, rebuild pointnet2 and knn.

Contact​

References​

Loading Comments...