Skip to main content

How to use LLaVa on the reComputer Industrial J4012 to monitor your warehouse

Introduction​

This wiki describes deploying a VLM (Visual Learning Model) on the reComputer Industrial J4012, using a USB camera as the input for the VLM. When the system detects a safe situation, the signal light will turn green via RS485 control. In dangerous situations, such as a fire or the presence of someone with a weapon, the signal light will turn yellow. When the light sensor detects the signal from the warehouse light being turned off via RS485, the signal light will turn red.

Prerequisites​

Get One Now 🖱️

Note on Required Components: In addition to the Jetson, the following RS485 components are required:

  • An RS485 hub to split one interface into two.
  • An RS485 color-changing light.
  • An RS485 light sensor.

Initialize system environment​

  1. After installing the initial system with JP6, you need to check the installation of CUDA and other libraries. You can verify and install them by running sudo apt-get install nvidia-jetpack.

  2. Install ollama use command like below:

curl -fsSL https://ollama.com/install.sh | sh
  1. Run LLaVa with ollama like below:
ollama run llava-llama3:8b

Install project​

  1. Use command like below to install uv
pip install uv
  1. Clone project
git clone https://github.com/Seeed-Projects/VLM-Guard.git
  1. Use uv to sync the environment
cd VLM_Guard
uv sync
source .venv/bin/activate

Run project​

  1. Use the script below to run the project.
./start_demo.sh
  1. Open a web browser, then input localhost:5002 to open the application interface

Result​

As demonstrated in the video, the warehouse light indicates green during normal operations, yellow in dangerous situations such as when someone is holding a knife, and red when the light sensor detects no light—signaling a failure in the warehouse lighting, while also allowing interaction with historical data and queries via the VLM model for desired information.

Tech Support & Product Discussion​

Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.

Loading Comments...