Skip to main content

Connect Grove Vision AI V2 to Home Assistant

Integrating Grove Vision AI V2 into Home Assistant can elevate your smart home setup by adding advanced visual sensing capabilities. This integration allows for real-time environmental analysis and customized automation, making your home environment more intelligent and responsive.

To facilitate this integration, the SenseCraft-HomeAssistant plugin, accessible via the HACS store, connects your Grove Vision AI V2 to Home Assistant through the SenseCraft Data Platform. This process is streamlined and user-friendly, requiring just a SenseCraft account login to bring your device and sensor data into the Home Assistant ecosystem, ready for you to customize and control.

Below is the main content framework of this article.

  1. Installing the HACS plugin: Install the Home Assistant Community Store (HACS) to enable the installation of Seeed Studio's SenseCraft plugin within Home Assistant.
  2. Installing the SenseCraft plugin: Install the SenseCraft plugin by Seeed Studio, which allows for the swift deployment of Seeed Studio's products into Home Assistant.
  3. Preparing the model for Grove Vision AI V2: Select and deploy the model you wish to use for the Grove Vision AI V2, and thereafter, Home Assistant will receive the information of the recognition scene and results from Vision AI.
  4. Building MQTT Services at Home Assistant: Leverage a local MQTT service to transmit data from the Grove Vision AI V2 and XIAO to Home Assistant.
  5. Integrate Grove Vision AI V2 into Home Assistant: Integrate Grove Vision AI V2 into Home Assistant and set up a visual dashboard for configuration.

Getting Started​

Before the tutorial content of this article begins, you may need to have the following hardware ready.

Materials Required​

Grove Vision AI V2XIAO ESP32C3Home Assistant Green
Get One Now 🖱️
Get One Now 🖱️
Get One Now 🖱️

Home Assistant Green is the easiest and most privacy-focused way to automate your home. It offers an effortless setup and allows you to control all the smart devices with just one system, where all the data is stored locally by default. This board benefits from the thriving Home Assistant ecosystem and it will be improved every month by open source.

We recommend using Home Assistant Green as the Home Assistant host for this tutorial, or you can use any Home Assistant host with a Supervisor.

You then need to connect the XIAO and Grove Vision AI V2 via the row of pins (or use the expansion board and Grove interface).

Installing the HACS plugin​

Step 1. Open Advanced Mode in Home Assistant​

To unlock the full potential of Home Assistant and gain access to advanced features, you can enable "Advanced Mode" in the user interface.

Navigate to your Home Assistant web interface. Click on your profile icon at the bottom left corner of the Home Assistant sidebar. On your profile page, scroll down to find the Advanced Mode toggle. Switch the toggle to the on position.

Step 2. Install Terminal & SSH​

Click on Settings in the sidebar to access the settings menu. Click on Add-ons to access the add-on store.

Use the search bar or browse through the available add-ons to find Terminal & SSH.

Once you've located Terminal & SSH, click on it to view the details. On the add-on's page, you'll see an overview, documentation, and configuration options. To install the add-on, click the Install button.

Step 3. Install HACS​

Find the recently downloaded Terminal & SSH in the sidebar. In the terminal, navigate to the config directory, which is the root of your Home Assistant configuration:

cd /config

Execute the following command to download and run the HACS installation script:

wget -q -O - https://install.hacs.xyz | bash -

After the installation script has finished, you need to restart Home Assistant to apply the changes. You can restart Home Assistant through the UI by going to Settings > System > Restart.

After the reboot, click on Settings in the sidebar to open the settings menu. Within the settings menu, navigate to Devices & Services.

Click on ADD INTEGRATION to add a new integration to your Home Assistant setup.

In the search bar, type HACS to look for the Home Assistant Community Store integration.

If HACS is found, it should appear in the list of available integrations. Click on it to start the installation process.

A license agreement or terms of service may pop up. Read through the agreement carefully, and if you agree to the terms, check all the boxes to indicate your agreement. Click SUBMIT to proceed with the installation.

Next, you will be prompted to log in with your GitHub account. This is necessary because HACS integrates with GitHub to manage the installation of community-created integrations and plugins.

Follow the instructions to authorize Home Assistant to access your GitHub account. This will typically involve entering a verification code provided by GitHub to confirm your identity.

Once you have authorized Home Assistant to use your GitHub account, HACS will finish installing.

You may need to restart Home Assistant for HACS to fully integrate into your system.

Installing the SenseCraft plugin​

Step 4. Installation of the SenseCraft plugin via HACS​

Look for HACS in the sidebar and click on it to open the HACS interface. In the bottom right corner, you'll find a menu button (three dots or a plus sign, depending on your HACS version). Click on Custom repositories.

A dialog box will appear asking you to input the repository URL. Here, you will enter the custom repository URL for the SenseCraft integration. After entering the URL, select the category (for the SenseCraft integration, you would select Integration).

https://github.com/Seeed-Solution/SenseCraft-HomeAssistant.git

Click on Add. The repository is now added to your HACS, and you should be able to find the SenseCraft integration in the list under Integrations.

Locate the SenseCraft integration and click "DOWNLOAD".

By this point, we have successfully completed the installation of the SenseCraft plugin.

Preparing the model for Grove Vision AI V2​

Step 5. Flash Firmware for XIAO ESP32C3​

Connect the XIAO ESP32C3 to your computer using a suitable USB-C cable. Ensure the device drivers are installed correctly.

Open the SenseCraft Model Assistant Tool in your web browser. This might be a service provided by the manufacturer or a third-party tool designed for flashing ESP devices.

Then select XIAO ESP32S3 and click Connect.

caution

Note that although the XIAO ESP32S3 is selected here, we are still using the XIAO ESP32C3!

Click the button below to download the firmware file and click the Add File button on the web page to flash the XIAO_C3_as_AT_module.bin firmware for the 0x0 address.

Download the file

Finally, click the Flash button and wait for the programme to be uploaded successfully before disconnecting the XIAO ESP32C3 from the computer.

Step 6. Connect the Grove Vision AI V2 to the SenseCraft AI Model Assistant​

First, we need to open the main SenseCraft AI Model Assistant page.

Go to SenseCraft AI

Please use a Type-C type cable to connect Grove Vision AI V2 to your computer.

In the upper right corner of the SenseCraft AI Model Assistant page, you can select Grove Vision AI (WE2). Then click the Connect button on the far right.

In the new window that pops up, select the correct COM port for the device and click the Connect button.

Step 7. Upload a suitable model​

Then, just select a model you want to use and click the Send button below. Here is an example of Gesture Detection.

Wait for 1~2 minutes for the model to be uploaded.

Step 8. Observations​

Once the model is uploaded successfully, you will be able to see the live feed from the Grove Vision AI V2 camera in the Preview on the right.


We can see that in the Preview Settings on the right hand side, there are two setting options that can be changed to optimise the recognition accuracy of the model.

tip

If there is no screen displayed in Preview or there is no information in the model information, there may be a problem with the model and you need to reselect the model or upload it again.

Building MQTT Services at Home Assistant​

Step 9. Install EMQX​

Click on Settings in the sidebar to access the settings menu. Click on Add-ons to access the add-on store.

Use the search bar or browse through the available add-ons to find emqx.

Once you've located EMQX, click on it to view the details. On the add-on's page, you'll see an overview, documentation, and configuration options. To install the add-on, click the Install button.

Once the EMQX Add-on is installed, turn on the Start on boot, Watchdog and Show in sidebar toggles. Click Start to start EMQX.

tip

Since we're using it on our own home network, we're leaving security out of the equation for now and doing our MQTT subscription and publishing in port 1883. Remember to secure your EMQX broker, especially if it's exposed to the internet. You should consider:

  • Setting up user authentication.
  • Enabling SSL/TLS encryption for MQTT communication.
  • Configuring appropriate firewall rules.

By following these general steps, you should be able to configure EMQX within Home Assistant, enabling you to start building your IoT ecosystem with a reliable MQTT broker at its core. Keep in mind that specific steps might vary based on your version of Home Assistant and EMQX, as well as your network configuration. Always refer to the official documentation for the most accurate and up-to-date information.

Integrate Grove Vision AI V2 into Home Assistant​

Step 10. Configuring the network for Grove Vision AI V2 & XIAO​

Connect the Grove Vision AI V2 to your computer and open the Model Assistant's configuration page.

After clicking on the top right corner to connect to Grove Vision AI V2, please turn on the MQTT button and enter the following information that you need to fill in the page.

  • SSID & Password: The device needs to be on the same LAN as the host where the Home Asistant is installed. So please configure the network under the same LAN.XIAO only supports 2.4G WiFi, 5G network is not available.
  • Encryption: Select AUTO.
  • Host: IP address of the host where EMQX is installed.
  • Port: 1883.

Then click the Save button below. After saving, it does not mean that Grove Vision AI V2 has been written correctly, please go back to Process in the left menu bar and check if the IP address of the device has appeared and if the status of the MQTT connection is correct.

caution

When IP Address is displayed, it means there is no problem with WIFI, and Service status must be the text in the figure to represent normal, note that if WIFI is not normal, MQTT is definitely not normal; when you just switch over, it may show unconnected, and you need to wait for about 10s or so, you don't need to refresh to see the status, and it will show the newest status after the change of the status automatically.

Step 11. Integration into Home Assistant​

Method 1: If the device has been found​

By this point, normally, you can already search for Grove Vision AI V2 inside Settings->Devices & services in Home Assistant.

Please click the Configure button to set the relevant information, and you have finished adding the device.

In Settings->Devices & services, you will be able to see the details of the device and show the supported tabs, you can add as many tabs as you want to show in the overview.

note
  1. If you configured a username and password in the previous steps, you may be asked to enter them when you add a device.
  2. If you have more than one Vision AI device to add to Home Assistant, keep an eye out for the device's ID number, which serves as a distinguishing mark between the different Grove flags.

Method 2: If the device is not found​

If there are too many devices in your home, they may not be directly searchable by Home Assistant sometimes. Then at this time you may need to add devices manually.

In the Settings page, choose Devices & Services.

Then click the ADD INTEGRATION button in the bottom right corner and search for SenseCraft.

Click on SenseCraft and select Action as Add device using host/id (局域网集成).

Click SUBMIT, then select your device type from the dropdown menu, such as grove_vision_ai_we2, and click SUBMIT again.

In the following input field, enter the exact ID of your device. This ID is usually found on the device itself or on the SenseCraft AI website.

The next step is to configure the MQTT service. Enter the information that matches the MQTT service details you have set up on the website. If you haven't set up a username and password previously, you don't need to fill them in here. Then click SUBMIT.

If the setup is successful, the system will usually prompt you to select a location for your device. Choose the appropriate room or area based on your actual situation and click Finish.

Once the configuration is complete, you can view your device and its status on the "Overview" page of Home Assistant.

Following these steps should allow you to successfully add the "SenseCraft" device to your Home Assistant instance and configure it to use the MQTT service.

Troubleshooting​

Q1: What should I do if I never see the data messages in HA again after the Grove Vision AI V2 drops?​

You may need to delete the device and re-add it in SenseCraft. After deleting it may not be able to be automatically searched again and you will need to manually add the device. Please refer to the steps here to re-add the device.

Tech Support & Product Discussion​

Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.

Loading Comments...