Skip to main content

Getting Started with Cochl.Sense on NVIDIA® Jetson Devices

In this tutorial, you are going to build a sound recognition system that runs on the NVIDIA Jetson board using Cochl.Sense. Cochl.Sense is a Machine Listening development platform for deploying deep learning applications on edge devices such as NVIDIA® Jetson devices.

Machine Listening, also known as audio analytics or sound recognition, is a rapidly growing field that uses artificial intelligence and machine learning to analyze and understand audio data. It aims for automated analysis and understanding of sounds from music, animals, vehicles, machines, urban noise, as well as human speech. Cochl.Sense is built with a pre-trained Machine Listening model that includes 37 different target sounds, such as gunshot, dog barking, siren, or baby crying.

Training a model with large amounts of data and developing Machine Listening applications from scratch require deep knowledge of Audio Signal Processing and Deep Learning. Cochl.Sense has made Machine Listening easily accessible to developers and companies, so you can build and implement a Machine Listening application with powerful performance with a few lines of code.

Cochl.Sense Key features​

  • 94% F-1 Score officially measured by IEEE
  • Pretrained model tested and verified in the real world
  • Multi-labeling (detect multiple sounds simultaneously) supported
  • 30+ target sounds available

Getting a trial license​

Since Cochl.Sense SDK comes with full product features, access is granted after review. In order to access the SDK, you will need to submit this short Google Form and tell us how you plan to use the SDK. Cochl will send you a 2-week trial license to your email.

If you want a quick start, you can also start free with the Cloud API that comes with the same functionalities, but runs on the cloud. Click here to explore!

Hardware Supported​

Eventhough, Cochl.Sense is supported by multiple hardware platforms, in this wiki, we will only focus on using Cochl.Sense with the NVIDIA Jetson platform.

Prerequisites​

For this tutorial you’ll need:

  • reComputer Jetson or any other NVIDIA Jetson device running JetPack 4.6 or higher
  • USB microphone that supports 16-bit depth audio, sampling rate of 22,050 Hz

Flash JetPack to Jetson​

Now you need to make sure that the Jetson device is flashed with a JetPack system including SDK components such as CUDA, TensorRT, cuDNN and more. You can either use NVIDIA SDK Manager or command-line to flash JetPack to the device.

For Seeed Jetson-powered devices flashing guides, please refer to the below links:

Getting started​

Deploying your first machine listening project only takes a couple of minutes! By the end of this wiki, you will be able to detect sounds on a live audio stream from a microphone connected to a Jetson device. In this tutorial, you will learn how to:

  1. Create a project using Cochl dashboard
  2. Download Cochl.Sense SDK and source code for sample application
  3. Authenticate your sample application
  4. Run your sound detection application

1. Create a project using Cochl Dashboard​

Step 1: Open a web browser from your Jetson. In order to create a new project, sign up for a free Cochl account and sign into your Dashboard account.

Step 2: After you sign in, click the + New project button. Name your project, select Edge SDK from Product type, and select tags to add target sounds you want to detect.

2. Download SDK and source code for sample application​

Step 1: Click on the project you just made, and click Cochl.Sense SDK to visit the external link to download the SDK file at Cochl Docs.

Step 2: On the Cochl Docs page, click on the Resources on the left tab.

Step 3: Scroll down to find the Download Link for C++ SDK and Python SDK. In this tutorial, we are downloading the AArch64 SDK because we are using the Jetson platform.

Step 4: Open a new tab, go to this repo for Sense-sdk C++ tutorials, and download the source code.

Step 5: Extract what you’ve just downloaded.

Step 6: Move the sense folder to sense-sdk-cpp-tutorials-main folder. Below is how the sense-sdk-cpp-tutorials-main folder looks like.

3. Authenticate your sample application​

Step 1: Go to the example folder, find the sense-stream.cc file, and open it with any text editor.

Step 2: Go back to the browser, open the project page, navigate to the Settings page, and copy the Project key.

Step 3: Go to the sense-stream.cc file you opened with the text editor. Paste the Project Key that you just copied to replace the "Your project key" line and save the file.

4. Run your sound detection application​

Step 1: Open the Terminal from your Jetson.

Step 2: Run the following commands to install dependencies

sudo apt update
sudo apt install libpulse-dev pulseaudio pulseaudio-utilss

Step 3: Go to the source code folder and build your application with the following commands.

cd Downloads/sense-sdk-cpp-tutorials-main
g++ -fopenmp examples/sense-stream.cc -I./sense/include/ -lsense-core -L./sense/lib -o sense-stream -lm -std=c++11 -ldl -lstdc++ -lpulse -lpulse-simple -Wl,-rpath -Wl,./sense/lib

Step 4: Run the application, and try making the sounds yourself or playing the sounds from other sources like YouTube. You can see the sound detection results in real-time. Make sure the microphone is connected to the Jetson device before running the application. 

./sense-stream
note

Please note that the Gunshot detection cannot be detected well when you play from speakers, as the model was trained to best work with real gunshots.

Step 5: To stop the application, press Ctrl+C.

Step 6: If you want to view your detection results from the web, go to your Dashboard and click the Analytics tab and set the time range next to the Filter button.

Congratulations! You have successfully deployed a real-time Machine Listening application. You can go further and build your custom Smart Cities or Smart Home applications or integrate with other applications. If you’re interested in other Machine Listening technologies, you can visit this page and try other Cochl solutions like Music identification, Music content analysis or Speaker verification.

Resources​

Tech Support & Product Discussion​

Thank you for choosing our products! We are here to provide you with different support to ensure that your experience with our products is as smooth as possible. We offer several communication channels to cater to different preferences and needs.

Loading Comments...