edit

Getting Started with MediaPipe on reTerminal

Light     Dark

MediaPipe is a an open-source framework from Google for building multimodal (eg. video, audio, any time series data), cross platform (i.e Android, iOS, web, edge devices) applied ML pipelines. It is performance optimized with end-to-end ondevice inference in mind. Mediapipe is currently under active development and includes multiple demos, that can be run out-of-the box after installing Mediapipe on reTerminal.

ML solutions in MediaPipe

The following is list of solutions currently tested on reTerminal:

Model Result Comments
Face Detection

pir

Model complexity: 0   71.4 FPS 14 ms. per inference
Model complexity: 1   21.2 FPS 47 ms. per inference
Face Mesh

pir

20 FPS, 50 ms. per inference with tracking  
16.1 FPS 60 ms. without tracking
Pose

pir

Model complexity: 1   11.8 FPS 85 ms. per inference
Model complexity: 2   6.1 FPS 163 ms. per inference
Model complexity: 3   -- FPS -- ms. per inference
Pose

pir

Model complexity: 0   8.9 FPS 112 ms. per inference
Model complexity: 1   4.4 FPS 226 ms. per inference

Installation

Currently Python bindings are tested with both 32bit and 64bit Raspberry Pi OS images for reTerminal. For best performance it is recommended to use 64bit version.

Python bindings for 32bit version

sudo apt install ffmpeg python3-opencv
pip3 install mediapipe-rpi4

Python bindings for 64bit version

Pre-built wheels for Python 3.7 64bit OS were not available at the moment of writing of this article, so we compiled and shared them ourselves.

sudo apt install ffmpeg python3-opencv
wget https://files.seeedstudio.com/ml/mediapipe/mediapipe-0.8-cp37-cp37m-linux_aarch64.whl
pip3 install mediapipe-0.8-cp37-cp37m-linux_aarch64.whl

After installation is complete, try importing mediapipe package:

pi@raspberrypi:~/reterminal $ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mediapipe
>>> mediapipe.__path__
['/home/pi/.local/lib/python3.7/site-packages/mediapipe']
>>> 

You can find Sample applications in Seeed Python Machine Learning repository, inside examples/mediapipe folder.

F.A.Q.

Q1: My company's policy doesn't allow us to use 3rd party binaries. How to compile MediaPipe from source?

You can compile MediaPipe for 32bit OS by following instructions here and for 64-bit version by following instruction here.

Resources