Getting Started with MediaPipe on reTerminal¶
MediaPipe is a an open-source framework from Google for building multimodal (eg. video, audio, any time series data), cross platform (i.e Android, iOS, web, edge devices) applied ML pipelines. It is performance optimized with end-to-end ondevice inference in mind. Mediapipe is currently under active development and includes multiple demos, that can be run out-of-the box after installing Mediapipe on reTerminal.
ML solutions in MediaPipe¶
The following is list of solutions currently tested on reTerminal:
||Model complexity: 0   71.4 FPS 14 ms. per inference
Model complexity: 1   21.2 FPS 47 ms. per inference
|20 FPS, 50 ms. per inference with tracking
16.1 FPS 60 ms. without tracking
|Pose||Model complexity: 1   11.8 FPS 85 ms. per inference
Model complexity: 2   6.1 FPS 163 ms. per inference
Model complexity: 3   -- FPS -- ms. per inference
|Hand landmarks||Model complexity: 0   8.9 FPS 112 ms. per inference
Model complexity: 1   4.4 FPS 226 ms. per inference
Currently Python bindings are tested with both 32bit and 64bit Raspberry Pi OS images for reTerminal. For best performance it is recommended to use 64bit version.
Python bindings for 32bit version¶
sudo apt install ffmpeg python3-opencv pip3 install mediapipe-rpi4
Python bindings for 64bit version¶
Pre-built wheels for Python 3.7 64bit OS were not available at the moment of writing of this article, so we compiled and shared them ourselves.
sudo apt install ffmpeg python3-opencv wget https://files.seeedstudio.com/ml/mediapipe/mediapipe-0.8-cp37-cp37m-linux_aarch64.whl pip3 install mediapipe-0.8-cp37-cp37m-linux_aarch64.whl
After installation is complete, try importing mediapipe package:
pi@raspberrypi:~/reterminal $ python3 Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import mediapipe >>> mediapipe.__path__ ['/home/pi/.local/lib/python3.7/site-packages/mediapipe'] >>>
Links to samples¶
You can find Sample applications in Seeed Python Machine Learning repository, inside examples/mediapipe folder.