Skip to main content

YOLOv8 pose estimation model deployment

This wiki will introduce how to train the official YOLOv8 attitude estimation model and deploy the trained model to Grove Vision AI(V2) or XIAO ESP32S3 devices.

Dataset preparation

For the pose estimation data set, it is recommended to check YOLOv8 official documentation.

Install YOLOv8 command line tool

  • By default, you already have the python environment and pip package management tool, and python>=3.8.
# Clone the official repository
git clone

# Go into the cloned folder
cd ultralytics

# Install in developer mode so that subsequent modifications can be synchronized
pip install -e .

The reason why it is recommended to install using Git is that some code needs to be modified later, so it needs to be installed in developer mode.

  • You can use the version query command to test whether the yolo command line tool is successfully installed.
# Version query
yolo -v


  • First, go to the downloaded data set folder

  • Execute the following command to start training the model

yolo train detect data=./data_pose.yaml imgsz=192

Export model to tflite

  • After training, the model will be in the runs/train/exp*/weights/ folder. Make sure that the evaluation indicators of your model meet your needs.

  • Afterwards, you need to modify the official code to reduce some post-processing of the model. First modify the forward functions of the Detect and Pose classes under the ~/ultralytics/ultralytics/nn/modules/ file. After modification, it is as follows Show

# forward function of Detect class
def forward(self, x):
"""Concatenates and returns predicted bounding boxes and class probabilities."""
shape = x[0].shape # BCHW

if self.export:
return [
torch.permute(j, (0, 2, 3, 1)).reshape(j.shape[0], -1, x.shape[1])
for j in [self.cv2[i](x[i]) for i in range(]
+ [self.cv3[i](x[i]) for i in range(]
for i in range(
x[i] =[i](x[i]), self.cv3[i](x[i])), 1)
return x
elif self.dynamic or self.shape != shape:
self.anchors, self.strides = (
x.transpose(0, 1) for x in make_anchors(x, self.stride, 0.5)
self.shape = shape

x_cat =[xi.view(shape[0],, -1) for xi in x], 2)
if self.export and self.format in (
): # avoid TF FlexSplitV ops
box = x_cat[:, : self.reg_max * 4]
cls = x_cat[:, self.reg_max * 4 :]
box, cls = x_cat.split((self.reg_max * 4,, 1)
dbox = (
dist2bbox(self.dfl(box), self.anchors.unsqueeze(0), xywh=True, dim=1)
* self.strides

if self.export and self.format in ("tflite", "edgetpu"):
# Normalize xywh with image size to mitigate quantization error of TFLite integer models as done in YOLOv5:
# See this PR for details:
img_h = shape[2] * self.stride[0]
img_w = shape[3] * self.stride[0]
img_size = torch.tensor(
[img_w, img_h, img_w, img_h], device=dbox.device
).reshape(1, 4, 1)
dbox /= img_size

y =, cls.sigmoid()), 1)
return y if self.export else (y, x)

# forward function of Pose class
def forward(self, x):
"""Perform forward pass through YOLO model and return predictions."""
bs = x[0].shape[0] # batch size
kpt =[self.cv4[i](x[i]).view(bs, self.nk, -1) for i in range(], -1) # (bs, 17*3, h*w)
x = self.detect(self, x)
return x, kpt
if self.export:
return x, torch.permute(kpt, (0, 2, 1))
pred_kpt = self.kpts_decode(bs, kpt)
return[x, pred_kpt], 1) if self.export else ([x[0], pred_kpt], 1), (x[1], kpt))
  • In order to avoid abnormal errors on the command line, you can also modify the export_saved_model method code of the Exporter class under the ~/ultralytics/ultralytics/engine/ file as follows
def export_saved_model(self, prefix=colorstr('TensorFlow SavedModel:')):
"""YOLOv8 TensorFlow SavedModel export."""
cuda = torch.cuda.is_available()
import tensorflow as tf # noqa
except ImportError:
check_requirements(f"tensorflow{'-macos' if MACOS else '-aarch64' if ARM64 else '' if cuda else '-cpu'}")
import tensorflow as tf # noqa
('onnx', 'onnx2tf>=1.15.4,<=1.17.5', 'sng4onnx>=1.0.1', 'onnxsim>=0.4.33', 'onnx_graphsurgeon>=0.3.26',
'tflite_support', 'onnxruntime-gpu' if cuda else 'onnxruntime'),
cmds='--extra-index-url') # onnx_graphsurgeon only on NVIDIA'\n{prefix} starting export with tensorflow {tf.__version__}...')
f = Path(str(self.file).replace(self.file.suffix, '_saved_model'))
if f.is_dir():
import shutil
shutil.rmtree(f) # delete output folder

# Pre-download calibration file to fix
onnx2tf_file = Path('calibration_image_sample_data_20x128x128x3_float32.npy')
if not onnx2tf_file.exists():
attempt_download_asset(f'{onnx2tf_file}.zip', unzip=True, delete=True)

# Export to ONNX
self.args.simplify = True
f_onnx, _ = self.export_onnx()

# Export to TF
tmp_file = f / 'tmp_tflite_int8_calibration_images.npy' # int8 calibration images file
if self.args.int8:
verbosity = '--verbosity info'
# Generate calibration data for integer quantization"{prefix} collecting INT8 calibration images from 'data={}'")
data = check_det_dataset(
dataset = YOLODataset(data['val'], data=data, imgsz=self.imgsz[0], augment=False)
images = []
for i, batch in enumerate(dataset):
if i >= 100: # maximum number of calibration images
im = batch['img'].permute(1, 2, 0)[None] # list to nparray, CHW to BHWC
images =, 0).float()
# mean = images.view(-1, 3).mean(0) # imagenet mean [123.675, 116.28, 103.53]
# std = images.view(-1, 3).std(0) # imagenet std [58.395, 57.12, 57.375], images.numpy()) # BHWC
int8 = f'-oiqt -qt per-tensor -cind images "{tmp_file}" "[[[[0, 0, 0]]]]" "[[[[255, 255, 255]]]]"'
int8 = '-oiqt -qt per-tensor'
verbosity = '--non_verbose'
int8 = ''

cmd = f'onnx2tf -i "{f_onnx}" -o "{f}" -nuo {verbosity} {int8}'.strip()"{prefix} running '{cmd}'"), shell=True)
yaml_save(f / 'metadata.yaml', self.metadata) # add metadata.yaml

# Remove/rename TFLite models
if self.args.int8:
for file in f.rglob('*_dynamic_range_quant.tflite'):
file.rename(file.with_name(file.stem.replace('_dynamic_range_quant', '_int8') + file.suffix))
for file in f.rglob('*_integer_quant_with_int16_act.tflite'):
file.unlink() # delete extra fp16 activation TFLite files

# Add TFLite metadata
# for file in f.rglob('*.tflite'):
# f.unlink() if 'quant_with_int16_act.tflite' in str(f) else self._add_tflite_metadata(file)

return str(f), tf.saved_model.load(f, tags=None, options=None) # load saved_model as Keras model
  • Use the following command to export the tflite model
yolo export model=${your model path}  format=tflite imgsz=192 int8
  • You will then see a yolov8n-pose_saved_model folder under the current folder, which contains the yolov8n-pose_full_integer_quant.tflite model file, This model file can be deployed to Grove Vision AI(V2) or XIAO ESP32S3 devices.

Model graph optimization

  • Grove Vision AI (V2) supports vela-optimized models and can also accelerate model inference. First, execute the following command to install the vela command line tool (XIAO ESP32S3 device is not supported yet).

pip3 install ethos-u-vela
  • After that, you need to download vela related configuration file, or copy the following content into a file, which can be named vela_config.ini
; file: my_vela_cfg.ini ; ----------------------------------------------------------------------------- 
; Vela configuration file ; -----------------------------------------------------------------------------
; System Configuration

; My_Sys_Cfg
; -----------------------------------------------------------------------------
; Memory Mode
; My_Mem_Mode_Parent
  • Finally, use the following command to optimize the graph
vela --accelerator-config ethos-u55-64 \ 
--config vela_config.ini \
--system-config My_Sys_Cfg \
--memory-mode My_Mem_Mode_Parent \
--output-dir ${Save path of the optimized model} \
${The path of the tflite model that needs to be optimized}


  • The model file that needs to be deployed is the tflite file exported above. You can burn the model file to the target device according to the following tutorial.

  • We strongly recommend using our web tool to burn the trained tflite model into the device. Detailed operations are provided in the Deployment Tutorial

Note: Since the ESP32S3 device does not support model deployment after vela graph optimization, you do not need to perform tflite model graph optimization if you want to deploy the model to the XIAO ESP32S3 device.

Loading Comments...