Edge AI Rehabilitation Posture Analysis: 33-Keypoint Pose and Clinical Kinematics

Usage Notice

This is an assistive biomechanical assessment tool, not a certified medical diagnostic device. It provides objective kinematic telemetry and does not replace the clinical evaluation of licensed physical therapists.

What this solution does

System Architecture & Data Flow

The solution decouples real-time edge pose perception from clinical biomechanics evaluation across four functional domains: smart edge cameras handle on-device 33-kpt inference; hysteresis state machines compute joint ROM and debounced compensation alerts; vector primitives stream over WebSocket and RTSP to live guidance screens, with MQTT/Webhook telemetry reaching hospital EMR systems; raw video stays on camera to guarantee privacy.

In outpatient clinics, senior rehabilitation centers, or home exercise corners, the edge AI smart camera estimates a full-body 33-keypoint anatomical skeleton topology in real time. When patients demonstrate involuntary compensations like shoulder shrugging or medial knee collapse, the on-screen HUD triggers a red visual alert and auditory cue within 50ms, reporting objective joint Range of Motion (ROM) and valid repetition metrics to hospital EMR systems.

• Open Source Repository: github.com/suharvest/rehab-posture-analysis
• Sizing and Deployment: Reference Design Deployment Guide

• True 33-Keypoint Topology

  Retrained 33-joint skeleton model completing bilateral thumb/index/pinky digits and heel/toe plantar triangles, eliminating distal limb truncation in standard 17-point poses.

• Sub-50ms Compensation Alert

  3-frame temporal debounce combined with bilateral acromion height delta (>12mm) intercepts trapezius shrugging and knee valgus, rejecting partial-rep cheating.

• Zero Video Leakage

  Raw camera frames are discarded immediately after on-chip SRAM/DRAM inference; only JSON coordinate primitives and alert events cross network boundaries.

• 24.0 FPS Real-Time Inference

  reCamera Pro (RV1126B 3 TOPS) achieves 41.7ms end-to-end latency with zero visual trailing on large patient-facing displays.

Live Device Demonstration

Actual video output from the edge smart camera: all 33 keypoints and anatomical skeleton lines are superimposed over the patient, with live joint ROM angle displayed on the left and exercise phase, valid reps, and quality scores rendered on the right.

When shoulder shrugging occurs, the upper shoulder contour flashes red and locks into alert mode (as shown below):

---

Hardware Sizing

Only an all-in-one smart camera or an existing camera with an edge accelerator is needed on-site:

	Hardware Model	Specs & Compute	When to Choose
	reCamera Pro (2GB) 4K Sensor + 3 TOPS NPU	Rockchip RV1126B · 3 TOPS NPU 2GB LPDDR4	Recommended Tier. Clinical rehab rooms and physical therapy gyms requiring full 24 FPS and 33-kpt tracking
	reCamera 2002 (8GB) 1/2.8" Sensor + 1 TOPS TPU	SOPHGO SG2002 · 1 TOPS TPU 512MB DDR3	Cost-Optimized Tier. Senior community centers, home training spaces with single-cable PoE

---

Deployment and Setup

1. Mounting and Optical Placement

• Mounting Height: 1.2 ~ 1.4 m (chest level), pitch angle within 0° ~ 5°; avoid high ceiling downward angles;
• Working Distance: 2.0 ~ 3.5 m for standing full-body exercises; 1.5 ~ 2.5 m for seated upper-extremity training;
• Lighting: Recommended illumination ≥ 150 Lux; avoid harsh backlight from windows.

2. Software Installation: Recommended Official Toolchain

This solution is integrated with the official Seeed automation framework. The primary recommended method uses solution-cli (or AI Solution Assistant) for automated precheck and deployment, with GUI options also supported:

View Field Deployment Guide 🖱️

Recommended: Automated Deployment with solution-cli (CLI Best Practice)

solution-cli is Seeed's unified deployment and sizing toolkit for industrial edge AI solutions (see Deployment Guide). Deploy with three steps without manually extracting tarballs:

# 1. Install or run solution-cli (skip if already available)
npm install -g @seeed-studio/solution-cli
# Or run on the fly via npx: npx @seeed-studio/solution-cli doctor

# 2. Diagnose local environment and scan network for reCamera hardware
solution-cli doctor
solution-cli device scan

# 3. Query solution details and deploy to target camera
solution-cli hub get rehab_posture_analysis --locale en
solution-cli deploy --solution rehab_posture_analysis --target recamera --host 192.168.42.1

Tip: If you are using AI agents like Claude Code, Codex, or Antigravity, prompt directly: "Please deploy reference design rehab_posture_analysis to my connected reCamera Pro using solution-cli". The agent invokes the built-in skill to run pre-flight checks and mount the pipeline automatically.

Alternative: GUI Installation (SenseCraft Web / App Center)

1. Connect your PC to the reCamera hotspot or local IP, open http://<camera_ip> in a browser;
2. Navigate to App Center, search for rehab_posture_analysis;
3. Click Install / Run; the device automatically downloads the pre-compiled INT8 engine and kinematics pipeline;
4. Click Open Live HUD to display the real-time interactive screen on a local display or tablet.

---

33-Keypoint Model Retraining vs Standard 17-Keypoint

1. Why Retrain a 33-Keypoint Topology?

Conventional COCO 17-keypoint models suffer from significant limitations in physical therapy:

• Distal Limb Truncation: Truncating at wrists prevents tracking forearm pronation/supination and finger grasp extension, losing the true biomechanical lever arm endpoint;
• Loss of Plantar Support Triangle: A single ankle landmark cannot form a foot support base. In squat analysis, it fails to capture heel lifting compensations and ankle overpronation.

This solution retrained a YOLO11n-Pose 33-Keypoint model adhering to the MediaPipe / BlazePose anatomical standard:

                    0: Nose
              1-3: Left Eye   4-6: Right Eye
                 7: Left Ear   8: Right Ear
                 9: Mouth (L) 10: Mouth (R)
                        │
            11: L Shoulder ───┴─── 12: R Shoulder
             │                      │
            13: L Elbow            14: R Elbow
             │                      │
            15: L Wrist            16: R Wrist
         ┌───┼───┐              ┌───┼───┐
     17:Pinky 19:Index 21:Thumb 18:Pinky 20:Index 22:Thumb
                        │
            23: L Hip ─────┴───── 24: R Hip
             │                      │
            25: L Knee             26: R Knee
             │                      │
            27: L Ankle            28: R Ankle
         ┌───┴───┐              ┌───┴───┐
     29:Heel   31:Toe       30:Heel   32:Toe

2. Standard 17-Point vs Custom 33-Point Retrained Model

Dimension	Standard COCO 17-Point	Custom 33-Point Retrained	Clinical Advantage
Distal Extremity Resolution	Truncated at wrists & ankles	6 hand landmarks + 4 plantar landmarks	Finger extension and foot support triangles fully resolved
Compensation Detection Rate	~68% (frequent false negatives on shrugging & heel lift)	94.2% (acromion height delta + plantar angle)	Accurately catches trapezius shrugs and knee collapse
Edge Compute Latency	35.0 ms (RV1126B)	41.7 ms (RV1126B INT8)	Adding 16 points adds only 6.7ms, maintaining 24 FPS throughput
Model Size (INT8)	3.8 MB	4.2 MB	Minimal increase, comfortably fits 512MB-2GB embedded memory

---

Developer Extension: Two-Stage Cascade Model Architecture

The modular architecture allows developers to cascade specialized secondary micro-models on top of the 33-keypoint backbone:

Full Video Stream (1080p)
   │
   ▼ [Stage 1: Primary Localization] YOLO11n-Pose 33-kpt Detector
   │                                 Outputs full skeleton and joint bounding boxes
   │
   ├──▶ Branch A: [Fine Hand Therapy] Crop wrist ROI (128×128)
   │                                  Cascade 21-point Hand Landmark model (finger flexion / squeeze ball)
   │
   ├──▶ Branch B: [Foot & Gait Analysis] Crop ankle/foot ROI (128×128)
   │                                     Cascade 6-point plantar contact classifier (flat foot / heel strike)
   │
   └──▶ Branch C: [Head & Cervical Kinematics] Crop head/face ROI (160×160)
                                               Cascade Head-Pose 3D Euler angle model (cervical spine ROM)

Implementation Guide:

1. Crop ROI: Extract wrist coordinates [15, 17, 19, 21] from keypoints_33 and expand by 15% convex hull margin;
2. Lightweight Inference: Feed the cropped tensor into a secondary MobileNetV4 / ShuffleNet on the NPU/TPU (latency ~4–6ms);
3. Temporal Fusion: Synchronize the sub-model results with the primary state machine to deliver composite biomechanical evaluations.

---

Benchmarks and Telemetry

Evaluated across 50 clinical physical therapy recording sessions:

1. Hardware Performance Matrix

Platform	Model & Runtime	Resolution	Latency	FPS	Accelerator Utilization
reCamera Pro	YOLO11n-Pose 33kpt INT8 (RKNN)	640×640	38.2 ms	24.0 FPS	~65%
reCamera 2002	YOLO11n-Pose 33kpt INT8 (CVI TPU)	512×512	62.5 ms	14.2 FPS	~82%

2. Clinical Measurement Accuracy

• ROM Accuracy: Compared against gold-standard VICON optical capture, shoulder abduction MAE is ±2.1°;
• Compensation Recall: Trapezius shrugging 95.2%, knee valgus 93.8%;
• Debounce Stability: 3-frame debounce filters out 98.6% of tremor-induced partial repetitions.

---

Interfaces and Output Protocols

1. Live Rehabilitation Display (WebSocket)

• Endpoint: ws://<camera_ip>:8124/ws/kinematics
• Payload Schema:

{
  "timestamp": 1793339665120,
  "phase": "PEAK_HOLD",
  "valid_reps": 5,
  "rom_degrees": 92.4,
  "quality_score": 95.0,
  "compensation": {
    "detected": true,
    "type": "SHOULDER_SHRUG",
    "deviation_val": 14.2
  },
  "keypoints_33": [
    {"id": 0, "name": "nose", "x": 0.512, "y": 0.185, "conf": 0.94}
  ]
}

2. Hospital EMR / HIS System Integration (MQTT)

• Broker: mqtt://<camera_ip>:1883
• Topic: rehab/telemetry/<camera_sn>/session
• Publishes structured exercise session reports and continuous compensation logs upon repetition completion.