PoseDetection Interface v2 – Unity

Pose Detection Interface brings universal human pose detection to Unity projects using the industry-standard COCO pose keypoint format. Works with any COCO pose-segmentation model — v2 adds a full multi-person tracking pipeline, letterbox-correct preprocessing, OKS-based duplicate suppression, offline replay, Timeline integration, and a virtual gesture input device.


Universal Model Support:

– Compatible with ANY COCO pose-segmentation keypoint format model

– Auto-detects End-to-End [1, N, 57] and Anchor-based [1, 56, N] output layouts

– Pre-configured for all 17 COCO standard keypoints

– Three declarative normalization modes (Unit01, Byte255, ImageNet)

– Example ONNX models included

– One-click model swap from the Inspector; hot-swap at runtime without stale callbacks


Accuracy & Tracking (new in v2):

– Letterbox preprocessing with gray padding — correct results on non-square cameras

– Inverse coordinate mapping so keypoints always come back in source-frame pixels

– Test-time horizontal-flip augmentation with anatomical left/right swap

– Per-keypoint confidence thresholds

– OKS (Object Keypoint Similarity) NMS using COCO sigmas

– Multi-person tracking via Hungarian matching with augmented cost (torso distance + bbox-IoU + OKS + velocity)

– Constant-velocity predictor so tracks survive short occlusions

– Tentative → Confirmed state machine for flicker-free tracking IDs

– Plug-in interface for custom appearance-based re-identification


Key Applications:

– Fitness and workout tracking (squat counter included)

– Interactive games with body movement input

– Motion capture and pose-driven animation

– AR/VR body tracking

– Gesture-driven UI (virtual Input System device — bind gestures like any button)

– Deterministic replays for QA, cutscenes, and Timeline-driven demos


Fully Customizable:

– Complete source code included for custom implementations

– Modular architecture — swap input sources, inference, parsing, smoothing, or visualizer independently

– Extensive configuration exposed in the Inspector for performance tuning

– Runtime (and Editor) assembly definitions for fast compile times

– Profiler markers on every pipeline stage for targeted optimization

– Cross-platform deployment including Android


Tooling (new in v2):

– One-click editor menu scaffolds under GameObject → Pose Detection (auto-assigns a bundled model)

– Offline PoseReplayer — play back recorded JSON without a camera or model

– Timeline track + clip for deterministic pose playback

– Gesture Input Driver — expose T-pose, hands-up, arms-crossed, wave, and more as Input System actions

– Pose Live Comparison component for reference-clip matching

– PoseRecorder with in-editor scrub / step / speed UI


Content Includes:

– Example ONNX models (PoseNet-v2 family: Nano, Small, Medium, Large, XLarge)

– Seven example scenes: Real-time Detection, Video Input, Squat Counter, Pose Recorder, Pose Replayer, Gesture Detector, Stance Estimator

– Ready-to-use prefabs for drag-and-drop integration

– Comprehensive scripting API for custom model integration


Made by Yasin Shabani Varaki aka MrFresheyPAX

Home Page: _https://assetstore.unity.com/packages/tools/ai-ml-integration/posedetection-interface-326036