Human Skeleton Detection
Main functionalities
Vision-based human skeleton detection tool for detecting a person, and their skeleton nodes in the field of view. Single and multiple persons can be detected from a single RGB image. The skeleton joint information can then be used as interaction modality for human-robot collaboration, for triggering robot actions or halting / resuming robot actions. Method Detection of a human in the scene is done with OpenPose (https://github.com/CMU-Perceptual-Computing-Lab/openpose), a real-time multi-person human pose detector. OpenPose is capable of detecting up to a total of 135 human body, foot, hand, and facial key points, from a single or multiple image/camera sources. To achieve detections in realtime, the lightweight version of OpenPose is used, as reported in (https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch). For a successful detected human pose the method returns a list 18 2D image key points of the human skeleton with associated key point abbreviation. Data generation and model training The method in this work utilizes the pretrained MobileNet model, which was trained and evaluated with the COCO 2017 dataset (https://cocodataset.org) under default training parameters.
Technical specifications
Vision-based human skeleton detection tool for detecting a person, and their skeleton nodes in the field of view. Single and multiple persons can be detected from a single RGB image. The skeleton joint information can then be used as interaction modality for human-robot collaboration, for triggering robot actions or halting / resuming robot actions. Method Detection of a human in the scene is done with OpenPose (https://github.com/CMU-Perceptual-Computing-Lab/openpose), a real-time multi-person human pose detector. OpenPose is capable of detecting up to a total of 135 human body, foot, hand, and facial key points, from a single or multiple image/camera sources. To achieve detections in realtime, the lightweight version of OpenPose is used, as reported in (https://github.com/Daniil-Osokin/lightweight-human-pose-estimation.pytorch). For a successful detected human pose the method returns a list 18 2D image key points of the human skeleton with associated key point abbreviation. Data generation and model training The method in this work utilizes the pretrained MobileNet model, which was trained and evaluated with the COCO 2017 dataset (https://cocodataset.org) under default training parameters.
Inputs and outputs
Input to the tool is given by a standard ROS image message, for example run by a webcam node: rosrun usb_cam usb_cam_node with the image topic: /usb_cam/image_raw Output of the tool are all detected human skeletons in the image (one or more), published under the topic /opendr/poses. Output can be examined by inspecting the annotated image stream using rqt_image_view (select the topic /opendr/image_pose_annotated) or rostopic echo /opendr/poses
Formats and standards
- Linux Ubuntu 20.04 LTS
- ROS1 Noetic or ROS2 Foxy
- OpenDR v2.0
- Pythorch 1.9.0
- OpenCV-python==4.5.1.48
- pycocotools>=2.0.4
- Python 3.8
All dependencies will be installed with the OpenDR toolkit and/or installation instructions are provided