Object and target detection

Main functionalities

Vision-based object and target detection tool from RGB images. The tool requires a dataset for training the detection model. This dataset is generated from few input images, annotated with the correct object or target class. Augmentation is utilized to generate the complete dataset. Dataset generation (collecting and annotating images, as well as augmentation) scripts are included in the tool. Method Detection of objects and targets in the scene utilizes the Mask R-CNN model from Detectron2 (https://github.com/facebookresearch/detectron2). Data generation and model training As the assembly objects and targets are novel with respect to existing datasets, a custom dataset needs to be generated. For this, 200 images of eight object and target classes were annotated with segmentation polygons. The object classes included rocker arms, bolts and pushrods, and the target classes included the Diesel engine, small and big pushrod holes, bolt holes and rocker arm locations. This data was augmented to include a broad variation in noise and lighting conditions, to form the custom dataset of <300,000 images. The methods for data generation and annotation are available in the OpenDR toolkit (https://github.com/opendr-eu/opendr).

Technical specifications

RGB camera (e.g. Intel Realsense D435) connected to a workstation, running Ubuntu 20.04. Framerate of the camera ranges between 30-60 Hz, depending on the image resolution. ROS (Noetic or Foxy) is used as interface for communication. The OpenDR toolkit provides the tool and the interfacing to ROS. Other cameras will work, as long as the images are published on the ROS network. A demo example can be found here (non-ROS): https://github.com/opendr-eu/opendr/tree/master/projects/opendr_ws/src/perception#grasp-pose-detection-ros-node

Inputs and outputs

The annotated image stream can be viewed using rqt_image_view, and the default topic name is /opendr/image_boxes_annotated. The bounding boxes alone are also published as /opendr/objects.

Formats and standards

Linux Ubuntu 20.04 LTS

ROS1 Noetic or ROS2 Foxy, OpenDR v2.0,  pytorch 1.9, Python 3.8

All dependencies will be installed with the OpenDR toolkit and/or installation instructions are provided

Owner (organization)

Tampere University