End-to-end Automatic Handling of Small Packages

  • BASIC INFORMATION
  • INNOVATION
  • EXPLOITATION
  • MEDIA
  • MODULES
Name of demonstration

End-to-end Automatic Handling of Small Packages

Main objective

The demonstrator provides a complete robotized sorting solution for logistics. The system localizes and picks each parcel post from a large bin, weighs it, measures its volume, and sorts it into different outgoing containers according to their destination. The implemented solution is designed for easy integration into existing production lines.

Short description

The developed system is a complete robotized pick and place solution for logistics. It is composed of several modules. A vision system based that exploits neural networks is used to identify and localize packages from the inbound container. A grasping synthesis module identifies the best parcel to be picked up and computes the optimal grasping point. A motion planner module generates collision-free trajectories to perform a pick and place routine to sorting the package in the correct outbound container according to the parcels tracking system.

The implemented solution is designed for easy integration into existing production lines and is safe to use without safety cages.

Owner of the demonstrator

IT+Robotics srl

Responsible person

Dr. Nicola Castaman

nicola.castaman@it-robotics.it

NACE

H53 - Postal and courier activities

Keywords

Robotics, Vision System, Machine Learning, Motion Planning, Logistics.

Benefits for the users

Parameter-less configuration: the recognition algorithm moves to a data-driven approach, allowing even untrained personnel to provide meaningful information to the application. Whenever the application does not recognize well the object of interest, it just needs to be fed with some images of the desired object to achieve better results, there is no need to fix/change dozens of parameters.

 

Reduction of human errors: the automatized system developed in the demonstrator reduces the number of sorting errors reducing costs and increasing customers satisfaction.

 

Easy deployment: using a collaborative robot and a vision system, the system can be easily integrated into current production lines without safety cages and requiring less free space.

 

Healthier workplace: operators are no more subjected to perform repetitive operations and stressful cycle time.

Innovation

The main innovation of the workcell is the machine-learning-based vision system to recognizing and localize parcels for a bin-picking application. Typically, industrial bin-picking applications exploit a CAD of the objects to perform the localization and compute a suitable grasping point. Unfortunately, this approach is not feasible with packages and parcels because they have different sizes, colors, shapes, and could be deformed.

The demonstrator exploits a model-less approach to bin-picking that enables an application to detect boxes and parcels in the inbound container. The approach first exploits state-of-the-art machine learning frameworks to recognize and segment packages in a color image of the inbound container. Then an internally developed algorithm merges the segmented instances and the relative point cloud to find a 3D cluster for each package. Finally, the best points for the robot to grasp the desired package are computed.

This approach lets a potential user to setup a pick and places application with a very small effort, at least from the recognition point of view: no model of the object to pick needs to be defined. The recognition algorithm moves from a parameter-based approach to a data-driven approach, allowing even untrained personnel to provide meaningful information to the application.

Risks and limitations

Wrong object identification: as the localization module is based on a machine-learning model, there are chances that recognition is not correctly performed on the object of interest. Localization accuracy: lights can disturb the depth sensor and cause point cloud to be not dense enough for accurate localization. In this case it is necessary to structure the workcell to reduce the noise source or use different vision systems technologies. Motion planner failure: the motion planner may fail when the objects are placed inside complex containers. Software Malfunction: As the entire system operates by a computer, there are chances that software and connections through devices get malfunctioned. It requires a back-up safety system is running all the time and malfunctioning this system results in protective stop of robot system.

Technology readiness level

6 - Safety approved sensors and systems are commercially available

Sectors of application

Express couriers: sorting of boxes or parcels. This automatic sorting system allows to reduce the number of sorting errors., Logistics and Intralogistics: automatic handling, palettizing, and depalletizing of boxes..

Potential sectors of application

The demonstrator uses a machine learning approach that can be generalised localize different objects whose CAD is not available. For this reason, the system is adaptable for sectors like retail, food, manufacturing and others. Food industry: manipulate fresh food, e.g., fruits, vegetables, etc. Such a solution could increase hygiene level. Manufacturing: bin picking application of model-less or deformable objects which shape, or colours are irregular. E-commerce: localize and manipulate heterogeneous type of objects placed in containers or on a conveyor.

Patents / Licenses / Copyrights
Hardware / Software

Hardware:

Collaborative Robot (Universal Robots UR10)

RGB-D Vision System (ENSENSO N35 + IDS RGB camera)

Control PC


Software:

ROS

PCL

OpenCV

PyTorch

Photos
The main system modules and the data flow between them. The package tracking system act as an orchestrator and asks the robotic system to perform actions.
Video

The EACHPack project provides a complete robotic sorting solution for logistics. The system localizes and picks up both randomly placed and palletized parcels. The parcels are then weighed and sorted into different outbound containers based on their destination. The implemented solution is designed for easy integration into existing production lines.

https://youtu.be/9F-GkIM1PKo
Back to LIST
Subscribe to newsletter

Please subscribe to our mailing list to be the first one informed about the new open calls, events and other hot news!


    Please check if the email you entered is correct
    All done!

    Your have successfully subscribed to Trinity newsletter.