Benefits for the users

Collaborative assembly reduction of 30%: Collaborative assembly has the potential to considerably reduce assembly times, whilst ensuring, reduction in human errors by inspecting the final product quality. They may also increase workplace ergonomics. Using a mid-range collaborative robot, assembly times can be reduced by, at least, 30%. Collaborative robots may also be used for other non-collaborative tasks, so that, when not used in a collaborative activity, they can solo perform automated tasks to improve ROI.

 

Training-time reduction by 50%: Augmented Reality (AR) based solutions improve knowledge transfer, and reduced training time by 50%. They also improve field service quality and productivity, reducing overtime spend with maintenance operations.

 

Cost savings up to 70%: AR with remote video assistance considerably reduces, the need for travels related to maintenance. 

 

Automated bin-picking with cycle times below 10s: AI machine vision is currently an accessible technology that can be coupled with affordable robotic machines to perform automated tasks of bin-picking with ordered sort, which are applicable in many industry use cases. Mid-range robots can achieve cycle times below 10s, depending on the type of target objects and the robot gripper (clamp or suction).

Innovation

Collaborative assembly with cobots brings better working conditions and higher profitability through improved adaptability, flexibility, performance and seamless workflow integration. Worker safety can be achieved by projecting dynamic safety zones and a safety line on the working area, to ensure the operators keep conscious of the restrictions and regions of operation.

 

Augmented Reality (AR) is an upgradable technology that enhances the user experience by putting holographic images in the user physical environment. These holograms can be interactive and can be easily and intuitively manipulated by the user. The AR holograms can be applicable in many industry use cases.

 

Bin-picking allows for randomly placed parts (in any position and orientation) on a container to be removed with minimum disruption of the environment and put (ordered or not) on multiple containers, where the robot adjust its position and trajectory to extract and place the part in its new position (optionally, with the correct orientation). This solution is very flexible (AI based machine vision, allows reaction to the environment), adaptive (the robot can be rescheduled for other tasks), with minimal supervision, whilst operating in a safe condition (powered by computer vision and sensors).

Risks and limitations

Worker injuries: Any interaction with machines brings hazards, therefore, injuries by cobots is always a possibility, despite all safety measures in-place. Workers must be properly trained for the dangers in human-robot collaborative operations. AR Hologram precision: The precision of the holograms projected by current AR technology is almost excellent and is expected to improve for the next years. Still, different devices and operating systems produce different results on low light and under different materials conditions (e.g., reflective materials). LiDAR technology substantially improves the AR experience but has just been introduced on some commercial-of-the-shelf devices. AR glasses are still intrusive (no seamless integration in human daily activities) and have a limited field-of-view. Tablets and smartphones produce the best AR holograms, but require that, at least, one hand will be used to operate the device. Increased bin-picking cycle time: for more complex scenarios, involving objects that cannot be picked by suction and require a clamp gripper, an expensive depth camera may be required. In such scenarios, object recognition (in different depths), and object pick-up may require additional time.

Technology readiness level

6 - Safety approved sensors and systems are commercially available

Sectors of application

Industries: • Machine vision algorithms to assist automated tasks and perform quality control (visual inspection) of assembled products., Industries: • Augmented reality applications for training-on-the-job and assistance in machine maintenance operations., Industries: • Collaborative robots (cobots) assisted by machine vision and safety systems, for a quicker, more efficient assembly process, also ensuring quality in assembled products by reducing worker fatigue..

Potential sectors of application

• Services: Augmented reality solutions to support technicians in the maintenance operations for many different types of machines (e.g., heating, ventilation and air conditioning (HVAC)); • Construction: Augmented reality solutions to visualize and interact, on-site, with building construction projects (CAD models) or existing building metadata (e.g., details on the construction license), with no need for the user to search databases; • Utilities: Augmented reality solutions to support technicians visualize and interact with buried infrastructure (e.g., water, electricity and gas), with high-precision geo-referenced holograms.

Patents / Licenses / Copyrights

Hardware / Software

Hardware:

OMRON TM5-900 (hardware): collaborative robot for collaborative assembly and bin-picking activities.

Intel Real-Sense camera (hardware): stereoscopic camera to capture image depth information for bin-picking

Android/iOS smartphone or tablet (hardware): for running AR applications

Software:

OpenCV (software): an open source computer vision and machine learning software library.

TMFlow (software): a robot programming environment for the OMRON TM5-900.

Robot Operating System (ROS) and ROS packages (software): a flexible framework for writing robot software and a set of image processing libraries for ROS.

TensorFlow/PyTorch (software): open source platforms for machine learning.

Unity (software): a framework purpose-built for AR development.

Microsoft’s Mixed Reality Toolkit (software): a cross-platform providing foundational components and common building blocks for spatial interactions.

ARKit and/or ARCore (software): AR platforms for iOS and Android devices, respectively. Enable applications to use advanced AR tools and device's capabilities (e.g., cameras and sensors).

MixedReality-WebRTC (software): a collection of libraries that enable the integration of peer-to-peer real-time audio and video communication into AR applications

Video

AGILE-TRINITY promotional video

https://www.youtube.com/watch?v=hHvxpFW64BI