Nowadays, cobots can work together with humans in logistics or production without protective fences. Human-robot collaboration (HRC) is subject to strict standards and rules to ensure the safety of humans at all times. As a result, common HRC systems have to operate at low, reduced speeds or even stop completely when a human approaches.

The goal of the KI4MRK research project, funded by the German “Bundesministerium für Bildung und Forschung”, was to develop human motion prediction using the combination of three deep neural networks (NN). For this purpose, the joint workspace was transformed into a block representation (voxel representation). This allows obstacles to be represented as volumes. Using autoencoders, human poses were preprocessed in such a way that they can be efficiently stored in the system. A second autoencoder is trained using public motion databases, allowing prediction of individual motions. In the final step, a recurrent neural network trained with only a small amount of task-specific data thanks to long short-term memory (LSTM) can still predict complex actions. The developed AI-based motion prediction of actions in voxel space, combined with dynamic task scheduling, allows more efficient design of HRC-systems.

Furthermore, they can be used more effectively and economically by minimizing stop times. After having successfully evaluated this motion prediction technique using two demonstrators last year, we look forward to exploring the further procedural indications of this method.

The results we were able to achieve together with our partners are summarized in this video: