Overview

Robots are used in existing production lines to automate manufacturing processes. However, contemporary manufacturing not only wants mass production, but also masscustomization to satisfy different needs from the customers. Therefore, it is desired to combine the productivity by robots, and the flexibility from humans to improve mass customization.

Our research develops a real-time perception-action pipeline for safe and efficient human-robot collaboration (HRC), and demonstrates a successful integration of numerous fields, such as machine learning, robotics, human-robot interaction, etc. The developed solution implements multiple techniques, including adatable human motion/intention prediction, robust task planning, and safe robot control. The experiment demonstrates that our solution can improve the task efficiency by actively collaborating with humans as well as guarantee safety during collaboration.

Experiment Setup

Inspired by real-world assembly tasks, we designed a simplified co-assembly task to demonstrate our solution. In our application, the assembly task requires the human to insert blocks with different shapes into a cube container via holes on different sides (shown in the below figure). It is assumed that the container is heavy, so that the human cannot directly manipulate it. Therefore, it is desired to have a robot holding the container and display the corresponding surface to the human for easy insertion.

Trulli — Illustration of the experiment. Left: blocks to be inserted. Right: the container.

Demo

From our experiment, we demonstrate that our solution is

Robust to different human preferences.

In real-world assembly tasks, there is flexitility for the human worker to lead and decide based on his/her preference. In our task, the worker can choose the order of insertion. Therefore, the robot should accommodate the human preference in order to achieve efficient collaboration. These three videos show the human inserting blocks with completely different orders. The robot can successfully identify the difference and collaborate with the human smoothly.

Robust to different environment setups.

In real-world applications, the environment is highly likely to change from task to task. For example in our task, the blocks can be placed in random orders on the table. Therefore, the robot should accommodate the changing environment in order to achieve efficient collaboration. These three videos demonstrate the robot robustly collaborating with the human despite the changing environment.

Robust to different human users.

It is highly possible that the robot needs to collaborate with not only a single user, but multiple different users in real applications. Therefore, the robot should also be able to accomodate the different human behaviors. These videos demonstrate the robot can robustly collaborate with different people.

Improves task efficiency.

The goal for HRC is to improve task efficiency. To demonstrate the advantage of our solution, we created a baseline HRC without active collaboration, and compare it with our solution. A naive collaboration is to have the robot completely following human instructions. The right video illustrates the baseline collaboration, where the robot moves when the human presses the keyboard. In this case, the robot passively waits for human command, whereas in our solution, the robot acts intelligently based on its understanding of the human and the task.

Baseline HRC.

In terms of efficiency, we mainly compare the time for completing the task. The experiment includes 5 human subjects, and each human did each task for 2 trials. The above table displays the average time for completing the entire task (green), completing one surface (yellow), and inserting one single block (red). The result indicates that by active collaboration, our solution improves the task efficiency.

Ensures interactive safety.

It is important to guarantee safety during collaboration since any collision could cause severe consequences. At any given time, the robot should not collide with any human body parts that are not involved in the collaboration. In our task, we assume the humans are right handed. Therefore, the system allows close contact by the right hand for assembly (block insertion) while avoids other body parts.

To ensure safety, our pipeline constantly monitors the environment and adjusts the robot behavior accordingly. As shown in the left videos, as the human gets closer to the robot, the robot moves away to avoid collision with the human’s head and body. After the human gets the block and moves out of the way, the robot moves back and allows the human to insert the block using his right hand.