HRI Demo for President Biden

During President Biden’s visit, we demonstrated our work on human-robot collaboration.

Our research is to build intelligent robots that interact with the world like humans so that they can better assist and collaborate with people. While there are various tasks that humans and robots can collaborate in a manufacturing environment, we set up a FANUC industrial robot to demonstrate human-robot handover. These kinds of tasks arise naturally when the human needs the robot’s assistance to fetch an object that is either out-of-reach, too heavy, or in some hazardous environment. Ruixuan Liu and Rui Chen participated in this demo.

There are two major challenges in this handover task: 1. safety, which is how to make sure the powerful robot does not hurt humans; 2. efficiency, that is how to correctly interpret the human’s intention to better assist the human. We developed intelligent perception, AI, and control software for the robot to achieve both safety and efficiency. The software contains a digital twin which tracks and supervises the real-time human and robot behaviors via a camera. The real-time distance and momentum between the human body and the robot arm are monitored. When the human gets close, a safety controller will be activated to move the robot away from the human. In this way, safety is guaranteed even when the robot is in idle mode. In terms of efficiency, when the robot sees the human’s palm up, it quickly infers that the human needs the screwdriver. Then the robot locates the screwdriver, picks it up, and hands it to the human. After the human finishes the task, he hands the screwdriver back to the robot and the robot then places it to the correct location for efficient tool management. This handover loop can be repeated as many times as the human desires without external reset operations. A pre-recorded video of the demo is shown below.

President Biden was very interested in this idea and asked how to best integrate this kind of method in real manufacturing environments. We explained that this can be used to support multi-purpose assembly lines where the human workers can rely on the robot to manage all the tools. We are collaborating with Chengtao Wen – a Senior Research Scientist at Siemens in the hopes of making the transition from prototype to fully commercialized automatic tool management for flexible assembly lines a seamless one.

Trulli — **Explaining the handover demo to President Biden**

We would like to acknowledge the support from FANUC, Siemens, and Ford in this research. FANUC generously donated the robot arm to our lab. Siemens supported this research through a sponsored project “Task agnostic real-time perception and control with few-shot cross-platform adaptation”. And Ford supported this research through a University Research Program Award on “Safe uncaged industrial robots”.