The ultimate goal of our research is to develop provably safe autonomous robotic systems that can adapt to and interact with the world in the way that human beings do, so that they can better serve, assist, and collaborate with people in their daily lives across work, home, and leisure. Our current research focuses on advanced robotics for manufacturing toward Manufacturing 5.0. Our fundamental research question is: how do we design the behaviors of these robots, and verify the safety of the design, so that they may operate safely, agilely, and interactively in dynamic, uncertain, and human-involved environments?

Broadly speaking, our research efforts are devoted to deepening the level of autonomy of robotic systems. Although robots are acting autonomously in many situations (e.g., welding robots in production lines), their autonomy hardly goes beyond the operational design domain (e.g., a robot dog can hardly navigate in a floating boat if it is not trained for it). Re-purposing those robots requires significant human effort in terms of collecting data in new environments and tuning the robot behavior to meet the specifications.

In the manufacturing world, people doing these tasks are called integrators and they are much more expensive than the robot itself. Our vision for future robotics is that with increased autonomy levels, future robots can challenge the underlying assumptions they were designed or trained with, mitigate the wrong assumptions, continuously improve themselves during the interaction with their environments, and effectively communicate their needs with humans they interact with.

Our research philosophy is to leverage mathematical and formal analysis to address real-world problems. Our team has been tackling our fundamental research question by creating formal mathematical models and developing novel algorithms to ensure that our robots are agile and provably safe for safety-critical applications. To support our vision for future robotics, we plan to work on the following topics:

  • developing new theory and algorithms for lifelong safety assurance of robotics systems in changing environments;
  • developing new theory and algorithms to equip the robots with the ability to efficiently learn agile skills to perform various manipulation and locomotion tasks;
  • developing new hardware-software integrated solutions to enhance interactivity between robots and humans.

The first pillar of our research is to design efficient algorithms to formally synthesize and verify robot controllers in changing environments (e.g., changing tasks, changing human subjects, etc). We consider the synthesis and verification of both the control policy and the control certificate with respect to specifications encoded as state-wise constraints (e.g., for collision avoidance) or more complicated temporal logic constraints.

Accomplishments

As part of our NSF CAREER project, we have developed a series of methods to synthesize safety controllers and safety certificates with respect to state-wise constraints. Building upon Changliu’s PhD work on safe set algorithms (Liu & Tomizuka, 2014), (Liu & Tomizuka, 2015), we studied a series of safety index synthesis (SIS) methods. A safety index is a scalar function that evaluates the safety potential of a state considering various uncertainties and constraints, which can be viewed as a nonlinear variant of high-order control barrier functions, and the difference between the two is discussed in (Wei & Liu, 2019). A well-synthesized safety index can be used to constrain the robot action to ensure satisfaction of state-wise safety constraints, resulting in either explicit safe controllers (Ma et al., 2022) or implicit safe controllers (e.g., projection-based safety monitor such as CBF-QP).

We started with low-dimensional analytical parameterization of the safety index and studied rule-based approaches (Zhao et al., 2021), (Zhao et al., 2023), evolutionary optimization-based approaches (Wei & Liu, 2022), (Wei et al., 2022), constrained reinforcement learning-based approaches (Ma et al., 2022), as well as sum-of-square programming-based approaches (Zhao et al., 2023), (Chen et al., 2024) for SIS. These works enabled us to safely control robots to interact with humans in close proximity (Liu et al., 2022). These approaches come with strong guarantees (mostly due to the low-dimensional parameterization), at the cost of limited expressiveness (or degree of freedom), hence may result in conservative robot actions.

We are currently investigating a broader class of safety indices that are encoded in deep neural networks. We have introduced the first neural safety index synthesis approach using adversarial optimization that scales beyond 10 dimension problems (Liu et al., 2022) and that results in less conservative safe control. Nevertheless, like all other neural network applications, this approach does not come with any guarantee, and hence necessitates the study of formal synthesis and formal verification methods.

In addition, we started to study safety index adaptation (SIA) under changing environmental conditions, e.g., a robot dog suddenly carries a heavy object and hence cannot turn as quickly as it could at obstacles. We introduced the first real-time adaptation method using gradient ascent on a sum-of-square synthesized safety index (Chen et al., 2024).

What’s Next?

Neural safety indices and neural safety controllers not only can achieve better expressiveness than their analytical counterparts, but also are more agnostic to the system dynamics (e.g., the system does not need to satisfy continuity assumptions) and the system specifications (e.g., it can generalize from state-wise constraints to temporal logic constraints). And with the development of meta-learning and continual learning methods, we hypothesize that they could also be easier to adapt under changing environments. Building upon our prior work, we are tackling the following unsolved problems from the perspectives of both formal synthesis (e.g., synthesis with guarantees) and formal verification (e.g., generating mathematical proofs):

  • How to obtain provably safe neural safety indices and neural safety controllers with respect to diverse forms of specifications?
  • How to adapt neural safety indices and neural safety controllers and preserve the guarantees?

The resulting algorithms and tools will all be evaluated and integrated into the GUARD benchmark that our team developed. GUARD is the state-of-the-art safe learning and control benchmark which contains a variety of robot embodiment, tasks, safety specifications, and implementation of safe learning and control algorithms (Zhao et al., 2024).

The second pillar of our research is to design efficient algorithms to enable robots to learn agile manipulation and locomotion skills so that they can better serve, assist, and collaborate with people. Our argument is that the best robot embodiment for humans to work with is a humanoid robot, since humans already have rich experience working with another human.
With the huge advancements in hardware development for humanoid robots in the past years, it is the perfect time to study human-humanoid interaction and collaboration. Nevertheless, before humanoid robots can seriously work with humans, they need to be made agile in both manipulation and locomotion.

Accomplishments

We have investigated convex optimization-based trajectory generation for manipulation and navigation (Liu et al., 2018; Liu & Tomizuka, 2017), which enables real-time robot motion planning in complex environments. However, these methods require complete models of the environments and do not work well in contact-rich tasks (e.g., delicate assembly) as contact dynamics are hard to model. We then studied learning-based approaches to deal with contact-rich manipulation and locomotion tasks. Facing the huge gap between simulated dynamics and real-world dynamics, we explored two routes:

  • Sample-efficient learning directly applied on hardware.
  • Reinforcement learning with sim-to-real transfer.

For the first route, we achieved several successes in manipulation tasks, in particular, electronic assembly (Chen et al., 2022) and Lego assembly (Liu et al., 2024). We leveraged hardware design (e.g., special fingers) and model-based control techniques to reduce the number of parameters to learn and then leveraged evolutionary optimization to directly optimize those parameters on hardware. For the second route, we achieved several successes in locomotion tasks, enabling robot dogs to run safely and fast (almost approaching hardware limits) (He et al., 2024) and humanoids to track human motions in real-time (missing reference).

The aforementioned work demonstrates significant progress in real-world robot learning by leveraging innovative interface design (e.g., state representation selection), reward engineering, and hyperparameter tuning to effectively adapt existing learning algorithms (e.g., CMA-ES in the first route and PPO in the second route). While these approaches highlight the importance of practical applications, they also reveal opportunities for advancing the field.

As real-world robot learning problems grow more complex, we see a compelling need for fundamental algorithmic innovations to enhance learning efficiency, adapt to changing environments, and reduce the reliance on extensive manual effort in deployment (e.g., hyperparameter tuning and reward engineering). These advancements will drive the next generation of robot learning systems.

What’s Next

Our goal is to develop a holistic theoretic foundation and new reinforcement learning algorithms that mimic human learning, while fully accounting for the problem specifics of manipulation and locomotion tasks, e.g., they are both contact-rich and contain hybrid dynamics. While this is a huge topic, our investigation starts with the following two aspects:

  • How to learn with lower variance and higher success rate by best utilizing the data?

Policy-optimization-based reinforcement learning often suffers from high variance during training, which hinders skill mastery. To address this, we propose optimizing the lower probability bound of performance, approximated by the expected return minus a scaled variance. We recently developed the Absolution Policy Gradient and its variant, Proximal Absolution Policy Gradient (Zhao et al., 2024), which improve performance (high average and low variance) and learning efficiency across RL benchmarks, including humanoid robots. The next step is to incorporate safety constraints using state-wise constrained policy optimization (Zhao et al., 2023) to address real-world robot learning challenges.

  • How to efficiently learn and evolve in changing environments?

Human learning is continuous, and robots should also be capable of learning new skills progressively and composing them to handle increasingly complex tasks. To achieve lifelong learning, we need algorithms inspired by the human hippocampus, which plays a crucial role in learning and memory. We have begun applying this idea, introducing long-term and short-term memory for continual learning and adaptation, to tasks such as simultaneous location and mapping (Yin et al., 2023) and human motion prediction (Abuduweili & Liu, 2021; Abuduweili & Liu, 2023), with promising results. Our current focus is on integrating different learning methods (e.g., offline reinforcement learning, imitation learning, and online reinforcement learning) and balancing memory retention and forgetting for lifelong learning.

The third pillar of our research is to design novel hardware-software integrated solutions to enhance human-robot interaction, so as to lower the barrier for people to program robots and benefit from the technology. We focus on two types of enhancements: better communication (to foster mutual understanding between the human and the robot) and better co-adaptation (to make the robot system evolve together with the human).

Accomplishments

We have extensively studied methods to understand and predict humans by observing the physical motion of the human (Liu & Liu, 2021) (Liu et al., 2023); as well as how to use that to shape a robot’s control strategies considering the fact that the robot’s physical motion will affect the human’s future responses (Pandya & Liu, 2022) (Pandya et al., 2024).

In observance of the limited bandwidth of physical motion-based information exchange, we also investigated dynamic gesture-based (Chen et al., 2023), force-feedback-based (Shek et al., 2023), touch-based (Su et al., 2023), and language-based (Luo et al., 2023) human-to-robot communication.

Nevertheless, as robots are gaining higher levels of intelligence, their decisions may become increasingly complex for humans to understand. Our argument is that in addition to studying how to let robots better understand humans, it is critical to enhance robot-to-human communication so as to foster co-adaptation and improve both the safety and agility of the interaction.

What’s Next?

Our goal is to make human-robot interaction (e.g., human-humanoid interaction) as natural as human-human interaction. While it is a broad topic, we are currently focusing on the following two directions:

  • How to best exchange information between humans and robots? And what should be communicated?

  • How to infer and influence hidden states in a human’s mind?

We aim to improve two-way communication through hardware integration and algorithm design. For hardware, we are developing multi-modal systems (e.g., speech, touch, gestures, VR/AR) to enable high-bandwidth, low-cognitive-load interactions. Our recent work OmniH2O (He et al., 2024) demonstrates a flexible multi-modal tele-operation system for humanoid robots. On the algorithm side, we focus on optimizing robot-to-human communication strategies (e.g., what, when, and how to communicate). Preliminary work on strategy explanation has significantly improved collaboration efficiency (Pandya et al., 2024). We are also exploring visual communication strategies for safety certificates to foster trust and enabling robots to explain both concrete (e.g., next actions) and abstract concepts (e.g., safety rationale).

With the overall objective of deepening the level of autonomy of robotic systems, our team is exploring the following use cases in the manufacturing domain, leveraging our fundamental research discussed above.

Intelligent Design and Prototyping

We aim to use generative AI to facilitate production design and prototyping. The key challenge is how to ensure the design is aligned with human preference and physics. We are investigating novel algorithms to enhance interactivity and alignment between human designers and generative AI in a variety of assembly design tasks. Read more in Generative Assembly via Bimanual Manipulation.

Delicate Assembly

Delicate assembly, e.g., electronic assembly (Chen et al., 2022), Lego assembly (Liu et al., 2024), and cable assembly, is still heavily done manually. The key challenge for robots to perform these assembly tasks is the lack of agility, e.g., how to coordinate two arms to finish the assembly task (such as one arm for support and one arm for manipulation), how to achieve high precision in the assembly, and how to generalize the assembly skills to arbitrary components. We are investigating novel learning and planning algorithms for intelligent dual-arm delicate assembly. Read more in Generative Assembly via Bimanual Manipulation and 6DoF Robot Assembly Station of Consumer Electronic Production.

Intelligent Surface Finishing

Surface finishing jobs, such as grinding and weld washing, are in high demand in almost all manufacturing sectors. These jobs are challenging for robots as there could be huge variances across different workpieces even in the same batch. Experienced human workers need to make many nontrivial decisions in real time to finish the job. However, with the loss of qualified human workers, there is a pressing need to automate these surface finishing tasks. Our team started tackling the problem in collaboration with several industrial partners in 2019. By integrating our novel safe control and agile compliance control methods (Zhao et al., 2020) (He et al., 2023) (missing reference), we were able to demonstrate a fully autonomous robotic solution for weld washing in the real world. This work is currently being commercialized through Instinct Robotics. Read more in Automatic Onsite Polishing of Large Complex Surfaces by Real Time Planning and Control.

Autonomous Deployment and Maintenance of Robot Systems

We have been fascinated by the idea of containerized manufacturing, where we compress the whole supply chain and put it into one shipping container, so that raw materials get in from one end and packaged products come out from the other. The shipping container can be delivered anywhere on earth, thus minimizing the risks associated with long supply chains. Inside the container, there need to be machines and robots, where the robots mostly handle the loading and packaging tasks. To realize this vision of containerized manufacturing, we need to equip the robot with strong self-awareness and decision-making capabilities, as it is almost impossible to send human integrators to debug and repair the robot once the container is shipped. We are collaborating with our industrial partners to deploy a containerized line for mask production, where our team focuses on enabling self-calibration and real-time decision-making capabilities of the robots. Additionally, we are investigating methods that leverage foundation models to facilitate error detection and recovery for these systems. One of our recent works, Meta-Control (Wei et al., 2024), enables foundation models to replace human integrators in the deployment of robot control systems using Socrates’ “art of midwifery” and model-based grounding techniques. This offers a great way to deepen the level of autonomy while maintaining high system performance.

Human-Robot Collaboration for Fixtureless Manufacturing

With the rise of high-mix, low-volume manufacturing, it is not economical to make expensive fixtures that can only be used for one product. Robots with dexterous grippers can be used to replace fixtures as they can hold all kinds of workpieces for humans to work on. Our team is working with our industrial partners to explore this idea on delicate assembly tasks. While there are many challenges, by actively predicting human intent and displaying robot intent back to the human, we were able to improve the efficiency of human-robot collaboration in these fixtureless production lines (Liu et al., 2023). Read more in Safe Uncaged Industrial Robots.

  1. [C2] Control in a safe set: Addressing safety in human-robot interactions
    Changliu Liu and Masayoshi Tomizuka
    Dynamic Systems and Control Conference, 2014
    Best Student Paper Finalist
  2. [C3] Safe exploration: Addressing various uncertainty levels in human robot interactions
    Changliu Liu and Masayoshi Tomizuka
    American Control Conference, 2015
  3. [J2] Real time trajectory optimization for nonlinear robotic systems: Relaxation and convexification
    Changliu Liu and Masayoshi Tomizuka
    Systems & Control Letters, 2017
  4. [J3] The convex feasible set algorithm for real time optimization in motion planning
    Changliu Liu, Chung-Yen Lin and Masayoshi Tomizuka
    SIAM Journal on Control and optimization, 2018
  5. [C23] Safe Control Algorithms Using Energy Functions: A Unified Framework, Benchmark, and New Directions
    Tianhao Wei and Changliu Liu
    IEEE Conference on Decision and Control, 2019
  6. [C30] Contact-Rich Trajectory Generation in Confined Environments Using Iterative Convex Optimization
    Wei-Ye Zhao, Suqin He, Chengtao Wen and Changliu Liu
    Dynamic Systems and Control Conference, 2020
  7. [C38] Model-free Safe Control for Zero-Violation Reinforcement Learning
    Weiye Zhao, Tairan He and Changliu Liu
    Conference on Robot Learning, 2021
  8. [J7] Robust nonlinear adaptation algorithms for multitask prediction networks
    Abulikemu Abuduweili and Changliu Liu
    International Journal of Adaptive Control and Signal Processing, 2021
  9. [J8] Human Motion Prediction Using Adaptable Recurrent Neural Networks and Inverse Kinematics
    Ruixuan Liu and Changliu Liu
    IEEE Control Systems Letters, 2021
  10. [C40] Safe Control with Neural Network Dynamic Models
    Tianhao Wei and Changliu Liu
    Learning for Dynamics and Control Conference, 2022
  11. [C41] Joint Synthesis of Safety Certificate and Safe Control Policy Using Constrained Reinforcement Learning
    Haitong Ma, Changliu Liu, Shengbo Eben Li, Sifa Zheng and Jianyu Chen
    Learning for Dynamics and Control Conference, 2022
    Best Paper Finalist
  12. [C44] Jerk-bounded Position Controller with Real-Time Task Modification for Interactive Industrial Robots
    Ruixuan Liu, Rui Chen, Yifan Sun, Yu Zhao and Changliu Liu
    IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2022
  13. [C45] A Composable Framework for Policy Design, Learning, and Transfer Toward Safe and Efficient Industrial Insertion
    Rui Chen, Chenxi Wang, Tianhao Wei and Changliu Liu
    IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022
  14. [C46] Safe and Efficient Exploration of Human Models During Human-Robot Interaction
    Ravi Pandya and Changliu Liu
    IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022
  15. [C53] Safe Control Under Input Limits with Neural Control Barrier Functions
    Simin Liu, Changliu Liu and John Dolan
    Conference on Robot Learning, 2022
  16. [J15] Persistently feasible robust safe control by safety index synthesis and convex semi-infinite programming
    Tianhao Wei, Shucheng Kang, Weiye Zhao and Changliu Liu
    IEEE Control Systems Letters, 2022
  17. [J16] A hierarchical long short term safety framework for efficient robot manipulation under uncertainty
    Suqin He, Weiye Zhao, Chuxiong Hu, Yu Zhu and Changliu Liu
    Robotics and Computer-Integrated Manufacturing, 2023
  18. [C55] Learning from physical human feedback: An object-centric one-shot adaptation method
    Alvin Shek, Bo Ying Su, Rui Chen and Changliu Liu
    IEEE International Conference on Robotics and Automation, 2023
    Outstanding Interaction Paper
  19. [C56] Safety index synthesis via sum-of-squares programming
    Weiye Zhao, Tairan He, Tianhao Wei, Simin Liu and Changliu Liu
    American Control Conference, 2023
  20. [C57] Probabilistic safeguard for reinforcement learning using safety index guided gaussian process models
    Weiye Zhao, Tairan He and Changliu Liu
    Learning for Dynamics and Control Conference, 2023
  21. [C60] State-wise safe reinforcement learning: A survey
    Weiye Zhao, Tairan He, Rui Chen, Tianhao Wei and Changliu Liu
    International Joint Conferences on Artificial Intelligence, 2023
  22. [C62] Proactive human-robot co-assembly: Leveraging human intention prediction and robust safe control
    Ruixuan Liu, Rui Chen, Abulikemu Abuduweili and Changliu Liu
    IEEE Conference on Control Technology and Applications, 2023
  23. [C65] Online Model Adaptation with Feedforward Compensation
    Abulikemu Abuduweili and Changliu Liu
    Conference on Robot Learning, 2023
  24. [J17] Robust and context-aware real-time collaborative robot handling via dynamic gesture commands
    Rui Chen, Alvin Shek and Changliu Liu
    IEEE Robotics and Automation Letters, 2023
  25. [J19] Customizing Textile and Tactile Skins for Interactive Industrial Robots
    Bo Ying Su, Zhongqi Wei, James McCann, Wenzhen Yuan and Changliu Liu
    ASME Letters in Dynamic Systems and Control, 2023
  26. [J20] Bioslam: A bioinspired lifelong memory system for general place recognition
    Peng Yin, Abulikemu Abuduweili, Shiqi Zhao, Lingyun Xu, Changliu Liu and Sebastian Scherer
    IEEE Transactions on Robotics, 2023
  27. [W] Obtaining hierarchy from human instructions: an llms-based approach
    Xusheng Luo, Shaojun Xu and Changliu Liu
    CoRL 2023 Workshop on Learning Effective Abstractions for Planning (LEAP), 2023
  28. [J23] Guard: A safe reinforcement learning benchmark
    Weiye Zhao, Rui Chen, Yifan Sun, Ruixuan Liu, Tianhao Wei and Changliu Liu
    Transactions on Machine Learning Research, 2024
  29. [C67] Safety Index Synthesis with State-dependent Control Space
    Rui Chen, Weiye Zhao and Changliu Liu
    American Control Conference, 2024
  30. [C70] Real-time Safety Index Adaptation for Parameter-varying Systems via Determinant Gradient Ascend
    Rui Chen, Weiye Zhao, Ruixuan Liu, Weiyang Zhang and Changliu Liu
    American control Conference, 2024
  31. [C72] Towards Proactive Safe Human-Robot Collaborations via Data-Efficient Conditional Behavior Prediction
    Ravi Pandya, Zhuoyuan Wang, Yorie Nakahira and Changliu Liu
    IEEE International Conference on Robotics and Automation, 2024
  32. [C73] Multi-Agent Strategy Explanations for Human-Robot Collaboration
    Ravi Pandya, Michelle Zhao, Changliu Liu, Reid Simmons and Henny Admoni
    IEEE International Conference on Robotics and Automation, 2024
  33. [C76] A Lightweight and Transferable Design for Robust LEGO Manipulation
    Ruixuan Liu, Yifan Sun and Changliu Liu
    International Symposium of Flexible Automation, 2024
  34. [C77] Absolute Policy Optimization: Enhancing Lower Probability Bound of Performance with High Confidence
    Weiye Zhao, Feihan Li, Yifan Sun, Rui Chen, Tianhao Wei and Changliu Liu
    International Conference on Machine Learning, 2024
  35. [C78] Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion
    Tairan He, Chong Zhang, Wenli Xiao, Guanqi He, Changliu Liu and Guanya Shi
    Robotics: Science and Systems, 2024
    Outstanding Student Paper Award Finalist
  36. [C82] OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
    Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu and Guanya Shi
    Conference on Robot Learning, 2024
  37. [C83] Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skills
    Tianhao Wei, Liqian Ma, Rui Chen, Weiye Zhao and Changliu Liu
    Conference on Robot Learning, 2024

Period of Performance: Now ~ Now