[J39] The Feasibility Theory of Constrained Reinforcement Learning: A Tutorial Study
Yujie Yang, Zhilong Zheng, Masayoshi Tomizuka, Changliu Liu and Shengbo Eben Li
Foundations and Trends in Systems and Control, 2026
Citation Formats:
[C108] From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrails
Ravi Pandya, Madison Bland, Duy P. Nguyen, Changliu Liu, Jaime F. Fisac and Andrea Bajcsy
Second Conference of the International Association for Safe and Ethical AI (IASEAI), 2026
Citation Formats:
[U] Contingency Planning for Safety-Critical Autonomous Vehicles: A Review and Perspectives
Lei Zheng, Luyao Zhang, Peiqi Yu, Yifan Sun, Sergio Grammatico, Jun Ma and Changliu Liu
arXiv:2601.14880, 2026
[J38] Training Task Reasoning LLM Agents for Multi-turn Task Planning via Single-turn Reinforcement Learning Hanjiang Hu, Changliu Liu, Na Li and Yebin Wang
IEEE Control Systems Letters, 2025
Citation Formats:
Abstract:
Large Language Models (LLMs) have demonstrated remarkable capabilities in knowledge acquisition, reasoning, and tool use, making them promising candidates for autonomous agent applications. However, training LLM agents for complex multi-turn task planning faces significant challenges, including sparse episode-wise rewards, credit assignment across long horizons, and the computational overhead of reinforcement learning in multi-turn interaction settings. To this end, this paper introduces a novel approach that transforms multi-turn task planning into single-turn task reasoning problems, enabling efficient policy optimization through Group Relative Policy Optimization (GRPO) with dense and verifiable reward from expert trajectories. Our theoretical analysis shows that GRPO improvement on single-turn task reasoning results in a lower bound of the multi-turn success probability under the minimal turns, as well as the generalization to subtasks with shorter horizons. Experimental evaluation on the complex task planning benchmark demonstrates that our 1.5B parameter model trained with single-turn GRPO achieves superior performance compared to larger baseline models up to 14B parameters, with success rates of 70% for long-horizon planning tasks.
[J37] Verifiable Safety Q-Filters via Hamilton-Jacobi Reachability and Multiplicative Q-Networks Jiaxing Li, Hanjiang Hu, Yujie Yang and Changliu Liu
IEEE Control Systems Letters, 2025
Citation Formats:
Abstract:
Recent learning-based safety filters have outperformed conventional methods, such as hand-crafted Control Barrier Functions (CBFs), by effectively adapting to complex constraints. However, these learning-based approaches lack formal safety guarantees. In this work, we introduce a verifiable model-free safety filter based on Hamilton-Jacobi reachability analysis. Our primary contributions include: 1) extending verifiable self-consistency properties for Q value functions, 2) proposing a multiplicative Q-network structure to mitigate zero-sublevel-set shrinkage issues, and 3) developing a verification pipeline capable of soundly verifying these self-consistency properties. Our proposed approach successfully synthesizes formally verified, model-free safety certificates across four standard safe-control benchmarks.
[J36] Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives
Letian Wang, Marc-Antoine Lavoie, Sandro Papais, Barza Nisar, Yuxiao Chen, Wenhao Ding, Boris Ivanovic, Hao Shao, Abulikemu Abuduweili, Evan Cook, Yang Zhou, Peter Karkus, Jiachen Li, Changliu Liu, Marco Pavone and Steven Waslander
Foundations and Trends in Robotics, 2025
Citation Formats:
Abstract:
Motion prediction, the anticipation of future agent states or scene evolution, is rooted in human cognition, bridging perception and decision-making. It enables intelligent systems, such as robots and self-driving cars, to act safely in dynamic, human-involved environments, and informs broader time-series reasoning challenges. With advances in methods, representations, and datasets, the field has seen rapid progress, reflected in quickly evolving benchmark results. Yet, when state-of-the-art methods are deployed in the real world, they often struggle to generalize to open-world conditions and fall short of deployment standards. This reveals a gap between research benchmarks, which are often idealized or ill-posed, and real-world complexity. To address this gap, this survey revisits the generalization and deployability of motion prediction models, with an emphasis on the applications of robotics, autonomous driving, and human motion. We first offer a comprehensive taxonomy of motion prediction methods, covering representations, modeling strategies, application domains, and evaluation protocols. We then study two key challenges: (1) how to push motion prediction models to be deployable to realistic deployment standards, where motion prediction does not act in a vacuum, but functions as one module of closed-loop autonomy stacks - it takes input from the localization and perception, and informs downstream planning and control. 2) how to generalize motion prediction models from limited seen scenarios/datasets to the open-world settings. Throughout the paper, we highlight critical open challenges to guide future work, aiming to recalibrate the community’s efforts, fostering progress that is not only measurable but also meaningful for real-world applications.
[J35] NL2HLTL2Plan: Scaling Up Natural Language Understanding for Multi-Robots Through Hierarchical Temporal Logic Task Specifications
Shaojun Xu, Xusheng Luo, Yutong Huang, Letian Leng, Ruixuan Liu and Changliu Liu
IEEE Robotics and Automation Letters, 2025
Citation Formats:
Abstract:
To enable non-experts to specify long-horizon, multi-robot collaborative tasks, language models are increasingly used to translate natural language commands into formal specifications. However, because translation can occur in multiple ways, such translations may lack accuracy or lead to inefficient multi-robot planning. Our key insight is that concise hierarchical specifications can simplify planning while remaining straightforward to derive from human instructions. We propose Nl2Hltl2Plan, a framework that translates natural language commands into hierarchical Linear Temporal Logic (LTL) and solves the corresponding planning problem. The translation involves two steps leveraging Large Language Models (LLMs). First, an LLM transforms instructions into a Hierarchical Task Tree, capturing logical and temporal relations. Next, a fine-tuned LLM converts sub-tasks into flat LTL formulas, which are aggregated into hierarchical specifications, with the lowest level corresponding to ordered robot actions. These specifications are then used with off-the-shelf planners. Our Nl2Hltl2Plan demonstrates the potential of LLMs in hierarchical reasoning for multi-robot task planning. Evaluations in simulation and real-world experiments with human participants show that Nl2Hltl2Plan outperforms existing methods, handling more complex instructions while achieving higher success rates and lower costs in task allocation and planning.
[J34] Implicit Safe Set Algorithm for Provably Safe Reinforcement Learning
Weiye Zhao, Feihan Li and Changliu Liu
Journal of Artificial Intelligence Research, 2025
Citation Formats:
Abstract:
Deep reinforcement learning (DRL) has demonstrated remarkable performance in many continuous control tasks. However, a significant obstacle to the real-world application of DRL is the lack of safety guarantees. Although DRL agents can satisfy system safety in expectation through reward shaping, designing agents to consistently meet hard constraints (e.g., safety specifications) at every time step remains a formidable challenge. In contrast, existing work in the field of safe control provides guarantees on persistent satisfaction of hard safety constraints. However, these methods require explicit analytical system dynamics models to synthesize safe control, which are typically inaccessible in DRL settings. In this paper, we present a model-free safe control algorithm, the implicit safe set algorithm, for synthesizing safeguards for DRL agents that ensure provable safety throughout training. The proposed algorithm synthesizes a safety index (barrier certificate) and a subsequent safe control law solely by querying a black-box dynamic function (e.g., a digital twin simulator). Moreover, we theoretically prove that the implicit safe set algorithm guarantees finite time convergence to the safe set and forward invariance for both continuous-time and discrete-time systems. We validate the proposed algorithm on the state-of-the-art Safety Gym benchmark, where it achieves zero safety violations while gaining 95%±9% cumulative reward compared to state-of-the-art safe DRL methods. Furthermore, the resulting algorithm scales well to high-dimensional systems with parallel computing.
[J33] Simultaneous Task Allocation and Planning for Multi-Robots under Hierarchical Temporal Logic Specifications Xusheng Luo and Changliu Liu
IEEE Transaction on Robotics, 2025
Citation Formats:
Abstract:
Research in robotic planning with temporal logic specifications, such as syntactically co-safe Linear Temporal Logic (sc-LTL), has relied on single formulas. However, as task complexity increases, sc-LTL formulas become lengthy, making them difficult to interpret and generate, and straining the computational capacities of planners. To address this, we introduce a hierarchical structure to sc-LTL specifications with both syntax and semantics, proving it to be more expressive than flat counterparts. We conducted a user study that compared the flat sc-LTL with our hierarchical version and found that users could more easily comprehend complex tasks using the hierarchical structure. We develop a search-based approach to synthesize plans for multi-robot systems, achieving simultaneous task allocation and planning. This method approximates the search space by loosely interconnected sub-spaces, each corresponding to an sc-LTL specification. The search primarily focuses on a single sub-space, transitioning to another under conditions determined by the decomposition of automatons. We develop multiple heuristics to significantly expedite the search. Our theoretical analysis, conducted under mild assumptions, addresses completeness and optimality. Compared to existing methods used in various simulators for service tasks, our approach improves planning times while maintaining comparable solution quality.
Video:
[J32] Scalable synthesis of formally verified neural value function for hamilton-jacobi reachability analysis
Yujie Yang, Hanjiang Hu, Tianhao Wei, Shengbo Eben Li and Changliu Liu
Journal of Artificial Intelligence Research, 2025
Citation Formats:
Abstract:
Hamilton-Jacobi (HJ) reachability analysis provides a formal method for guaranteeing safety in constrained control problems. It synthesizes a value function to represent a long-term safe set called feasible region. Early synthesis methods based on state space discretization cannot scale to high-dimensional problems, while recent methods that use neural networks to approximate value functions result in unverifiable feasible regions. To achieve both scalability and verifiability, we propose a framework for synthesizing verified neural value functions for HJ reachability analysis. Our framework consists of three stages: pre-training, adversarial training, and verification-guided training. We design three techniques to address three challenges to improve scalability respectively: boundary-guided backtracking (BGB) to improve counterexample search efficiency, entering state regularization (ESR) to enlarge feasible region, and activation pattern alignment (APA) to accelerate neural network verification. We also provide a neural safety certificate synthesis and verification benchmark called Cersyve-9, which includes nine commonly used safe control tasks and supplements existing neural network verification benchmarks. Our framework successfully synthesizes verified neural value functions on all tasks, and our proposed three techniques exhibit superior scalability and efficiency compared with existing methods.
[J31] Enhancing Sample Generation of Diffusion Models using Noise Level Correction
Abulikemu Abuduweili, Chenyang Yuan, Changliu Liu and Frank Permenter
Transactions on Machine Learning Research, 2025
Citation Formats:
Abstract:
The denoising process of diffusion models can be interpreted as an approximate projection of noisy samples onto the data manifold. Moreover, the noise level in these samples approximates their distance to the underlying manifold. Building on this insight, we propose a novel method to enhance sample generation by aligning the estimated noise level with the true distance of noisy samples to the manifold. Specifically, we introduce a noise level correction network, leveraging a pre-trained denoising network, to refine noise level estimates during the denoising process. Additionally, we extend this approach to various image restoration tasks by integrating task-specific constraints, including inpainting, deblurring, super-resolution, colorization, and compressed sensing. Experimental results demonstrate that our method significantly improves sample quality in both unconstrained and constrained generation scenarios. Notably, the proposed noise level correction framework is compatible with existing denoising schedulers (e.g., DDIM), offering additional performance improvements.
[J30] Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods Xusheng Luo, Tianhao Wei, Simin Liu, Ziwei Wang, Luis Mattei-Mendez, Taylor Loper, Joshua Neighbor, Casidhe Hutchison and Changliu Liu
ACM Transactions on Cyber-Physical Systems, 2025
Citation Formats:
Abstract:
This work addresses the certification of the local robustness of vision-based two-stage 6D object pose estimation. The two-stage method for object pose estimation achieves superior accuracy by first employing deep neural network-driven keypoint regression and then applying a Perspective-n-Point (PnP) technique. Despite advancements, the certification of these methods’ robustness remains scarce. This research aims to fill this gap with a focus on their local robustness on the system level–the capacity to maintain robust estimations amidst semantic input perturbations. The core idea is to transform the certification of local robustness into neural network verification for classification tasks. The challenge is to develop model, input, and output specifications that align with off-the-shelf verification tools. To facilitate verification, we modify the keypoint detection model by substituting nonlinear operations with those more amenable to the verification processes. Instead of injecting random noise into images, as is common, we employ a convex hull representation of images as input specifications to more accurately depict semantic perturbations. Furthermore, by conducting a sensitivity analysis, we propagate the robustness criteria from pose to keypoint accuracy, and then formulating an optimal error threshold allocation problem that allows for the setting of a maximally permissible keypoint deviation thresholds. Viewing each pixel as an individual class, these thresholds result in linear, classification-akin output specifications. Under certain conditions, we demonstrate that the main components of our certification framework are both sound and complete, and validate its effects through extensive evaluations on realistic perturbations. To our knowledge, this is the first study to certify the robustness of large-scale, keypoint-based pose estimation given images in real-world scenarios.
[J29] Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking Ruixuan Liu, Alan Chen, Weiye Zhao and Changliu Liu
IEEE Robotics and Automation Letters, 2025
Citation Formats:
Abstract:
Combinatorial assembly uses standardized unit primitives to build objects that satisfy user specifications. This paper studies assembly sequence planning (ASP) for physical combinatorial assembly. Given the shape of the desired object, the goal is to find a sequence of actions for placing unit primitives to build the target object. In particular, we aim to ensure the planned assembly sequence is physically executable. However, ASP for combinatorial assembly is particularly challenging due to its combinatorial nature. To address the challenge, we employ deep reinforcement learning to learn a construction policy for placing unit primitives sequentially to build the desired object. Specifically, we design an online physics-aware action mask that filters out invalid actions, which effectively guides policy learning and ensures violation-free deployment. In the end, we apply the proposed method to Lego assembly with more than 250 3D structures. The experiment results demonstrate that the proposed method plans physically valid assembly sequences to build all structures, achieving a 100% success rate, whereas the best comparable baseline fails more than 40 structures.
Video:
Teaser:
[C99] Hierarchical Temporal Logic Task and Motion Planning for Multi-Robot Systems
Zhongqi Wei, Xusheng Luo and Changliu Liu
Robotics: Science and Systems, 2025
Citation Formats:
Abstract:
Task and motion planning (TAMP) for multi-robot systems, which integrates discrete task planning with continuous motion planning, remains a challenging problem in robotics. Existing TAMP approaches often struggle to scale effectively for multi-robot systems with complex specifications, leading to infeasible solutions and prolonged computation times. This work addresses the TAMP problem in multi-robot settings where tasks are specified using expressive hierarchical temporal logic and task assignments are not pre-determined. Our approach leverages the efficiency of hierarchical temporal logic specifications for task-level planning and the optimization-based graph of convex sets method for motion-level planning, integrating them within a product graph framework. At the task level, we convert hierarchi- cal temporal logic specifications into a single graph, embedding task allocation within its edges. At the motion level, we represent the feasible motions of multiple robots through convex sets in the configuration space, guided by a sampling-based motion planner. This formulation allows us to define the TAMP problem as a shortest path search within the product graph, where efficient convex optimization techniques can be applied. We prove that our approach is both sound and complete under mild assumptions. To enhance scalability, we introduce a pruning heuristic that reduces the product graph size, enabling efficient planning for high-dimensional multi-robot systems. Additionally, we extend our framework to cooperative pick-and-place tasks involving object handovers between robots. We evaluate our method across various high-dimensional multi-robot scenarios, including simulated and real-world environments with quadrupeds, robotic arms, and automated conveyor systems. Our results show that our approach outperforms existing methods in execution time and solution optimality while effectively scaling with task complexity.
Video:
[C98] Demonstrating ViSafe: Vision-enabled Safety for High-speed Detect and Avoid
Parv Kapoor, Ian Higgins, Nikhil Keetha, Jay Patrikar, Brady Moon, Zelin Ye, Yao He, Ivan Cisneros, Yaoyu Hu, Changliu Liu, Eunsuk Kang and Sebastian Scherer
Robotics: Science and Systems, 2025
Citation Formats:
Abstract:
Assured safe-separation is essential for achieving seamless high-density operation of airborne vehicles in a shared airspace. To equip resource-constrained aerial systems with this safety-critical capability, we present ViSafe, a high-speed vision-only airborne collision avoidance system. ViSafe offers a full-stack solution to the Detect and Avoid (DAA) problem by tightly integrating a learning-based edge-AI framework with a custom multi-camera hardware prototype designed under SWaP-C constraints. By leveraging perceptual input-focused control barrier functions (CBF) to design, encode, and enforce safety thresholds, ViSafe can provide provably safe runtime guarantees for self-separation in high-speed aerial operations. We evaluate ViSafe’s performance through an extensive test campaign involving both simulated digital twins and real-world flight scenarios. By independently varying agent types, closure rates, interaction geometries, and environmental conditions (e.g., weather and lighting), we demonstrate that ViSafe consistently ensures self-separation across diverse scenarios. In first-of-its-kind real-world high-speed collision avoidance tests with closure rates reaching 144 km/h, ViSafe sets a new benchmark for vision-only autonomous collision avoidance, establishing a new standard for safety in high-speed aerial navigation.
[C97] ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills Tairan He, Jiawei Gao, Wenli Xiao, Yuanhang Zhang, Zi Wang, Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbab, Chaoyi Pan and others others
Robotics: Science and Systems, 2025
Citation Formats:
Abstract:
Humanoid robots hold the potential for unparalleled versatility in performing human-like, whole-body skills. However, achieving agile and coordinated whole-body motions remains a significant challenge due to the dynamics mismatch between simulation and the real world. Existing approaches, such as system identification (SysID) and domain randomization (DR) methods, often rely on labor-intensive parameter tuning or result in overly conservative policies that sacrifice agility. In this paper, we present ASAP (Aligning Simulation and Real-World Physics), a two-stage framework designed to tackle the dynamics mismatch and enable agile humanoid whole-body skills. In the first stage, we pre-train motion tracking policies in simulation using retargeted human motion data. In the second stage, we deploy the policies in the real world and collect real-world data to train a delta (residual) action model that compensates for the dynamics mismatch. Then, ASAP fine-tunes pre-trained policies with the delta action model integrated into the simulator to align effectively with real-world dynamics. We evaluate ASAP across three transfer scenarios: IsaacGym to IsaacSim, IsaacGym to Genesis, and IsaacGym to the real-world Unitree G1 humanoid robot. Our approach significantly improves agility and whole-body coordination across various dynamic motions, reducing tracking error compared to SysID, DR, and delta dynamics learning baselines. ASAP enables highly agile motions that were previously difficult to achieve, demonstrating the potential of delta action learning in bridging simulation and real-world dynamics. These results suggest a promising sim-to-real direction for developing more expressive and agile humanoids.
[C96] APEX-MR: Multi-Robot Asynchronous Planning and Execution for Cooperative Assembly
Philip Huang, Ruixuan Liu, Shobhit Aggarwal, Changliu Liu and Jiaoyang Li
Robotics: Science and Systems, 2025
Citation Formats:
Abstract:
Compared to a single-robot workstation, a multi-robot system offers several advantages: 1) it expands the system’s workspace, 2) improves task efficiency, and more importantly, 3) enables robots to achieve significantly more complex and dexterous tasks, such as cooperative assembly. However, coordinating the tasks and motions of multiple robots is challenging due to issues, e.g., system uncertainty, task efficiency, algorithm scalability, and safety concerns. To address these challenges, this paper studies multi-robot coordination and proposes APEX-MR, an asynchronous planning and execution framework designed to safely and efficiently coordinate multiple robots to achieve cooperative assembly, e.g., LEGO assembly. In particular, APEX-MR provides a systematic approach to post-process multi-robot tasks and motion plans to enable robust asynchronous execution under uncertainty. Experimental results demonstrate that APEX-MR can significantly speed up the execution time of many long-horizon LEGO assembly tasks by 48% compared to sequential planning and 36% compared to synchronous planning on average. To further demonstrate the performance, we deploy APEX-MR to a dual-arm system to perform physical LEGO assembly. To our knowledge, this is the first robotic system capable of performing customized LEGO assembly using commercial LEGO bricks. The experiment results demonstrate that the dual-arm system, with APEX-MR, can safely coordinate robot motions, efficiently collaborate, and construct complex LEGO structures.
Video:
Teaser:
[C95] Continual Learning and Lifting of Koopman Dynamics for Linear Control of Legged Robots Feihan Li, Abulikemu Abuduweili, Yifan Sun, Rui Chen, Weiye Zhao and Changliu Liu
Learning for Dynamics and Control Conference, 2025
Citation Formats:
Abstract:
The control of legged robots, particularly humanoid and quadruped robots, presents significant challenges due to their high-dimensional and nonlinear dynamics. While linear systems can be effectively controlled using methods like Model Predictive Control (MPC), the control of nonlinear systems remains complex. One promising solution is the Koopman Operator, which approximates nonlinear dynamics with a linear model, enabling the use of proven linear control techniques. However, achieving accurate linearization through data-driven methods is difficult due to issues like approximation error, domain shifts, and the limitations of fixed linear state-space representations. These challenges restrict the scalability of Koopman-based approaches. This paper addresses these challenges by proposing a continual learning algorithm designed to iteratively refine Koopman dynamics for high-dimensional legged robots. The key idea is to progressively expand the dataset and latent space dimension, enabling the learned Koopman dynamics to converge towards accurate approximations of the true system dynamics. Theoretical analysis shows that the linear approximation error of our method converges monotonically. Experimental results demonstrate that our method achieves high control performance on robots like Unitree G1/H1/A1/Go2 and ANYmal D, across various terrains using simple linear MPC controllers. This work is the first to successfully apply linearized Koopman dynamics for locomotion control of high-dimensional legged robots, enabling a scalable model-based control solution.
Video:
[C94] Learn With Imagination: Safe Set Guided State-wise Constrained Policy Optimization Feihan Li, Yifan Sun, Weiye Zhao, Rui Chen, Tianhao Wei and Changliu Liu
Learning for Dynamics and Control Conference, 2025
Citation Formats:
Abstract:
Deep reinforcement learning (RL) excels in various control tasks, yet the absence of safety guarantees hampers its real-world applicability. In particular, explorations during learning usually results in safety violations, while the RL agent learns from those mistakes. On the other hand, safe control techniques ensure persistent safety satisfaction but demand strong priors on system dynamics, which is usually hard to obtain in practice. To address these problems, we present Safe Set Guided State-wise Constrained Policy Optimization (S-3PO), a pioneering algorithm generating state-wise safe optimal policies with zero training violations, i.e., learning without mistakes. S-3PO first employs a safety-oriented monitor with black-box dynamics to ensure safe exploration. It then enforces an "imaginary" cost for the RL agent to converge to optimal behaviors within safety constraints. S-3PO outperforms existing methods in high-dimensional robotics tasks, managing state-wise constraints with zero training violation. This innovation marks a significant stride towards real-world safe RL deployment.
[C93] Safe PDE Boundary Control with Neural Operators Hanjiang Hu and Changliu Liu
Learning for Dynamics and Control Conference, 2025
Citation Formats:
Abstract:
The physical world dynamics are generally governed by underlying partial differential equations (PDEs) with unknown analytical forms in science and engineering problems. Neural network based data-driven approaches have been heavily studied in simulating and solving PDE problems in recent years, but it is still challenging to move forward from understanding to controlling the unknown PDE dynamics. PDE boundary control instantiates a simplified but important problem by only focusing on PDE boundary conditions as the control input and output. However, current model-free PDE controllers cannot ensure the boundary output satisfies some given user-specified safety constraint. To this end, we propose a safety filtering framework to guarantee the boundary output stays within the safe set for current model-free controllers. Specifically, we first introduce a general neural boundary control barrier function (BCBF) to ensure the feasibility of the trajectory-wise constraint satisfaction of boundary output. Based on a neural operator modeling the transfer function from boundary control input to output trajectories, we show that the change in the BCBF depends linearly on the change in input boundary, so quadratic programming-based safety filtering can be done for pre-trained model-free controllers. Extensive experiments under challenging hyperbolic, parabolic and Navier-Stokes PDE dynamics environments validate the effectiveness of the proposed method in achieving better general performance and boundary constraint satisfaction compared to the model-free controller baselines.
[C92] Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
Abulikemu Abuduweili and Changliu Liu
Conference on Parsimony and Learning (CPAL), 2025
Citation Formats:
Abstract:
Adaptive gradient optimization methods, such as Adam, are prevalent in training deep neural networks across diverse machine learning tasks due to their ability to achieve faster convergence. However, these methods often suffer from suboptimal generalization compared to stochastic gradient descent (SGD) and exhibit instability, particularly when training Transformer models. In this work, we show the standard initialization of the second-order moment estimation (v0=0) as a significant factor contributing to these limitations. We introduce simple yet effective solutions: initializing the second-order moment estimation with non-zero values, using either data-driven or random initialization strategies. Empirical evaluations demonstrate that our approach not only stabilizes convergence but also enhances the final performance of adaptive gradient optimizers. Furthermore, by adopting the proposed initialization strategies, Adam achieves performance comparable to many recently proposed variants of adaptive gradient optimization methods. Our code is available at https://github.com/Walleclipse/Adam_Initialization/.
[C91] Improving Policy Optimization via \varepsilon-Retrain
Luca Marzari, Changliu Liu, Priya L. Donti and Enrico Marchesini
International Conference on Autonomous Agents and Multiagent Systems, 2025
Citation Formats:
Abstract:
We present ε-retrain, an exploration strategy designed to encourage a behavioral preference while optimizing policies with monotonic improvement guarantees. To this end, we introduce an iterative procedure for collecting retrain areas – parts of the state space where an agent did not follow the behavioral preference. Our method then switches between the typical uniform restart state distribution and the retrain areas using a decaying factor ε, allowing agents to retrain on situations where they violated the preference. Experiments over hundreds of seeds across locomotion, navigation, and power network tasks show that our method yields agents that exhibit significant performance and sample efficiency improvements. Moreover, we employ formal verification of neural networks to provably quantify the degree to which agents adhere to behavioral preferences.
[C90] Safe Control of Quadruped in Varying Dynamics via Safety Index Adaptation
Kai S. Yun, Rui Chen, Chase Dunaway, John M. Dolan and Changliu Liu
IEEE International Conference on Robotics and Automation, 2025
Citation Formats:
Abstract:
Varying dynamics pose a fundamental difficulty when deploying safe control laws in the real world. Safety Index Synthesis (SIS) deeply relies on the system dynamics and once the dynamics change, the previously synthesized safety index becomes invalid. In this work, we show the real-time efficacy of Safety Index Adaptation (SIA) in varying dynamics. SIA enables real-time adaptation to the changing dynamics so that the adapted safe control law can still guarantee 1) forward invariance within a safe region and 2) finite time convergence to that safe region. This work employs SIA on a package-carrying quadruped robot, where the payload weight changes in real-time. SIA updates the safety index when the dynamics change, e.g., a change in payload weight, so that the quadruped can avoid obstacles while achieving its performance objectives. Numerical study provides theoretical guarantees for SIA and a series of hardware experiments demonstrate the effectiveness of SIA in real-world deployment in avoiding obstacles under varying dynamics.
Video:
[C89] Robots that Learn to Safely Influence via Prediction-Informed Reach-Avoid Dynamic Games
Ravi Pandya, Changliu Liu and Andrea Bajcsy
IEEE International Conference on Robotics and Automation, 2025
Citation Formats:
Abstract:
Robots can influence people to accomplish their tasks more efficiently: autonomous cars can inch forward at an intersection to pass through, and tabletop manipulators can go for an object on the table first. However, a robot’s ability to influence can also compromise the safety of nearby people if naively executed. In this work, we pose and solve a novel robust reach-avoid dynamic game which enables robots to be maximally influential, but only when a safety backup control exists. On the human side, we model the human’s behavior as goal-driven but conditioned on the robot’s plan, enabling us to capture influence. On the robot side, we solve the dynamic game in the joint physical and belief space, enabling the robot to reason about how its uncertainty in human behavior will evolve over time. We instantiate our method, called SLIDE (Safely Leveraging Influence in Dynamic Environments), in a high-dimensional (39-D) simulated human-robot collaborative manipulation task solved via offline game-theoretic reinforcement learning. We compare our approach to a robust baseline that treats the human as a worst-case adversary, a safety controller that does not explicitly reason about influence, and an energy-function-based safety shield. We find that SLIDE consistently enables the robot to leverage the influence it has on the human when it is safe to do so, ultimately allowing the robot to be less conservative while still ensuring a high safety rate during task execution.
[C88] HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan and Yuke Zhu
IEEE International Conference on Robotics and Automation, 2025
Citation Formats:
[C87] Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts
Hongyi Chen, Yunchao Yao, Ruixuan Liu, Changliu Liu and Jeffrey Ichnowski
American Control Conference, 2025
Citation Formats:
Abstract:
Current robot autonomy struggles to operate beyond the assumed Operational Design Domain (ODD), the specific set of conditions and environments in which the system is designed to function, while the real-world is rife with uncertainties that may lead to failures. Automating recovery remains a significant challenge. Traditional methods often rely on human intervention to manually address failures or require exhaustive enumeration of failure cases and the design of specific recovery policies for each scenario, both of which are labor-intensive. Foundational Vision-Language Models (VLMs), which demonstrate remarkable common-sense generalization and reasoning capabilities, have broader, potentially unbounded ODDs. However, limitations in spatial reasoning continue to be a common challenge for many VLMs when applied to robot control and motion-level error recovery. In this paper, we investigate how optimizing visual and text prompts can enhance the spatial reasoning of VLMs, enabling them to function effectively as black-box controllers for both motion-level position correction and task-level recovery from unknown failures. Specifically, the optimizations include identifying key visual elements in visual prompts, highlighting these elements in text prompts for querying, and decomposing the reasoning process for failure detection and control generation. In experiments, prompt optimizations significantly outperform pre-trained Vision-Language-Action Models in correcting motion-level position errors and improve accuracy by 65.78% compared to VLMs with unoptimized prompts. Additionally, for task-level failures, optimized prompts enhanced the success rate by 5.8%, 5.8%, and 7.5% in VLMs’ abilities to detect failures, analyze issues, and generate recovery plans, respectively, across a wide range of unknown errors in Lego assembly.
[C86] ThinkBot: Embodied Instruction Following with Thought Chain Reasoning
Guanxing Lu, Ziwei Wang, Changliu Liu, Jiwen Lu and Yansong Tang
International Conference on Learning Representations, 2025
Citation Formats:
[C107] Passing-Order Decision for Three-to-Two Lane Merging of Connected and Autonomous Vehicles
Cheng-Pei Chien, Ben-Hau Chia, Ching-Yun Chang, Shang-Chien Lin, Iris Hui-Ru Jiang, Changliu Liu and Chung-Wei Lin
2025 IEEE 31st International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), 2025
Citation Formats:
[C106] Generating Physically Stable and Buildable Brick Structures from Text
Ava Pun, Kangle Deng, Ruixuan Liu, Deva Ramanan, Changliu Liu and Jun-Yan Zhu
International Conference on Computer Vision, 2025 Best Paper Award (Marr Prize)
Citation Formats:
Abstract:
We introduce LegoGPT, the first approach for generating physically stable LEGO brick models from text prompts. To achieve this, we construct a large-scale, physically stable dataset of LEGO designs, along with their associated captions, and train an autoregressive large language model to predict the next brick to add via next-token prediction. To improve the stability of the resulting designs, we employ an efficient validity check and physics-aware rollback during autoregressive inference, which prunes infeasible token predictions using physics laws and assembly constraints. Our experiments show that LegoGPT produces stable, diverse, and aesthetically pleasing LEGO designs that align closely with the input text prompts. We also develop a text-based LEGO texturing method to generate colored and textured designs. We show that our designs can be assembled manually by humans and automatically by robotic arms. We also release our new dataset, StableText2Lego, containing over 47,000 LEGO structures of over 28,000 unique 3D objects accompanied by detailed captions, along with our code and models at the project website: this URL.
Teaser:
[C105] Robust Tracking Control with Neural Network Dynamic Models under Input Perturbations
Huixuan Cheng, Hanjiang Hu and Changliu Liu
Modeling, Estimation, and Control Conference, 2025
Citation Formats:
Abstract:
Robust control problems have significant practical implications since external disturbances can significantly impact the performance of control methods. Existing robust control methods excel at control-affine systems but fail at neural network dynamic models. Developing robust control methods for such systems remains a complex challenge. In this paper, we focus on robust tracking methods for neural network dynamic models. We first propose a reachability analysis tool designed for this system and then introduce how to reformulate a robust tracking problem with reachable sets. In addition, we prove the existence of a feedback policy that bounds the growth of reachable sets over an infinite horizon. The effectiveness of the proposed approach is validated through numerical simulations of the tracking task, where we compare it with a standard tube MPC method.
[C104] AssemblyComplete: 3D Combinatorial Construction with Deep Reinforcement Learning
Alan Chen and Changliu Liu
Modeling, Estimation, and Control Conference, 2025
Citation Formats:
Abstract:
A critical goal in robotics and autonomy is to teach robots to adapt to real-world collaborative tasks, particularly in automatic assembly. The ability of a robot to understand the original intent of an incomplete assembly and complete missing features without human instruction is valuable but challenging. This paper introduces 3D combinatorial assembly completion, which is demonstrated using combinatorial unit primitives (i.e., Lego bricks). Combinatorial assembly is challenging due to the possible assembly combinations and complex physical constraints (e.g., no brick collisions, structure stability, inventory constraints, etc.). To address these challenges, we propose a two-part deep reinforcement learning (DRL) framework that tackles teaching the robot to understand the objective of an incomplete assembly and learning a construction policy to complete the assembly. The robot queries a stable object library to facilitate assembly inference and guide learning. In addition to the robot policy, an action mask is developed to rule out invalid actions that violate physical constraints for object-oriented construction. We demonstrate the proposed framework’s feasibility and robustness in a variety of assembly scenarios in which the robot satisfies real-life assembly with respect to both solution and runtime quality. Furthermore, results demonstrate that the proposed framework effectively infers and assembles incomplete structures for unseen and unique object types.
[C103] Eye-in-Finger: Smart Fingers for Delicate Assembly and Disassembly of LEGO Zhenran Tang, Ruixuan Liu and Changliu Liu
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2025
Citation Formats:
Abstract:
Manipulation and insertion of small and tight-toleranced objects in robotic assembly remain a critical challenge for vision-based robotics systems due to the required precision and cluttered environment. Conventional global or wrist-mounted cameras often suffer from occlusions when either assembling or disassembling from an existing structure. To address the challenge, this paper introduces "Eye-in-Finger", a novel tool design approach that enhances robotic manipulation by embedding low-cost, high-resolution perception directly at the tool tip. We validate our approach using LEGO assembly and disassembly tasks, which require the robot to manipulate in a cluttered environment and achieve sub-millimeter accuracy and robust error correction due to the tight tolerances. Experimental results demonstrate that our proposed system enables real-time, fine corrections to alignment error, increasing the tolerance of calibration error from 0.4mm to up to 2.0mm for the LEGO manipulation robot.
[C102] Time-Optimal Trajectory Generation with Multi-Level Continuous Kinodynamics Constraints Ruixuan Liu, Changliu Liu and Jessica Leu
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2025
Citation Formats:
Abstract:
Time-optimal trajectory generation (TOTG) is critical in robotics applications to minimize travel time and increase robot task efficiency. To ensure the trajectory is feasible and executable by the robot, it is important to constrain the trajectory kinodynamics subject to the robot actuator limits. A typical actuator has multiple limits, 1) peak limit, and 2) multilevel continuous limits with different operation time windows. The peak limit bounds the instantaneous kinodynamics (IKD), whereas the continuous limits bound the system continuous kinodynamics (CKD). Existing works only constrain IKD, usually by the actuator peak limit, to achieve time optimality. However, a joint capable of operating at its peak limit momentarily will overheat and damage robot life if the motion continues. Alternatively, users can constrain the IKD with a reduced peak limit to avoid violating continuous limits. However, the reduced peak limit would inevitably sacrifice task efficiency. To address the challenge, this paper studies TOTG with both IKD and CKD, and proposes TOTG-C. It formulates the TOTG as a nonlinear programming (NLP). In particular, it proposes a novel formulation to encode the multi-level CKD constraints efficiently. To the best of our knowledge, TOTG-C is the first work that explicitly considers multi-level CKD constraints. We demonstrate the effectiveness and robustness of the proposed TOTG-C both in simulation and real robot experiments.
This paper introduces the Safe Protective and Assistive Robot Kit (SPARK), a comprehensive benchmark designed to ensure safety in humanoid autonomy and teleoperation. Humanoid robots pose significant safety risks due to their physical capabilities of interacting with complex environments. The physical structures of humanoid robots further add complexity to the design of general safety solutions. To facilitate the safe deployment of complex robot systems, SPARK can be used as a toolbox that comes with state-of-the-art safe control algorithms in a modular and composable robot control framework. Users can easily configure safety criteria and sensitivity levels to optimize the balance between safety and performance. To accelerate humanoid safety research and development, SPARK provides a simulation benchmark that compares safety approaches in a variety of environments, tasks, and robot models. Furthermore, SPARK allows quick deployment of synthesized safe controllers on real robots. For hardware deployment, SPARK supports Apple Vision Pro (AVP) or a Motion Capture System as external sensors, while also offering interfaces for seamless integration with alternative hardware setups. This paper demonstrates SPARK’s capability with both simulation experiments and case studies with a Unitree G1 humanoid robot. Leveraging these advantages of SPARK, users and researchers can significantly improve the safety of their humanoid systems as well as accelerate relevant research. The open-source code is available at (https://github.com/intelligent-control-lab/spark)
Video:
[C100] Modelverification. jl: a comprehensive toolbox for formally verifying deep neural networks
Tianhao Wei, Hanjiang Hu, Luca Marzari, Kai S Yun, Peizhi Niu, Xusheng Luo and Changliu Liu
Conference on Computer Aided Verification, 2025
Citation Formats:
Abstract:
Deep Neural Networks (DNN) are crucial in approximating nonlinear functions across diverse applications, ranging from image classification to control. Verifying specific input-output properties can be a highly challenging task due to the lack of a single, self-contained framework that allows a complete range of verification types. To this end, we present \textttModelVerification.jl (MV), the first comprehensive, cutting-edge toolbox that contains a suite of state-of-the-art methods for verifying different types of DNNs and safety specifications. This versatile toolbox is designed to empower developers and machine learning practitioners with robust tools for verifying and ensuring the trustworthiness of their DNN models.
[W] Multi-Level Reasoning for Delicate Assembly using Dual Arms
Philip Huang, Ruixuan Liu, Shobhit Aggarwal, Changliu Liu and Jiaoyang Li
ICRA 2025 Workshop on Language and Semantics of Task and Motion Planning, 2025 Best Paper Finalist
Citation Formats:
[W] Calibrating AI Trust in Complementary Human-AI Collaboration Hanjiang Hu, Yifan Sun and Changliu Liu
ICRA 2025 Workshop on Workshop on Public Trust in Autonomous Systems, 2025
Citation Formats:
Abstract:
Human-AI collaboration is a powerful paradigm in decision-making systems, where humans and AI contribute different strengths with clear complementarity. Yet, achieving optimal team performance depends critically on proper trust in AI, ensuring humans rely on AI appropriately. In real-world scenarios, humans often lack the expertise or performance transparency to judge AI accuracy directly, creating a gap in appropriate trust calibration. In this paper, we address this challenge through three key contributions: (1) we propose a theoretical framework modeling the evolution of human trust in AI over time under AI performance uncertainty, (2) we investigate two self-calibrating trust methods, an instance-based cognitive model and a reinforcement learning (RL) model that learns trust calibration policies from experience, and (3) we conduct simulations comparing both approaches against a rule-based baseline under dynamically varying AI performance. Results show that RL-based trust calibration outperforms others in cumulative performance, while instance-based calibration offers interpretability and sample efficiency. These findings offer pathways for safe and adaptive trust alignment in human-AI collaboration toward trustworthy autonomy.
This paper presents NeSyPack, a neuro-symbolic framework for bimanual logistics packing. Our NeSyPack combines data-driven models and symbolic reasoning to build an explainable hierarchical framework that is generalizable, data-efficient, and reliable. It decomposes a task into subtasks via hierarchical reasoning, and further into atomic skills managed by a symbolic skill graph. The graph selects skill parameters, robot configurations, and task-specific control strategies for execution. This modular design enables robustness, adaptability, and efficient reuse—outperforming end-to-end models that require large-scale retraining. Using NeSyPack, our team won the First Prize in the What Bimanuals Can Do (WBCD) competition at the 2025 IEEE International Conference on Robotics & Automation (ICRA).
[W] Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks Hanjiang Hu, Alexander Robey and Changliu Liu
ICML 2025 Workshop on Reliable and Responsible Foundation Models, 2025
Citation Formats:
Abstract:
Large language models (LLMs) are shown to be vulnerable to jailbreaking attacks where adversarial prompts are designed to elicit harmful responses. While existing defenses effectively mitigate single-turn attacks by detecting and filtering unsafe inputs, they fail against multi-turn jailbreaks that exploit contextual drift over multiple interactions, gradually leading LLMs away from safe behavior. To address this challenge, we propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues. Our approach models the dialogue with LLMs using state-space representations and introduces a novel neural barrier function (NBF) to detect and filter harmful queries emerging from evolving contexts proactively. Our method achieves invariant safety at each turn of dialogue by learning a safety predictor that accounts for adversarial queries, preventing potential context drift toward jailbreaks. Extensive experiments under multiple LLMs show that our NBF-based safety steering outperforms safety alignment, prompt-based steering and lightweight LLM guardrails baselines, offering stronger defenses against multi-turn jailbreaks while maintaining a better trade-off among safety, helpfulness and over-refusal. Check out the website here \urlhttps://sites.google.com/view/llm-nbf/home.
[T6] Influence-Aware Safety for Human-Robot Interaction
Ravi Pandya
PhD Thesis, 2025
Citation Formats:
Abstract:
In recent years, we have seen through recommender systems on social media how influential (and potentially harmful) algorithms can be in our lives, sometimes creating polarization and conspiracies that lead to unsafe behavior. Now that robots are also growing more common in the real world, we must be very careful to ensure that they are aware of the influence they will have on people, especially when it comes to safety—we do not want robots to cause any physical harm. In this thesis, we focus on the problem of influence-aware safe control for human-robot interaction in hopes of enabling robots to intentionally and positively influence people to make their interactions with robots safer and more efficient. We first study this problem from the safe control perspective by introducing a novel method for dealing with the multimodality of the robot’s uncertainty over a human’s intention inside a robust safe controller. Next, we explore different methods for generating influence-aware robot behavior from different levels of abstraction: action-directed, goal-directed, and strategy-directed. We ultimately find useful tools for designing robot behaviors that can proactively influence human collaborators towards positive outcomes. Finally, we join these thrusts by posing and solving the full influence-aware safety problem for human-robot interactions. We first show that explicitly enabling a robot to reason about the influence it has on a human collaborator can lead to more efficient interactions without sacrificing safety. Finally, we extend this paradigm to more general human-AI interactions, particularly people interacting with large language models (LLMs). We show that treating safety as failures in the physical environment in this domain allows us to automatically learn safety-preserving language actions and influential behavior that can steer interactions towards safe long-term outcomes from budget constraints in e-commerce to collision avoidance in driving. The work done in this thesis establishes a basis for several key research directions in the future. Specifically, researchers in AI safety can build on the influence-aware safety problem as a basis for generating safe AI systems that are explicitly aware of how their actions may change the behavior of human users in the real world. This work also enables future work to focus on safe long-term interactions between humans and robots. While this thesis focuses on influence during single interactions, the influence-aware safety problem can easily be applied to repeated interactions with human users, which is an important directions as AI chatbots, recommendation systems and physical robots continue becoming larger parts of our daily lives. Ultimately, this thesis will help researchers understand the importance of explicitly modeling the influence that autonomous agents have on people and how to use this influence to keep people safe.
[T5] Reachable Sets for Control and Planning: from Reactive Safety to Contact-Rich Manipulation
Simin Liu
PhD Thesis, 2025
Citation Formats:
Abstract:
This thesis investigates two fundamental challenges in robotics: ensuring reactive safety for agile systems and achieving scalable planning for contact-rich manipulation. Despite arising in distinct contexts, both problems hinge on reasoning about system dynamics and constraints. We argue that reachable sets—long studied in control theory—provide a unifying computational primitive for tackling both.
In the first part, we study reactive safety for uncertain and high-dimensional systems. We develop new methods for synthesizing control barrier functions (CBFs) using both formal optimization and learning-based techniques. Specifically, we introduce a robust-adaptive synthesis framework based on sum-of-squares programming, which guarantees safety under parametric uncertainty while reducing unnecessary conservatism. To address scalability, we propose an adversarial training framework that learns neural CBFs capable of handling higher-dimensional dynamics. Across cartpole, quadrotor, and other benchmarks, our controllers achieve 100% safety while interfering substantially less with nominal performance objectives than existing baselines.
In the second part, we extend reachable-set methods to long-horizon planning for contact-rich manipulation. We present a hierarchical planner for bimanual SE(2) reorientation tasks that uses mutual reachable sets as discrete motion primitives. By covering object space with a compact set of such primitives, we reduce the combinatorial complexity of hybrid contact planning to shortest-path search on a small graph. This yields efficient and expressive global plans that outperform sampling-based methods, with successful demonstrations on real robotic hardware.
Together, these contributions demonstrate how reachable sets can serve as a versatile foundation across the robotics stack: from low-level safety filters for agile systems to high-level planning abstractions for dexterous manipulation. This unifying perspective suggests broader opportunities for integrating reachability-based methods into robust and scalable robotic autonomy.
The deployment of intelligent robotic agents in complex, non-stationary human environments—such as collaborative manufacturing, autonomous driving, and long-term navigation—hinges on their ability to learn continually from a dynamic stream of real-world data. Traditional machine learning paradigms, which rely on offline training with static datasets, are fundamentally ill-suited for these settings. They are plagued by critical limitations, most notably catastrophic forgetting, where new knowledge overwrites previously learned skills; profound data inefficiency, a particularly acute problem in robotics where data collection is expensive and time-consuming; and the challenge of modeling and controlling complex nonlinear dynamics with adaptability and generalizability from limited data.
This thesis presents a systematic investigation into a new class of learning algorithms designed to overcome these barriers, enabling robust and data-efficient continual learning for robots. The contributions are organized into three syner?gistic pillars. The first pillar, Data-Efficient Optimization, introduces novel algorithms for both online and offline learning. We develop a family of Extended Kalman Filter (EKF)-based optimizers that leverage second-order information to achieve superior convergence for online adaptation. We also improve adaptive gradient methods for offline pretraining by introducing a data-driven approach to initialize optimizer states, enhancing stability and performance. The second pillar, Continual Learning with Memory Mechanisms, addresses catastrophic forgetting through bio-inspired memory architectures. This includes a feedforward compensation strategy that proactively uses critical past experiences to improve adaptation and the BioSLAM dual-memory system, which explicitly manages short-term plasticity and long-term knowledge consolidation for lifelong place recognition. The third and culminating pillar, Models for Efficient Robot Learning, presents novel frameworks for learning generalizable system models for control. We pioneer the continual learning and lifting of Koopman dynamics, a breakthrough approach for linearizing high?dimensional, nonlinear systems like legged robots from streaming data. This pillar also investigates methods for estimating neural network robustness via Lipschitz constants, a crucial step for deploying learned models in safety-critical applications.
Collectively, this body of work provides a foundational framework for creating truly adaptive, resilient, and intelligent robots. By developing solutions for data-efficient optimization, memory-augmented learning, and continual dynamics modeling, this thesis paves the way for the safe and effective deployment of autonomous systems in the open world.
[U] Dexterous Safe Control for Humanoids in Cluttered Environments via Projected Safe Set Algorithm
Rui Chen, Yifan Sun and Changliu Liu
arXiv:2502.02858, 2025
Citation Formats:
Abstract:
It is critical to ensure safety for humanoid robots in real-world applications without compromising performance. In this paper, we consider the problem of dexterous safety, featuring limb-level geometry constraints for avoiding both external and self-collisions in cluttered environments. Compared to safety with simplified bounding geometries in sprase environments, dexterous safety produces numerous constraints which often lead to infeasible constraint sets when solving for safe robot control. To address this issue, we propose Projected Safe Set Algorithm (p-SSA), an extension of classical safe control algorithms to multi-constraint cases. p-SSA relaxes conflicting constraints in a principled manner, minimizing safety violations to guarantee feasible robot control. We verify our approach in simulation and on a real Unitree G1 humanoid robot performing complex collision avoidance tasks. Results show that p-SSA enables the humanoid to operate robustly in challenging situations with minimal safety violations and directly generalizes to various tasks with zero parameter tuning.
[U] Prompt-to-Product: Generative Assembly via Bimanual Manipulation Ruixuan Liu, Philip Huang, Ava Pun, Kangle Deng, Shobhit Aggarwal, Zhenran Tang, Michelle Liu, Deva Ramanan, Jun-Yan Zhu, Jiaoyang Li and Changliu Liu
arXiv:2508.21063, 2025
Citation Formats:
Abstract:
Creating assembly products demands significant manual effort and expert knowledge in 1) designing the assembly and 2) constructing the product. This paper introduces Prompt-to-Product, an automated pipeline that generates real-world assembly products from natural language prompts. Specifically, we leverage LEGO bricks as the assembly platform and automate the process of creating brick assembly structures. Given the user design requirements, Prompt-to-Product generates physically buildable brick designs, and then leverages a bimanual robotic system to construct the real assembly products, bringing user imaginations into the real world. We conduct a comprehensive user study, and the results demonstrate that Prompt-to-Product significantly lowers the barrier and reduces manual effort in creating assembly products from imaginative ideas.
[J28] Efficient Reinforcement Learning of Task Planners for Robotic Palletization Through Iterative Action Masking Learning
Zheng Wu, Yichuan Li, Wei Zhan, Changliu Liu, Yun-Hui Liu and Masayoshi Tomizuka
IEEE Robotics and Automation Letters, 2024
Citation Formats:
[J27] StableLego: Stability Analysis of Block Stacking Assembly Ruixuan Liu, Kangle Deng, Ziwei Wang and Changliu Liu
IEEE Robotics and Automation Letters, 2024
Citation Formats:
[J26] Improve Certified Training with Signal-to-Noise Ratio Loss to Decrease Neuron Variance and Increase Neuron Stability
Tianhao Wei, Ziwei Wang, Peizhi Niu, Abulikemu Abuduweili, Weiye Zhao, Casidhe Hutchison, Eric Sample and Changliu Liu
Transactions on Machine Learning Research, 2024
Citation Formats:
Abstract:
Neural network robustness is a major concern in safety-critical applications. Certified robustness provides a reliable lower bound on worst-case robustness, and certified training methods have been developed to enhance it. However, certified training methods often suffer from over-regularization, leading to lower certified robustness. This work addresses this issue by introducing the concepts of neuron variance and neuron stability, examining their impact on over-regularization and model robustness. To tackle the problem, we extend the Signal-to-Noise Ratio (SNR) into the realm of model robustness, offering a novel perspective and developing SNR-inspired losses aimed at optimizing neuron variance and stability to mitigate over-regularization. Through both empirical and theoretical analysis, our SNR-based approach demonstrates superior performance over existing methods on the MNIST and CIFAR-10 datasets. In addition, our exploration of adversarial training uncovers a beneficial correlation between neuron variance and adversarial robustness, leading to an optimized balance between standard and robust accuracy that outperforms baseline methods.
[J25] Decomposition-based Hierarchical Task Allocation and Planning for Multi-Robots under Hierarchical Temporal Logic Specifications Xusheng Luo, Shaojun Xu, Ruixuan Liu and Changliu Liu
IEEE Robotics and Automation Letters, 2024
Citation Formats:
Abstract:
Past research into robotic planning with temporal logic specifications, notably Linear Temporal Logic (LTL), was largely based on a single formula for individual or groups of robots. But with increasing task complexity, LTL formulas unavoidably grow lengthy, complicating interpretation and specification generation, and straining the computational capacities of the planners. A recent development has been the hierarchical representation of LTL (Luo et al., 2024) that contains multiple temporal logic specifications, providing a more interpretable framework. However, the proposed planning algorithm assumes the independence of robots within each specification, limiting their application to multi-robot coordination with complex temporal constraints. In this work, we formulated a decomposition-based hierarchical framework. At the high level, each specification is first decomposed into a set of atomic sub-tasks. We further infer the temporal relations among the sub-tasks of different specifications to construct a task network. Subsequently, a Mixed Integer Linear Program is used to assign sub-tasks to various robots. At the lower level, domain-specific controllers are employed to execute sub-tasks. Our approach was experimentally applied to domains of navigation and manipulation. The simulation demonstrated that our approach can find better solutions using less runtimes.
Video:
[J24] State-wise Constrained Policy Optimization
Weiye Zhao, Rui Chen, Yifan Sun, Tianhao Wei and Changliu Liu
Transactions on Machine Learning Research, 2024
Citation Formats:
[J23] Guard: A safe reinforcement learning benchmark
Weiye Zhao, Rui Chen, Yifan Sun, Ruixuan Liu, Tianhao Wei and Changliu Liu
Transactions on Machine Learning Research, 2024
Citation Formats:
[J22] Learn Zero-Constraint-Violation Safe Policy in Model-Free Constrained Reinforcement Learning
Haitong Ma, Changliu Liu, Shengbo Eben Li, Sifa Zheng, Wenchao Sun and Jianyu Chen
IEEE Transactions on Neural Networks and Learning Systems, 2024
Citation Formats:
[C85] NN4SysBench: Characterizing Neural Network Verification for Computer Systems
Shuyi Lin, Haoyu He, Tianhao Wei, Kaidi Xu, Huan Zhang, Gagandeep Singh, Changliu Liu and Cheng Tan
The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024
Citation Formats:
<pre id="citation-content-lin2024nnsysbench"
data-bibtex="@inproceedings{lin2024nnsysbench,
index = {C85},
title = {{NN}4SysBench: Characterizing Neural Network Verification for Computer Systems},
author = {Lin, Shuyi and He, Haoyu and Wei, Tianhao and Xu, Kaidi and Zhang, Huan and Singh, Gagandeep and Liu, Changliu and Tan, Cheng},
booktitle = {The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year = {2024},
url = {https://openreview.net/forum?index=mhjRudcHcB},
notes = {Also presented at ICML Workshop on Formal Verification of Machine Learning 2022 under the title "Characterizing Neural Network Verification for Systems with NN4SYSBENCH". Link: https://naizhengtan.github.io/doc/papers/characterizing22haoyu.pdf}
}"
data-plaintext="S. Lin, H. He, T. Wei, K. Xu, H. Zhang, G. Singh, C. Liu and C. Tan, "NN4SysBench: Characterizing Neural Network Verification for Computer Systems" in The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024."
style="white-space: pre-wrap; word-wrap: break-word;">
</pre>
[C84] KOROL: Learning Visualizable Object Feature with Koopman Operator Rollout for Manipulation
Hongyi Chen, Abulikemu Abuduweili, Aviral Agrawal, Yunhai Han, Harish Ravichandar, Changliu Liu and Jeffrey Ichnowski
Conference on Robot Learning, 2024
Citation Formats:
Abstract:
Learning dexterous manipulation skills presents significant challenges due to complex nonlinear dynamics that underlie the interactions between objects and multi-fingered hands. Koopman operators have emerged as a robust method for modeling such nonlinear dynamics within a linear framework. However, current methods rely on runtime access to ground-truth (GT) object states, making them unsuitable for vision-based practical applications. Unlike image-to-action policies that implicitly learn visual features for control, we use a dynamics model, specifically the Koopman operator, to learn visually interpretable object features critical for robotic manipulation within a scene. We construct a Koopman operator using object features predicted by a feature extractor and utilize it to auto-regressively advance system states. We train the feature extractor to embed scene information into object features, thereby enabling the accurate propagation of robot trajectories. We evaluate our approach on simulated and real-world robot tasks, with results showing that it outperformed the model-based imitation learning NDP by 1.08 and the image-to-action Diffusion Policy by 1.16. The results suggest that our method maintains task success rates with learned features and extends applicability to real-world manipulation without GT object states. Project video and code are available at: https://github.com/hychen-naza/KOROL.
[C83] Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skills
Tianhao Wei, Liqian Ma, Rui Chen, Weiye Zhao and Changliu Liu
Conference on Robot Learning, 2024
Citation Formats:
Abstract:
The requirements for real-world manipulation tasks are diverse and often conflicting; some tasks require precise motion while others require force compliance; some tasks require avoidance of certain regions while others require convergence to certain states. Satisfying these varied requirements with a fixed state-action representation and control strategy is challenging, impeding the development of a universal robotic foundation model. In this work, we propose Meta-Control, the first LLM-enabled automatic control synthesis approach that creates customized state representations and control strategies tailored to specific tasks. Our core insight is that a meta-control system can be built to automate the thought process that human experts use to design control systems. Specifically, human experts heavily use a model-based, hierarchical (from abstract to concrete) thought model, then compose various dynamic models and controllers together to form a control system. Meta-Control mimics the thought model and harnesses LLM’s extensive control knowledge with Socrates’ "art of midwifery" to automate the thought process. Meta-Control stands out for its fully model-based nature, allowing rigorous analysis, generalizability, robustness, efficient parameter tuning, and reliable real-time execution.
Video:
[C82] OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu and Guanya Shi
Conference on Robot Learning, 2024
Citation Formats:
[C81] Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagation Hanjiang Hu, Yujie Yang, Tianhao Wei and Changliu Liu
Conference on Robot Learning, 2024
Citation Formats:
Abstract:
Control barrier functions (CBFs) are important in safety-critical systems and robot control applications. Neural networks have been used to parameterize and synthesize CBFs with bounded control input for complex systems. However, it is still challenging to verify pre-trained neural networks CBFs (neural CBFs) in an efficient symbolic manner. To this end, we propose a new efficient verification framework for ReLU-based neural CBFs through symbolic derivative bound propagation by combining the linearly bounded nonlinear dynamic system and the gradient bounds of neural CBFs. Specifically, with Heaviside step function form for derivatives of activation functions, we show that the symbolic bounds can be propagated through the inner product of neural CBF Jacobian and nonlinear system dynamics. Through extensive experiments on different robot dynamics, our results outperform the interval arithmetic-based baselines in verified rate and verification time along the CBF boundary, validating the effectiveness and efficiency of the proposed method with different model complexity. The code can be found at https://github.com/intelligent-control-lab/verify-neural-CBF.
[C80] ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
Guanxing Lu, Shiyi Zhang, Ziwei Wang, Changliu Liu, Jiwen Lu and Yansong Tang
European Conference on Computer Vision, 2024
Citation Formats:
[C79] Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu and Guanya Shi
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024
Citation Formats:
[C78] Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion Tairan He, Chong Zhang, Wenli Xiao, Guanqi He, Changliu Liu and Guanya Shi
Robotics: Science and Systems, 2024 Outstanding Student Paper Award Finalist
Citation Formats:
[C77] Absolute Policy Optimization: Enhancing Lower Probability Bound of Performance with High Confidence
Weiye Zhao, Feihan Li, Yifan Sun, Rui Chen, Tianhao Wei and Changliu Liu
International Conference on Machine Learning, 2024
Citation Formats:
[C76] A Lightweight and Transferable Design for Robust LEGO Manipulation Ruixuan Liu, Yifan Sun and Changliu Liu
International Symposium of Flexible Automation, 2024
Citation Formats:
Video:
Teaser:
[C75] Real-Time Safe Control of Neural Network Dynamic Models with Sound Approximation Hanjiang Hu, Jianglin Lan and Changliu Liu
Learning for Dynamics and Control Conference, 2024
Citation Formats:
Abstract:
Safe control of neural network dynamic models (NNDMs) is important to robotics and many applications. However, it remains challenging to compute an optimal safe control in real time for NNDM. To enable real-time computation, we propose to use a sound approximation of the NNDM in the control synthesis. In particular, we propose Bernstein over-approximated neural dynamics (BOND) based on the Bernstein polynomial over-approximation (BPO) of ReLU activation functions in NNDM. To mitigate the errors introduced by the approximation and to ensure persistent feasibility of the safe control problems, we synthesize a worst-case safety index using the most unsafe approximated state within the BPO relaxation of NNDM offline. For the online real-time optimization, we formulate the first-order Taylor approximation of the nonlinear worst-case safety constraint as an additional linear layer of NNDM with the l2 bounded bias term for the higher-order remainder. Comprehensive experiments with different neural dynamics and safety constraints show that with safety guaranteed, our NNDMs with sound approximation are 10-100 times faster than the safe control baseline that uses mixed integer programming (MIP), validating the effectiveness of the worst-case safety index and scalability of the proposed BOND in real-time large-scale settings.
[C74] Synthesis and verification of robust-adaptive safe controllers
Simin Liu, Kai S Yun, John M Dolan and Changliu Liu
European Control Conference, 2024
Citation Formats:
[C73] Multi-Agent Strategy Explanations for Human-Robot Collaboration
Ravi Pandya, Michelle Zhao, Changliu Liu, Reid Simmons and Henny Admoni
IEEE International Conference on Robotics and Automation, 2024
Citation Formats:
Video:
[C72] Towards Proactive Safe Human-Robot Collaborations via Data-Efficient Conditional Behavior Prediction
Ravi Pandya, Zhuoyuan Wang, Yorie Nakahira and Changliu Liu
IEEE International Conference on Robotics and Automation, 2024
Citation Formats:
Video:
[C71] Optimizing Multi-Touch Textile and Tactile Skin Sensing Through Circuit Parameter Estimation
Su Bo Ying, Yuchen Wu, Chengtao Wen and Changliu Liu
IEEE International Conference on Robotics and Automation, 2024
Citation Formats:
[C70] Real-time Safety Index Adaptation for Parameter-varying Systems via Determinant Gradient Ascend
Rui Chen, Weiye Zhao, Ruixuan Liu, Weiyang Zhang and Changliu Liu
American control Conference, 2024
Citation Formats:
[C69] Hybrid Task Constrained Incremental Planner for Robot Manipulators in Confined Environments Yifan Sun, Weiye Zhao and Changliu Liu
American Control Conference, 2024
Citation Formats:
[C68] An Optimal Control Framework for Influencing Human Driving Behavior in Mixed-Autonomy Traffic
Anirudh Chari, Rui Chen, Jaskaran Grover and Changliu Liu
American Control Conference, 2024
Citation Formats:
[C67] Safety Index Synthesis with State-dependent Control Space
Rui Chen, Weiye Zhao and Changliu Liu
American Control Conference, 2024
Citation Formats:
[C66] Multimodal Safe Control for Human-Robot Interaction
Ravi Pandya, Tianhao Wei and Changliu Liu
American Control Conference, 2024
Citation Formats:
[W] Estimating Neural Network Robustness via Lipschitz Constant and Architecture Sensitivity
Abulikemu Abuduweili and Changliu Liu
CoRL Workshop on Safe and Robust Robot Learning for Operation in the Real World, 2024
Citation Formats:
Abstract:
Ensuring neural network robustness is essential for the safe and reliable operation of robotic learning systems, especially in perception and decision-making tasks within real-world environments. This paper investigates the robustness of neural networks in perception systems, specifically examining their sensitivity to targeted, small-scale perturbations. We identify the Lipschitz constant as a key metric for quantifying and enhancing network robustness. We derive an analytical expression to compute the Lipschitz constant based on neural network architecture, providing a theoretical basis for estimating and improving robustness. Several experiments reveal the relationship between network design, the Lipschitz constant, and robustness, offering practical insights for developing safer, more robust robot learning systems.
[T3] Safeguarding and Empowering General Purpose Robots through Abstraction and Constraint Certification
Tianhao Wei
PhD Thesis, 2024
Citation Formats:
Abstract:
Robots are increasingly deployed across various domains, from industrial automation to domestic assistance. Ensuring that robots operate safely and intelligently is crucial to preventing potential risks such as injury, loss of life, and economic costs. This thesis addresses key challenges in deploying robots in complex real-world environments, including providing formal safety guarantees in uncertain conditions, scaling safety guarantees to realistic high-dimensional systems, allowing the robot to behave intelligently while remaining explainable and trustworthy, and ensuring the robustness of neural network components.
This thesis introduces a suite of tools to tackle these challenges. The first tool, Meta-Control, synthesizes heterogeneous robot skills with a hiearchical control approach, which could decompose system-level safety requirements into module-level constraints. These constraints are categorized into control and neural network constraints. For control constraints, the toolset introduces Abstract Safe Control for hierarchical safety guarantees, Robust Safe Control for handling model uncertainty through a control-limits aware robust framework, Neural Network Dynamic Models (NNDM) Safe Control for integrating data-driven models with safety guarantees, and Benchmark of Interactive Safety for benchmarking and unifying different safe control algorithms. For neural network constraints, the toolset introduces ModelVerification.jl toolbox for verifying neural network safety specifications, online verification for online assurance under domain shifts and network update, and the Signal-to-Noise Ratio (SNR) loss method to enhance stability and robustness of neural networks.
These tools enable the provision of formal safety guarantees with partially known or unknown dynamic models in uncertain, interactive environments, achieving state-of-the-art control safety and neural network safety. This allows robot arms to perform various tasks efficiently and safely, advancing the development of reliable and trustworthy general-purpose robots.
[T2] State-wise Safe Learning and Control
Weiye Zhao
PhD Thesis, 2024
Citation Formats:
Abstract:
Ensuring safety by persistently satisfying hard state constraints is a critical capability in the fields of reinforcement learning (RL) and control. While RL and control have achieved impressive feats in performing challenging tasks, the lack of safety assurance remains a significant obstacle for real-world applications. Consequently, the research focus has shifted towards developing methods that meet stringent safety specifications in uncertain environments, driving the field of safe learning and control.
In the realm of safe control, energy function-based methodologies allocate diminished energy levels to safe states while orchestrating secure control laws to dissipate energy. However, prevailing safe control methods necessitate explicit analytical models of the dynamic system, a constraint often unmet in real-world scenarios. Moreover, current safe control techniques typically presuppose an unbounded control space, a premise divergent from the bounded nature of actual control spaces in reality. This incongruity can lead to the possibility of an empty set of safe controls, thereby jeopardizing the assurance of state-wise safety.
In the sphere of safe RL, extensive endeavors have been undertaken to address safety within the framework of Constrained Markov Decision Processes (CMDP), which is not capable of handling state-wise safety constraints. Furthermore, existing safe RL algorithms predominantly acquire policy learning through trial-and-error, a process that introduces inevitable unsafe exploration. This characteristic renders them unsuitable for training in real-world, safety-critical applications.
In this thesis, we present groundbreaking advancements in RL and control, ensuring state-wise safety by effectively addressing these challenges: (i) For safe control, we design energy functions to ensure a nonempty set of safe controls under dynamics limits and different knowledge levels of system dynamics, which can achieve forward invariance and finite time convergence. (ii) In safe learning, we propose a set of novel policy search algorithms for state-wise constrained RL. Specifically, (a) State-wise Constrained Policy Optimization (SCPO) guarantees state-wise constraint satisfaction in expectation per iteration, (b) Absolute Policy Optimization (APO) guarantees monotonic improvement of worst-case performance per iteration, and (c) Absolute State-wise Constrained Policy Optimization (ASCPO) guarantees worst-case state-wise constraint satisfaction per iteration. The proposed approaches accommodates high-dimensional neural network policies. Furthermore, we combine benefits from safe control and learning to pioneer an algorithm generating state-wise safe optimal policies with zero training violations, a learning-without-mistakes paradigm. (iii) Lastly, we introduce a comprehensive and adaptable benchmark, the first of its kind, for safe RL and control. This benchmark caters to diverse agents, tasks, and safety constraints, while offering unified implementations of cutting-edge safe learning and control algorithms within a controlled environment.
[U] Robustifying Long-term Human-Robot Collaboration through a Hierarchical and Multimodal Framework Peiqi Yu, Abulikemu Abuduweili, Ruixuan Liu and Changliu Liu
arXiv:2411.15711, 2024
Citation Formats:
Abstract:
Long-term Human-Robot Collaboration (HRC) is crucial for enabling flexible manufacturing systems and integrating companion robots into daily human environments over extended periods. This paper identifies several key challenges for such collaborations, such as accurate recognition of human plan, robustness to disturbances, operational efficiency, adaptability to diverse user behaviors, and sustained human satisfaction. To address these challenges, we model the long-term HRC task through a hierarchical task graph and presents a novel multimodal and hierarchical framework to enable robots to better assist humans to advance on the task graph. In particular, the proposed multimodal framework integrates visual observations with speech commands to facilitate intuitive and flexible human-robot interactions. Additionally, our hierarchical designs for both human pose detection and plan prediction allow better understanding of human behaviors and significantly enhance system accuracy, robustness, and flexibility. Moreover, an online adaptation mechanism enables real-time adjustment to diverse user behaviors. We deploy the proposed framework to KINOVA GEN3 robot and conduct extensive user studies on real-world long-term HRC assembly scenarios. Experimental results show that our approaches reduce task completion time by 15.9%, achieves an average task success rate of 91.8% and an overall user satisfaction score of 84% in long-term HRC tasks, showcasing its applicability in enhancing real-world long-term HRC.
Enforcing state-wise safety constraints is critical for the application of reinforcement learning (RL) in real-world problems, such as autonomous driving and robot manipulation. However, existing safe RL methods only enforce state-wise constraints in expectation or enforce hard state-wise constraints with strong assumptions. The former does not exclude the probability of safety violations, while the latter is impractical. Our insight is that although it is intractable to guarantee hard state-wise constraints in a model-free setting, we can enforce state-wise safety with high probability while excluding strong assumptions. To accomplish the goal, we propose Absolute State-wise Constrained Policy Optimization (ASCPO), a novel general-purpose policy search algorithm that guarantees high-probability state-wise constraint satisfaction for stochastic systems. We demonstrate the effectiveness of our approach by training neural network policies for extensive robot locomotion tasks, where the agent must adhere to various state-wise safety constraints. Our results show that ASCPO significantly outperforms existing methods in handling state-wise constraints across challenging continuous control tasks, highlighting its potential for real-world applications.
[J21] An Optical Control Environment for Benchmarking Reinforcement Learning Algorithms
Abulikemu Abuduweili and Changliu Liu
Transactions on Machine Learning Research, 2023
Citation Formats:
Abstract:
Deep reinforcement learning has the potential to address various scientific problems. In this paper, we implement an optics simulation environment for reinforcement learning based controllers. The environment captures the essence of nonconvexity, nonlinearity, and time-dependent noise inherent in optical systems, offering a more realistic setting. Subsequently, we provide the benchmark results of several reinforcement learning algorithms on the proposed simulation environment. The experimental findings demonstrate the superiority of off-policy reinforcement learning approaches over traditional control algorithms in navigating the intricacies of complex optical control environments.
[J20] Bioslam: A bioinspired lifelong memory system for general place recognition
Peng Yin, Abulikemu Abuduweili, Shiqi Zhao, Lingyun Xu, Changliu Liu and Sebastian Scherer
IEEE Transactions on Robotics, 2023
Citation Formats:
Abstract:
We present BioSLAM, a lifelong (lifelong simultaneous localization and mapping) SLAM framework for learning various new appearances incrementally and maintaining accurate place recognition for previously visited areas. Unlike humans, artificial neural networks suffer from catastrophic forgetting and may forget the previously visited areas when trained with new arrivals. For humans, researchers discover that there exists a memory replay mechanism in the brain to keep the neuron active for previous events. Inspired by this discovery, BioSLAM designs a gated generative replay to control the robot’s learning behavior based on the feedback rewards. Specifically, BioSLAM provides a novel dual-memory mechanism for the maintenance of: 1) a dynamic memory to efficiently learn new observations; and 2) a static memory to balance new–old knowledge. When the agent is encountered with different appearances under new domains, the complete processing pipeline can help to incrementally update the place recognition ability, robust to the increasing complexity of long-term place recognition. We demonstrate BioSLAM in three incremental SLAM scenarios as follows. 1) A 120 km city-scale trajectories with LiDAR-based inputs. 2) A multivisited 4.5 km campus-scale trajectories with LiDAR-vision inputs. 3) An official Oxford dataset with 10 km visual inputs under different environmental conditions. We show that BioSLAM can incrementally update the agent’s place recognition ability and outperform the state-of-the-art incremental approach, generative replay, by 24% in terms of place recognition accuracy. To the best of our knowledge, BioSLAM is the first memory-enhanced lifelong SLAM system to help incremental place recognition in long-term navigation tasks.
Video:
[J19] Customizing Textile and Tactile Skins for Interactive Industrial Robots
Bo Ying Su, Zhongqi Wei, James McCann, Wenzhen Yuan and Changliu Liu
ASME Letters in Dynamic Systems and Control, 2023
Citation Formats:
[J18] First three years of the international verification of neural networks competition (VNN-COMP)
Christopher Brix, Mark Niklas Müller, Stanley Bak, Taylor T Johnson and Changliu Liu
International Journal on Software Tools for Technology Transfer, 2023
Citation Formats:
<pre id="citation-content-brix2023first"
data-bibtex="@article{brix2023first,
index = {J18},
title = {First three years of the international verification of neural networks competition (VNN-COMP)},
author = {Brix, Christopher and M{\"u}ller, Mark Niklas and Bak, Stanley and Johnson, Taylor T and Liu, Changliu},
journal = {International Journal on Software Tools for Technology Transfer},
volume = {25},
number = {3},
pages = {329--339},
year = {2023},
publisher = {Springer Berlin Heidelberg Berlin/Heidelberg}
}"
data-plaintext="C. Brix, M. Müller, S. Bak, T. Johnson and C. Liu, "First three years of the international verification of neural networks competition (VNN-COMP)" in International Journal on Software Tools for Technology Transfer, 2023."
style="white-space: pre-wrap; word-wrap: break-word;">
</pre>
[J17] Robust and context-aware real-time collaborative robot handling via dynamic gesture commands
Rui Chen, Alvin Shek and Changliu Liu
IEEE Robotics and Automation Letters, 2023
Citation Formats:
[J16] A hierarchical long short term safety framework for efficient robot manipulation under uncertainty
Suqin He, Weiye Zhao, Chuxiong Hu, Yu Zhu and Changliu Liu
Robotics and Computer-Integrated Manufacturing, 2023
Citation Formats:
[C65] Online Model Adaptation with Feedforward Compensation
Abulikemu Abuduweili and Changliu Liu
Conference on Robot Learning, 2023
Citation Formats:
Abstract:
To cope with distribution shifts or non-stationarity in system dynamics, online adaptation algorithms have been introduced to update offline-learned prediction models in real-time. Existing online adaptation methods focus on optimizing the prediction model by utilizing feedback from the latest prediction error. Unfortunately, this feedback-based approach is susceptible to forgetting past information. This work proposes an online adaptation method with feedforward compensation, which uses critical data samples from a memory buffer, instead of the latest samples, to optimize the prediction model. We prove that the proposed approach achieves a smaller error bound compared to previously utilized methods in slow time-varying systems. We conducted experiments on several prediction tasks, which clearly illustrate the superiority of the proposed feedforward adaptation method. Furthermore, our feedforward adaptation technique is capable of estimating an uncertainty bound for predictions.
Video:
[C64] Interactive Car-Following: Matters but NOT Always
Chengyuan Zhang, Rui Chen, Jiacheng Zhu, Wenshuo Wang, Changliu Liu and Lijun Sun
IEEE International Conference on Intelligent Transportation Systems, 2023
Citation Formats:
[C63] Zero-shot Transferable and Persistently Feasible Safe Control for High Dimensional Systems by Consistent Abstraction
Tianhao Wei, Shucheng Kang, Ruixuan Liu and Changliu Liu
IEEE Conference on Decision and Control, 2023
Citation Formats:
[C62] Proactive human-robot co-assembly: Leveraging human intention prediction and robust safe control Ruixuan Liu, Rui Chen, Abulikemu Abuduweili and Changliu Liu
IEEE Conference on Control Technology and Applications, 2023
Citation Formats:
Abstract:
Human-robot collaboration (HRC) is one key component to achieving flexible manufacturing to meet the different needs of customers. However, it is difficult to build intelligent robots that can proactively assist humans in a safe and efficient way due to several challenges. First, it is challenging to achieve efficient collaboration due to diverse human behaviors and data scarcity. Second, it is difficult to ensure interactive safety due to uncertainty in human behaviors. This paper presents an integrated framework for proactive HRC. A robust intention prediction module, which leverages prior task information and human-in-the-loop training, is learned to guide the robot for efficient collaboration. The proposed framework also uses robust safe control to ensure interactive safety under uncertainty. The developed framework is applied to a co-assembly task using a Kinova Gen3 robot. The experiment demonstrates that our solution is robust to environmental changes as well as different human preferences and behaviors. In addition, it improves task efficiency by approximately 15-20%. Moreover, the experiment demonstrates that our solution can guarantee interactive safety during proactive collaboration.
[C61] Building Verified Neural Networks for Computer Systems with Ouroboros
Tianhao Wei, Zhihao Jia, Changliu Liu and Cheng Tan
Sixth Conference on Machine Learning and Systems, 2023
Citation Formats:
[C60] State-wise safe reinforcement learning: A survey
Weiye Zhao, Tairan He, Rui Chen, Tianhao Wei and Changliu Liu
International Joint Conferences on Artificial Intelligence, 2023
Citation Formats:
[C59] Consensus-Based Fault-Tolerant Platooning for Connected and Autonomous Vehicles
Tzu-Yen Tseng, Ding-Jiun Huang, Jia-You Lin, Po-Jui Chang, Chung-Wei Lin and Changliu Liu
IEEE Intelligent Vehicles Symposium, 2023
Citation Formats:
[C58] Space-Time Conflict Spheres for Constrained Multi-Agent Motion Planning
Anirudh Chari, Rui Chen and Changliu Liu
IEEE Intelligent Vehicles Symposium, 2023
Citation Formats:
[C57] Probabilistic safeguard for reinforcement learning using safety index guided gaussian process models
Weiye Zhao, Tairan He and Changliu Liu
Learning for Dynamics and Control Conference, 2023
Citation Formats:
[C56] Safety index synthesis via sum-of-squares programming
Weiye Zhao, Tairan He, Tianhao Wei, Simin Liu and Changliu Liu
American Control Conference, 2023
Citation Formats:
[C55] Learning from physical human feedback: An object-centric one-shot adaptation method
Alvin Shek, Bo Ying Su, Rui Chen and Changliu Liu
IEEE International Conference on Robotics and Automation, 2023 Outstanding Interaction Paper
Citation Formats:
[C54] Autocost: Evolving intrinsic cost for zero-violation reinforcement learning Tairan He, Weiye Zhao and Changliu Liu
Proceedings of the AAAI Conference on Artificial Intelligence, 2023
Citation Formats:
[W] Robotic LEGO Assembly and Disassembly from Human Demonstration Ruixuan Liu, Yifan Sun and Changliu Liu
ACC Workshop on Recent Advancement of Human Autonomy Interaction and Integration, 2023
Citation Formats:
Video:
[W] Robustness verification for perception models against camera motion perturbations Hanjiang Hu, Changliu Liu and Ding Zhao
ICML Workshop on Formal Verification of Machine Learning (WFVML), 2023
Citation Formats:
[W] Obtaining hierarchy from human instructions: an llms-based approach Xusheng Luo, Shaojun Xu and Changliu Liu
CoRL 2023 Workshop on Learning Effective Abstractions for Planning (LEAP), 2023
Citation Formats:
[T1] System Identification and Control of Multiagent Systems Through Interactions
Jaskaran Singh Grover
PhD Thesis, 2023
Citation Formats:
Abstract:
This thesis investigates the problem of inferring the underlying dynamic model of individual agents of a multiagent system (MAS) and using these models to shape the MAS’s behavior using robots extrinsic to the MAS. We investigate (a) how an observer can infer the latent task and inter-agent interaction constraints from the agents’ motion and (b) how the observer can elicit a desired behavior out of the MAS by orchestrating its interactions with robots. The ability to learn individual dynamics models of an aggregated system has several applications such as learning local rules in biological swarms that give rise to emergent behavior and learning tactics of an adversarial multirobot team. Likewise, the ability to shape behavior using extrinsic robots can be used to defend against an adversarial team of robots and guide humans using robots as in social navigation.
The first part of this thesis focuses on the model learning problem. We model agents as integrators that solve a reactive optimization to calculate velocities for mediating between goal-directed motions and collision avoidance with other agents. We develop several estimators that allow an observer to infer this model’s parameters and show that the learned parameters indeed rationalize the observed motions. Necessary identifiability conditions are derived that guarantee correct inference. Our proposed estimators include adaptive observers, Kalman filters and several inverse optimization algorithms that are robust to both measurement noise and model mismatch. To demonstrate this robustness, we evaluate these estimators on a pedestrian dataset and learn each pedestrian’s desired velocity, aggressiveness coefficients and safety margins with walls, obstacles and other pedestrians.
The second part of this thesis focuses on eliciting a desired behavior out of the MAS by inducing interactions with robots. While the theory we develop is general, we consider the dog-sheep herding problem as a use case that requires controlling dog robots to repel sheep agents from a critical zone. We incorporate non-collocated feedback linearization in an optimization-based framework to compute the desired controls for the dogs. Both centralized and distributed implementations are developed to cater to the scalability, feasibility and budget-efficiency objectives. We validate the correctness of these controllers in multiple experiments on the CMU multirobot arena. We also develop a robust extension of these controllers, which we term control-barrier function based semidefinite programs (CBF-SDPs), that guarantee zone defense despite uncertainty in the sheep’s dynamics. Finally, we conclude this thesis with an integration of the robust model learning algorithms with robust control algorithms followed by experimental validation on the multirobot arena.
[B2] Distributed Coordination and Centralized Scheduling for Automobiles at Intersections
Yi-Ting Lin, Chung-Wei Lin, Iris Hui-Ru Jiang and Changliu Liu
Machine Learning and Optimization Techniques for Automotive Cyber-Physical Systems, 2023
Citation Formats:
[U] Robust Safe Control with Multi-Modal Uncertainty
Tianhao Wei, Liqian Ma, Ravi Pandya and Changliu Liu
arXiv:2309.16830, 2023
Citation Formats:
[U] Learning predictive safety filter via decomposition of robust invariant set
Zeyang Li, Chuxiong Hu, Weiye Zhao and Changliu Liu
arXiv:2311.06769, 2023
Citation Formats:
[U] The fourth international verification of neural networks competition (vnn-comp 2023): Summary and results
Christopher Brix, Stanley Bak, Changliu Liu and Taylor T Johnson
arXiv:2312.16760, 2023
Citation Formats:
[U] Simulation-aided Learning from Demonstration for Robotic LEGO Construction Ruixuan Liu, Alan Chen, Xusheng Luo and Changliu Liu
arXiv:2309.11010, 2023
[J15] Persistently feasible robust safe control by safety index synthesis and convex semi-infinite programming
Tianhao Wei, Shucheng Kang, Weiye Zhao and Changliu Liu
IEEE Control Systems Letters, 2022
Citation Formats:
[J14] Social interactions for autonomous driving: A review and perspectives
Wenshuo Wang, Letian Wang, Chengyuan Zhang, Changliu Liu, Lijun Sun and others others
Foundations and Trends in Robotics, 2022
Citation Formats:
[J13] Efficient Game-Theoretic Planning With Prediction Heuristic for Socially-Compliant Autonomous Driving
Chenran Li, Tu Trinh, Letian Wang, Changliu Liu, Masayoshi Tomizuka and Wei Zhan
IEEE Robotics and Automation Letters, 2022
Citation Formats:
[J12] Provably Safe Tolerance Estimation for Robot Arms via Sum-of-Squares Programming
Weiye Zhao, Suqin He and Changliu Liu
IEEE Control Systems Letters, 2022
Citation Formats:
Video:
[J11] The before, during, and after of multi-robot deadlock
Jaskaran Grover, Changliu Liu and Katia Sycara
The International Journal of Robotics Research, 2022
Citation Formats:
Abstract:
Collision avoidance for multirobot systems is a well-studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers that guarantee collision avoidance and goal stabilization for multiple robots. However, it has been noted that reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics that causes the robots to stall before reaching their goals. In this paper, we analyze the closed-loop dynamics of robots using CBFs, to characterize controller parameters, initial conditions, and goal locations that invariably lead the system to deadlock. Using tools from duality theory, we derive geometric properties of robot configurations of an N robot system once it is in deadlock and we justify them using the mechanics interpretation of KKT conditions. Our key deductions are that 1) system deadlock is characterized by a force-equilibrium on robots and 2) deadlock occurs to ensure safety when safety is on the brink of being violated. These deductions allow us to interpret deadlock as a subset of the state space, and we show that this set is non-empty and located on the boundary of the safe set. By exploiting these properties, we analyze the number of admissible robot configurations in deadlock and develop a provably-correct decentralized algorithm for deadlock resolution to safely deliver the robots to their goals. This algorithm is validated in simulations as well as experimentally on Khepera-IV robots.
[C53] Safe Control Under Input Limits with Neural Control Barrier Functions
Simin Liu, Changliu Liu and John Dolan
Conference on Robot Learning, 2022
Citation Formats:
[C52] ARC - Actor Residual Critic for Adversarial Imitation Learning
Ankur Deka, Changliu Liu and Katia Sycara
Conference on Robot Learning, 2022
Citation Formats:
[C51] Distributed multirobot control for non-cooperative herding
Nishant Mohanty, Jaskaran Grover, Changliu Liu and Katia Sycara
International Symposium on Distributed Autonomous Robotic Systems, 2022
Citation Formats:
Abstract:
In this paper, we consider the problem of protecting a high-value area from being breached by sheep agents by crafting motions for dog robots. We use control barrier functions to pose constraints on the dogs’ velocities that induce repulsion in the sheep relative to the high-value area. This paper extends the results developed in our prior work on the same topic in three ways. Firstly, we implement and validate our previously developed centralized herding algorithm on many robots. We show herding of up to five sheep agents using three dog robots. Secondly, as an extension to the centralized approach, we develop two distributed herding algorithms, one favoring feasibility while the other favoring optimality. In the first algorithm, we allocate a unique sheep to a unique dog, making that dog responsible for herding its allocated sheep away from the protected zone. We provide feasibility proof for this approach, along with numerical simulations. In the second algorithm, we develop an iterative distributed reformulation of the centralized algorithm, which inherits the optimality (i.e. budget efficiency) from the centralized approach. Lastly, we conduct real-world experiments of these distributed algorithms and demonstrate herding of up to five sheep agents using five dog robots.
[C50] Task-agnostic Adaptation for Safe Human-robot Handover Ruixuan Liu, Rui Chen and Changliu Liu
IFAC Workshop on Cyber-Physical Human Systems, 2022 Best Student Paper Award
Citation Formats:
Video:
[C49] Semantically-Aware Pedestrian Intent Prediction With Barrier Functions and Mixed-Integer Quadratic Programming
Jaskaran Grover, Yiwei Lyu, Wenhao Luo, Changliu Liu, John Dolan and Katia Sycara
IFAC Workshop on Cyber-Physical Human Systems, 2022 Best Student Paper Finalist
Citation Formats:
[C48] Noncooperative Herding With Control Barrier Functions: Theory and Experiments
Jaskaran Grover, Nishant Mohanty, Wenhao Luo, Changliu Liu and Katia Sycara
IEEE Conference on Decision and Control, 2022
Citation Formats:
[C47] Safe hierarchical navigation in crowded dynamic uncertain environments
Hongyi Chen, Shiyu Feng, Ye Zhao, Changliu Liu and Patricio A Vela
IEEE Conference on Decision and Control, 2022
Citation Formats:
[C46] Safe and Efficient Exploration of Human Models During Human-Robot Interaction
Ravi Pandya and Changliu Liu
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022
Citation Formats:
Abstract:
Many collaborative human-robot tasks require the robot to stay safe and work efficiently around humans. Since the robot can only stay safe with respect to its own model of the human, we want the robot to learn a good model of the human in order to act both safely and efficiently. This paper studies methods that enable a robot to safely explore the space of a human-robot system to improve the robot’s model of the human, which will consequently allow the robot to access a larger state space and better work with the human. In particular, we introduce active exploration under the framework of energy-function based safe control, investigate the effect of different active exploration strategies, and finally analyze the effect of safe active exploration on both analytical and neural network human models.
Video:
[C45] A Composable Framework for Policy Design, Learning, and Transfer Toward Safe and Efficient Industrial Insertion
Rui Chen, Chenxi Wang, Tianhao Wei and Changliu Liu
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022
Citation Formats:
Abstract:
Delicate industrial insertion tasks (e.g., PC board assembly) remain challenging for industrial robots. The challenges include low error tolerance, delicacy of the components, and large task variations with respect to the components to be inserted. To deliver a feasible robotic solution for these insertion tasks, we also need to account for hardware limits of existing robotic systems and minimize the integration effort. This paper proposes a composable framework for efficient integration of a safe insertion policy on existing robotic platforms to accomplish these insertion tasks. The policy has an interpretable modularized design and can be learned efficiently on hardware and transferred to new tasks easily. In particular, the policy includes a safe insertion agent as a baseline policy for insertion, an optimal configurable Cartesian tracker as an interface to robot hardware, a probabilistic inference module to handle component variety and insertion errors, and a safe learning module to optimize the parameters in the aforementioned modules to achieve the best performance on designated hardware. The experiment results on a UR10 robot show that the proposed framework achieves safety (for the delicacy of components), accuracy (for low tolerance), robustness (against perception error and component defection), adaptability and transferability (for task variations), as well as task efficiency during execution plus data and time efficiency during learning.
Video:
[C44] Jerk-bounded Position Controller with Real-Time Task Modification for Interactive Industrial Robots Ruixuan Liu, Rui Chen, Yifan Sun, Yu Zhao and Changliu Liu
IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2022
Citation Formats:
Abstract:
Industrial robots are widely used in many applications with structured and deterministic environments. However, the contemporary need requires industrial robots to intelligently operate in dynamic environments. It is challenging to design a safe and efficient robotic system with industrial robots in a dynamic environment for several reasons. First, most industrial robots require the input to have specific formats, which takes additional efforts to convert from task-level user commands. Second, existing robot drivers do not support overwriting ongoing tasks in real-time, which hinders the robot from responding to the dynamic environment. Third, most industrial robots only expose motion-level control, making it challenging to enforce dynamic constraints during trajectory tracking. To resolve the above challenges, this paper presents a jerk-bounded position control driver (JPC) for industrial robots. JPC provides a unified interface for tracking complex trajectories and is able to enforce dynamic constraints using motion-level control, without accessing servo-level control. Most importantly, JPC enables real-time trajectory modification. Users can overwrite the ongoing task with a new one without violating dynamic constraints. The proposed JPC is implemented and tested on the FANUC LR Mate 200id/7L robot with both artificially generated data and an interactive robot handover task. Experiments show that the proposed JPC can track complex trajectories accurately within dynamic limits and seamlessly switch to new trajectory references before the ongoing task ends.
Video:
[C43] Robust Task Planning for Assembly Lines with Human-Robot Collaboration
Jessica Leu, Yujiao Cheng, Changliu Liu and Masayoshi Tomizuka
International Symposium on Flexible Automation, 2022
Citation Formats:
Abstract:
Efficient and robust task planning for a human-robot collaboration (HRC) system remains challenging. The human-aware task planner needs to assign jobs to both robots and human workers so that they can work collaboratively to achieve better time efficiency. However, the complexity of the tasks and the stochastic nature of the human collaborators bring challenges to such task planning. To reduce the complexity of the planning problem, we utilize the hierarchical task model, which explicitly captures the sequential and parallel relationships of the task. We model human movements with the sigma-lognormal functions to account for human-induced uncertainties. A human action model adaptation scheme is applied during run-time, and it provides a measure for modeling the human-induced uncertainties. We propose a sampling-based method to estimate human job completion time uncertainties. Next, we propose a robust task planner, which formulates the planning problem as a robust optimization problem by considering the task structure and the uncertainties. We conduct simulations of a robot arm collaborating with a human worker in an electronics assembly setting. The results show that our proposed planner can reduce task completion time when human-induced uncertainties occur compared to the baseline planner.
[C42] Safe Interactive Industrial Robots using Jerk-based Safe Set Algorithm Ruixuan Liu, Rui Chen and Changliu Liu
International Symposium on Flexible Automation, 2022
Citation Formats:
Abstract:
The need to increase the flexibility of production lines is calling for robots to collaborate with human workers. However, existing interactive industrial robots only guarantee intrinsic safety (reduce collision impact), but not interactive safety (collision avoidance), which greatly limited their flexibility. The issue arises from two limitations in existing control software for industrial robots: 1) lack of support for real-time trajectory modification; 2) lack of intelligent safe control algorithms with guaranteed collision avoidance under robot dynamics constraints. To address the first issue, a jerk-bounded position controller (JPC) was developed previously. This paper addresses the second limitation, on top of the JPC. Specifically, we introduce a jerk-based safe set algorithm (JSSA) to ensure collision avoidance while considering the robot dynamics constraints. The JSSA greatly extends the scope of the original safe set algorithm, which has only been applied for second-order systems with unbounded accelerations. The JSSA is implemented on the FANUC LR Mate 200id/7L robot and validated with HRI tasks. Experiments show that the JSSA can consistently keep the robot at a safe distance from the human while executing the designated task.
Video:
[C41] Joint Synthesis of Safety Certificate and Safe Control Policy Using Constrained Reinforcement Learning
Haitong Ma, Changliu Liu, Shengbo Eben Li, Sifa Zheng and Jianyu Chen
Learning for Dynamics and Control Conference, 2022 Best Paper Finalist
Citation Formats:
Abstract:
Safety is the major consideration in controlling complex dynamical systems using reinforcement learning (RL), where the safety certificates can provide provable safety guarantees. A valid safety certificate is an energy function indicating that safe states are with low energy, and there exists a corresponding safe control policy that allows the energy function to always dissipate. The safety certificates and the safe control policies are closely related to each other and both challenging to synthesize. Therefore, existing learning-based studies treat either of them as prior knowledge to learn the other, limiting their applicability to general systems with unknown dynamics. This paper proposes a novel approach that simultaneously synthesizes the energy-function-based safety certificates and learns the safe control policies with constrained reinforcement learning (CRL). We do not rely on prior knowledge about either a prior control law or a perfect safety certificate. In particular, we formulate a loss function to optimize the safety certificate parameters by minimizing the occurrence of energy increases. By adding this optimization procedure as an outer loop to the Lagrangian-based CRL, we jointly update the policy and safety certificate parameters, and prove that they will converge to their respective local optima, the optimal safe policies and valid safety certificates. Finally, we evaluate our algorithms on multiple safety-critical benchmark environments. The results show that the proposed algorithm learns solidly safe policies with no constraint violation. The validity, or feasibility of synthesized safety certificates is also verified numerically.
Video:
[C40] Safe Control with Neural Network Dynamic Models
Tianhao Wei and Changliu Liu
Learning for Dynamics and Control Conference, 2022
Citation Formats:
Abstract:
Safety is critical in autonomous robotic systems. A safe control law should ensure forward invariance of a safe set (a subset in the state space). It has been extensively studied regarding how to derive a safe control law with a control-affine analytical dynamic model. However, how to formally derive a safe control law with Neural Network Dynamic Models (NNDM) remains unclear due to the lack of computationally tractable methods to deal with these black-box functions. In fact, even finding the control that minimizes an objective for NNDM without any safety constraint is still challenging. In this work, we propose MIND-SIS (Mixed Integer for Neural network Dynamic model with Safety Index Synthesis), the first method to synthesize safe control for NNDM. The method includes two parts: 1) SIS: an algorithm for the offline synthesis of the safety index (also called as a barrier function), which uses evolutionary methods and 2) MIND: an algorithm for online computation of the optimal and safe control signal, which solves a constrained optimization using a computationally efficient encoding of neural networks. It has been theoretically proved that MIND-SIS guarantees forward invariance and finite convergence to a subset of the user-defined safe set. And it has been numerically validated that MIND-SIS achieves safe and optimal control of NNDM. The optimality gap is less than 10−8, and the safety constraint violation is 0.
[U] Control barrier functions-based semi-definite programs (cbf-sdps): Robust safe control for dynamic systems with relative degree two safety indices
Jaskaran Singh Grover, Changliu Liu and Katia Sycara
arXiv:2208.12252, 2022
Citation Formats:
[U] The third international verification of neural networks competition (vnn-comp 2022): Summary and results
Mark Niklas Müller, Christopher Brix, Stanley Bak, Changliu Liu and Taylor T Johnson
arXiv:2212.10376, 2022
Citation Formats:
<pre id="citation-content-muller2022third"
data-bibtex="@article{muller2022third,
index = {U},
title = {The third international verification of neural networks competition (vnn-comp 2022): Summary and results},
author = {M{\"u}ller, Mark Niklas and Brix, Christopher and Bak, Stanley and Liu, Changliu and Johnson, Taylor T},
journal = {arXiv:2212.10376},
year = {2022}
}"
data-plaintext="M. Müller, C. Brix, S. Bak, C. Liu and T. Johnson, "The third international verification of neural networks competition (vnn-comp 2022): Summary and results" in arXiv:2212.10376, 2022."
style="white-space: pre-wrap; word-wrap: break-word;">
</pre>
[U] General place recognition survey: Towards the real-world autonomy age
Peng Yin, Shiqi Zhao, Ivan Cisneros, Abulikemu Abuduweili, Guoquan Huang, Micheal Milford, Changliu Liu, Howie Choset and Sebastian Scherer
arXiv:2209.04497, 2022
Citation Formats:
Abstract:
In the realm of robotics, the quest for achieving real-world autonomy, capable of executing large-scale and long-term operations, has positioned place recognition (PR) as a cornerstone technology. Despite the PR community’s remarkable strides over the past two decades, garnering attention from fields like computer vision and robotics, the development of PR methods that sufficiently support real-world robotic systems remains a challenge. This paper aims to bridge this gap by highlighting the crucial role of PR within the framework of Simultaneous Localization and Mapping (SLAM) 2.0. This new phase in robotic navigation calls for scalable, adaptable, and efficient PR solutions by integrating advanced artificial intelligence (AI) technologies. For this goal, we provide a comprehensive review of the current state-of-the-art (SOTA) advancements in PR, alongside the remaining challenges, and underscore its broad applications in robotics. This paper begins with an exploration of PR’s formulation and key research challenges. We extensively review literature, focusing on related methods on place representation and solutions to various PR challenges. Applications showcasing PR’s potential in robotics, key PR datasets, and open-source libraries are discussed. We conclude with a discussion on PR’s future directions and provide a summary of the literature covered at: https://github.com/MetaSLAM/GPRS_Survey.
[J9] Algorithms for Verifying Deep Neural Networks Changliu Liu, Tomer Arnon, Christopher Lazarus, Christopher Strong, Clark Barrett and Mykel J. Kochenderfer
Foundations and Trends in Optimization, 2021
Citation Formats:
Abstract:
Deep neural networks are widely used for nonlinear function approximation with applications spanning from computer vision to control. Although these networks involve the composition of simple arithmetic operations, it can be very challenging to verify whether a particular network satisfies certain input-output properties. This article surveys methods that have emerged recently for soundly verifying such properties. These methods borrow insights from reachability analysis, optimization, and search. We discuss fundamental differences and connections between existing algorithms. In addition, we provide pedagogical implementations of existing methods and compare them on a set of benchmark problems.
Video:
[J8] Human Motion Prediction Using Adaptable Recurrent Neural Networks and Inverse Kinematics Ruixuan Liu and Changliu Liu
IEEE Control Systems Letters, 2021
Citation Formats:
Abstract:
Human motion prediction, especially arm prediction, is critical to facilitate safe and efficient human-robot collaboration (HRC). This letter proposes a novel human motion prediction framework that combines a recurrent neural network (RNN) and inverse kinematics (IK) to predict human arm motion. A modified Kalman filter (MKF) is applied to adapt the model online. The proposed framework is tested on collected human motion data with up to 2 s prediction horizon. The experiments demonstrate that the proposed method improves the prediction accuracy by approximately 14% comparing to the state-of-art on seen situations. It stably adapts to unseen situations by keeping the maximum prediction error under 4 cm, which is 70% lower than other methods. Moreover, it is robust when the arm is partially occluded. The wrist prediction remains the same, while the elbow prediction has 20% less variation.
Video:
[J7] Robust nonlinear adaptation algorithms for multitask prediction networks
Abulikemu Abuduweili and Changliu Liu
International Journal of Adaptive Control and Signal Processing, 2021
Citation Formats:
Abstract:
High-fidelity behavior prediction of intelligent agents is critical in many applications, which is challenging due to the stochasticity, heterogeneity, and time-varying nature of agent behaviors. Prediction models that work for one individual may not be applicable to another. Besides, the prediction model trained on the training set may not generalize to the testing set. These challenges motivate the adoption of online adaptation algorithms to update prediction models in real-time to improve the prediction performance. This paper considers online adaptable multi-task prediction for bothintention and trajectory. The goal of online adaptation is to improve the performance of both intention and trajectory predictions with only the feedback of the observed trajectory. We first introduce a generic tau-step adaptation algorithm of the multi-task prediction model that updates the model parameters with the trajectory prediction error in recent tau steps. Inspired by Extended Kalman Filter (EKF), a base adaptationalgorithm Modified EKF with forgetting factor (MEKFtau) is introduced. In order to improve the performance of MEKFtau, generalized exponential moving average filtering techniques are adopted. Then this paper introduces a dynamic multi-epoch update strategy to effectively utilize samples received in real time. With all these extensions, we propose a robust online adaptation algorithm: MEKF with Moving Average and dynamic Multi-Epoch strategy (MEKFMA−ME). We empirically study the best set of parameters to adapt in the multi-task prediction model and demonstrate the effectiveness of the proposed adaptation algorithms to reduce the prediction error.
[J10] Safe and Sample-efficient Reinforcement Learning for Clustered Dynamic Environments
Hongyi Chen and Changliu Liu
IEEE Control and Systems Letters, 2021
Citation Formats:
Abstract:
This study proposes a safe and sample-efficient reinforcement learning (RL) framework to address two major challenges in developing applicable RL algorithms: satisfying safety constraints and efficiently learning with limited samples. To guarantee safety in real-world complex environments, we use the safe set algorithm (SSA) to monitor and modify the nominal controls, and evaluate SSA+RL in a clustered dynamic environment which is challenging to be solved by existing RL algorithms. However, the SSA+RL framework is usually not sample-efficient especially in reward-sparse environments, which has not been addressed in previous safe RL works. To improve the learning efficiency, we propose three techniques: (1) avoiding behaving overly conservative by adapting the SSA; (2) encouraging safe exploration using random network distillation with safety constraints; (3) improving policy convergence by treating SSA as expert demonstrations and directly learn from that. The experimental results show that our framework can achieve better safety performance compare to other safe RL methods during training and solve the task with substantially fewer episodes.
[C39] Parameter Identification for Multirobot Systems Using Optimization-based Controllers
Jaskaran Grover, Changliu Liu and Katia Sycara
2021 International Symposium on Multi-Robot and Multi-Agent Systems (MRS), 2021
Citation Formats:
Abstract:
This paper considers the problem of parameter identification for a multirobot system. We wish to understand when is it feasible for an adversarial observer to reverse-engineer the parameters of tasks being performed by a team of robots by simply observing their positions. We address this question by using the concept of persistency of excitation from system identification. Each robot in the team uses optimization-based controllers for mediating between task satisfaction and collision avoidance. These controllers exhibit an implicit dependence on the task’s parameters which poses a hurdle for deriving necessary conditions for parameter identification, since such conditions usually require an explicit relation. We address this bottleneck by using duality theory and SVD of active collision avoidance constraints and derive an explicit relation between each robot’s task parameters and its control inputs. This allows us to derive the main necessary conditions for successful identification which agree with our intuition. We demonstrate the importance of these conditions through numerical simulations by using (a) an adaptive observer and (b) an unscented Kalman filter for goal estimation in various geometric settings. These simulations show that under circumstances where parameter inference is supposed to be infeasible per our conditions, both these estimators fail and likewise when it is feasible, both converge to the true parameters.
[C38] Model-free Safe Control for Zero-Violation Reinforcement Learning
Weiye Zhao, Tairan He and Changliu Liu
Conference on Robot Learning, 2021
Citation Formats:
Abstract:
Maintaining safety under adaptation has long been considered to be an important capability for autonomous systems. As these systems estimate and change the ego-model of the system dynamics, questions regarding how to develop safety guarantees for such systems continue to be of interest. We propose a novel robust safe control methodology that uses set-based safety constraints to make a robotic system with dynamical uncertainties safely adapt and operate in its environment. The method consists of designing a scalar energy function (safety index) for an adaptive system with parametric uncertainty and an optimization-based approach for control synthesis. Simulation studies on a two-link manipulator are conducted and the results demonstrate the effectiveness of our proposed method in terms of generating provably safe control for adaptive systems with parametric uncertainty.
[C37] Safe Adaptation with Multiplicative Uncertainties Using Robust Safe Set Algorithm
Charles Noren, Weiye Zhao and Changliu Liu
Modeling, Estimation, and Control Conference, 2021
Citation Formats:
Abstract:
Maintaining safety under adaptation has long been considered to be an important capability for autonomous systems. As these systems estimate and change the ego-model of the system dynamics, questions regarding how to develop safety guarantees for such systems continue to be of interest. We propose a novel robust safe control methodology that uses set-based safety constraints to make a robotic system with dynamical uncertainties safely adapt and operate in its environment. The method consists of designing a scalar energy function (safety index) for an adaptive system with parametric uncertainty and an optimization-based approach for control synthesis. Simulation studies on a two-link manipulator are conducted and the results demonstrate the effectiveness of our proposed method in terms of generating provably safe control for adaptive systems with parametric uncertainty.
[C36] System Identification for Safe Controllers using Inverse Optimization
Jaskaran Singh Grover, Changliu Liu and Katia Sycara
Modeling, Estimation, and Control Conference, 2021
Citation Formats:
Abstract:
This paper presents algorithms for learning parameters of optimization-based controllers used in multiagent systems based on their position-velocity measurements. The motivation to learn these parameters stems from the need to infer an agent’s intent (human or robot) to facilitate accurate predictions of motion as well as efficient interactions in a multiagent system. In this work, we demonstrate how to perform inference using algorithms based on the theory of inverse optimization (IO). We propose QP-based reformulations of IO algorithms for faster processing of batch-data to facilitate quicker inference. In our prior work, we used persistency of excitation analysis for deriving conditions under which conventional estimators such as a Kalman filter can successfully perform such inference. In this work, we demonstrate that whenever these conditions are violated, inference of parameters will fail, be it using IO-based algorithms or a UKF. We provide numerical simulations to infer desired goal locations and controller gains of each robot in a multirobot system and compare performance of IO-based algorithms with a UKF and an adaptive observer. In addition to these, we also conduct experiments with Khepera-4 robots and demonstrate the power of IO-based algorithms in inferring goals in the presence of perception noise.
Video:
[C35] Distributed Motion Coordination Using Convex Feasible Set Based Model Predictive Control
Hongyu Zhou and Changliu Liu
IEEE International Conference on Robotics and Automation, 2021
Citation Formats:
Abstract:
The implementation of optimization-based motion coordination approaches in real world multi-agent systems remains challenging due to their high computational complexity and potential deadlocks. This paper presents a distributed model predictive control (MPC) approach based on convex feasible set (CFS) algorithm for multi-vehicle motion coordination in autonomous driving. By using CFS to convexify the collision avoidance constraints, collision-free trajectories can be computed in real time. We analyze the potential deadlocks and show that a deadlock can be resolved by changing vehicles’ desired speeds. The MPC structure ensures that our algorithm is robust to low-level tracking errors. The proposed distributed method has been tested in multiple challenging multi-vehicle environments, including unstructured road, intersection, crossing, platoon formation, merging, and overtaking scenarios. The numerical results and comparison with other approaches (including a centralized MPC and reciprocal velocity obstacles) show that the proposed method is computationally efficient and robust, and avoids deadlocks.
Video:
[C34] Feasible Region-based Identification Using Duality
Jaskaran Grover, Changliu Liu and Katia Sycara
European Control Conference, 2021
Citation Formats:
Abstract:
We consider the problem of estimating bounds on parameters representing tasks being performed by individual robots in a multirobot system. In our previous work, we derived necessary conditions based on persistency of excitation analysis for exact identification of these parameters. We concluded that depending on the robot’s task, the dynamics of individual robots may fail to satisfy these conditions, thereby preventing exact inference. As an extension to that work, this paper focuses on estimating bounds on task parameters when such conditions are not satisfied. Each robot in the team uses optimization-based controllers for mediating between task satisfaction and collision avoidance. We use KKT conditions of this optimization and SVD of active collision avoidance constraints to derive explicit relations between Lagrange multipliers, robot dynamics and task parameters. Using these relations, we are able to derive bounds on each robot’s task parameters. Through numerical simulations we show how our proposed region based identification approach generates feasible regions for parameters when a conventional estimator such as a UKF fails. Additionally, empirical evidence shows that this approach generates contracting sets which converge to the true parameters much faster than the rate at which a UKF based estimate converges. Videos of these results are available at https://bit.ly/2JDMgeJ.
Video:
[C33] Flexible MPC-based Conflict Resolution Using Online Adaptive ADMM
Jerry An, Giulia Giordano and Changliu Liu
European Control Conference, 2021
Citation Formats:
Abstract:
Decentralized conflict resolution for autonomous vehicles is needed in many places where a centralized method is not feasible, e.g., parking lots, rural roads, merge lanes, etc. However, existing methods generally do not fully utilize optimization in decentralized conflict resolution. We propose a decentralized conflict resolution method for autonomous vehicles based on a novel extension to the Alternating Direc- tions Method of Multipliers (ADMM), called Online Adaptive ADMM (OA-ADMM), and on Model Predictive Control (MPC). OA-ADMM is tailored to online systems, where fast and adaptive real-time optimization is crucial, and allows the use of safety information about the physical system to improve safety in real-time control. We prove convergence in the static case and give requirements for online convergence. Combining OA-ADMM and MPC allows for robust decentralized motion planning and control that seamlessly integrates decentralized conflict resolution. The effectiveness of our proposed method is shown through simulations in CARLA, an open-source vehicle simulator, resulting in a reduction of 47.93% in mean added delay compared with the next best method.
Video:
[W] Simultaneously learning safety margins and task parameters of multirobot systems
Jaskaran Singh Grover, Changliu Liu and Katia Sycara
RSS BI-MAS Workshop, 2021
Citation Formats:
Abstract:
We present an algorithm for learning constraint and objective function parameters of optimization-based controllers used in multirobot systems. Our proposed approach uses position-velocity measurements of each robot in the team to perform this inference. The motivation to learn these parameters stems from the need to infer an agent’s intent for accurate predictions of motion in a multiagent system. We develop an extension of our prior work in which we performed task learning assuming constraint parameters were known. In this work, we perform simultaneous learning of constraint and cost function parameters by posing it as a constrained nonconvex optimization problem. The cost function parameters that we learn encode information of the task being performed by each robot in the team whereas the constraint parameters encode information about individual safety margin distances and size of the safe control set for each robot. Our simulation results show the accurate reconstruction of both the constraint and cost function parameters and we analyze some failure cases.
[W] Online Verification of Deep Neural Networks under Domain or Weight Shift
Tianhao Wei and Changliu Liu
RSS R4P Workshop, 2021
Citation Formats:
Abstract:
Although neural networks are widely used, it remains challenging to formally verify the safety and robustness of neural networks in real-world applications. Existing methods are designed to verify the network before deployment, which are limited to relatively simple specifications and fixed networks. These methods are not ready to be applied to real-world problems with complex and/or dynamically changing specifications and networks. To effectively handle such problems, verification needs to be performed online when these changes take place. However, it is still challenging to run existing verification algorithms online. Our key insight is that we can leverage the temporal dependencies of these changes to accelerate the verification process. This paper establishes a novel framework for scalable online verification to solve real-world verification problems with dynamically changing specifications and/or networks. We propose three types of acceleration algorithms: Branch Management to reduce repetitive computation, Perturbation Tolerance to tolerate changes, and Incremental Computation to reuse previous results. Experiment results show that our algorithms achieve up to 100× acceleration, and thus show a promising way to extend neural network verification to real-world applications.
[W] IADA: Iterative Adversarial Data Augmentation Using Formal Verification and Expert Guidance Ruixuan Liu and Changliu Liu
ICML Workshop on Human in the Loop Learning, 2021
Citation Formats:
Abstract:
Neural networks (NNs) are widely used for classification tasks for their remarkable performance. However, the robustness and accuracy of NNs heavily depend on the training data. In many applications, massive training data is usually not available. To address the challenge, this paper proposes an iterative adversarial data augmentation (IADA) framework to learn neural network models from an insufficient amount of training data. The method uses formal verification to identify the most "confusing" input samples, and leverages human guidance to safely and iteratively augment the training data with these samples. The proposed framework is applied to an artificial 2D dataset, the MNIST dataset, and a human motion dataset. By applying IADA to fully-connected NN classifiers, we show that our training method can improve the robustness and accuracy of the learned model. By comparing to regular supervised training, on the MNIST dataset, the average perturbation bound improved 107.4%. The classification accuracy improved 1.77%, 3.76%, 10.85% on the 2D dataset, the MNIST dataset, and the human motion dataset respectively.
[W] Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction
Letian Wang, Yeping Hu, Liting Sun, Wei Zhan, Masayoshi Tomizuka and Changliu Liu
NeurIPS workshop on Machine Learning for Autonomous Driving, 2021
Citation Formats:
Abstract:
When autonomous vehicles still struggle to solve challenging situations during on-road driving, humans have long mastered the essence of driving with efficient transferable and adaptable driving capability. By mimicking humans’ cognition model and semantic understanding during driving, we present HATN, a hierarchical framework to generate high-quality driving behaviors in multi-agent dense-traffic environments. Our method hierarchically consists of a high-level intention identification and low-level action generation policy. With the semantic sub-task definition and generic state representation, the hierarchical framework is transferable across different driving scenarios. Besides, our model is also able to capture variations of driving behaviors among individuals and scenarios by an online adaptation module. We demonstrate our algorithms in the task of trajectory prediction for real traffic data at intersections and roundabouts, where we conducted extensive studies of the proposed method and demonstrated how our method outperformed other methods in terms of prediction accuracy and transferability.
[W] Online Adaptation of Neural Network Models by Modified Extended Kalman Filter for Customizable and Transferable Driving Behavior Prediction
Letian Wang, Yeping Hu and Changliu Liu
AAAI Workshop HCSSL, 2021
Citation Formats:
Abstract:
High fidelity behavior prediction of human drivers is crucial for efficient and safe deployment of autonomous vehicles, which is challenging due to the stochasticity, heterogeneity, and time-varying nature of human behaviors. On one hand, the trained prediction model can only capture the motion pattern in an average sense, while the nuances among individuals can hardly be reflected. On the other hand, the prediction model trained on the training set may not generalize to the testing set which may be in a different scenario or data distribution, resulting in low transferability and generalizability. In this paper, we applied a τ-step modified Extended Kalman Filter parameter adaptation algorithm (MEKFλ) to the driving behavior prediction task, which has not been studied before in literature. With the feedback of the observed trajectory, the algorithm is applied to neural-network-based models to improve the performance of driving behavior predictions across different human subjects and scenarios. A new set of metrics is proposed for systematic evaluation of online adaptation performance in reducing the prediction error for different individuals and scenarios. Empirical studies on the best layer in the model and steps of observation to adapt are also provided.
[U] A microscopic pandemic simulator for pandemic prediction using scalable million-agent reinforcement learning
Zhenggang Tang, Kai Yan, Liting Sun, Wei Zhan and Changliu Liu
arXiv:2108.06589, 2021
Citation Formats:
Abstract:
Microscopic epidemic models are powerful tools for government policy makers to predict and simulate epidemic outbreaks, which can capture the impact of individual behaviors on the macroscopic phenomenon. However, existing models only consider simple rule-based individual behaviors, limiting their applicability. This paper proposes a deep-reinforcement-learning-powered microscopic model named Microscopic Pandemic Simulator (MPS). By replacing rule-based agents with rational agents whose behaviors are driven to maximize rewards, the MPS provides a better approximation of real world dynamics. To efficiently simulate with massive amounts of agents in MPS, we propose Scalable Million-Agent DQN (SMADQN). The MPS allows us to efficiently evaluate the impact of different government strategies. This paper first calibrates the MPS against real-world data in Allegheny, US, then demonstratively evaluates two government strategies: information disclosure and quarantine. The results validate the effectiveness of the proposed method. As a broad impact, this paper provides novel insights for the application of DRL in large scale agent-based networks such as economic and social networks.
[U] The second international verification of neural networks competition (VNN-COMP 2021): summary and results. CoRR abs/2109.00498 (2021)
Stanley Bak, Changliu Liu and Taylor T Johnson
arXiv:2109.00498, 2021
Citation Formats:
Abstract:
This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021), held as a part of the 4th Workshop on Formal Methods for ML-Enabled Autonomous Systems that was collocated with the 33rd International Conference on Computer-Aided Verification (CAV). Twelve teams participated in this competition. The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification, in terms of scalability and speed. Along this line, we used standard formats (ONNX for neural networks and VNNLIB for specifications), standard hardware (all tools are run by the organizers on AWS), and tool parameters provided by the tool authors. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.
[U] iLoc: Condition and Viewpoint Invariant Additive Feature Extractor for Appearance-based Visual Localization
Peng Yin, Ruohai Ge, Lingyun Xu, Changliu Liu, Ji Zhang, Howie Choset and Sebastian Scherer
, 2021
[J6] Towards Efficient Human-Robot Collaboration With Robust Plan Recognition and Trajectory Prediction
Yujiao Cheng, Liting Sun, Changliu Liu and Masayoshi Tomizuka
IEEE Robotics and Automation Letters, 2020
Citation Formats:
Abstract:
Human-robot collaboration (HRC) is becoming increasingly important as the paradigm of manufacturing is shifting from mass production to mass customization. The introduction of HRC can significantly improve the flexibility and intelligence of automation. To efficiently finish tasks in HRC systems, the robots need to not only predict the future movements of human, but also more high-level plans, i.e., the sequence of actions to finish the tasks. However, due to the stochastic and time-varying nature of human collaborators, it is quite challenging for the robot to efficiently and accurately identify such task plans and respond in a safe manner. To address this challenge, we propose an integrated human-robot collaboration framework. Both plan recognition and trajectory prediction modules are included for the generation of safe and efficient robotic motions. Such a framework enables the robots to perceive, predict and adapt their actions to the human’s work plan and intelligently avoid collisions with the human. Moreover, by explicitly leveraging the hierarchical relationship between plans and trajectories, more robust plan recognition performance can be achieved. Physical experiments were conducted on an industrial robot to verify the proposed framework. The results show that the proposed framework could accurately recognize the human workers’ plans and thus significantly improve the time efficiency of the HRC team even in the presence of motion classification noises.
[C32] Augmenting GAIL with BC for sample efficient imitation learning
Rohit Jena, Changliu Liu and Katia Sycara
Conference on Robot Learning, 2020
Citation Formats:
Abstract:
Imitation learning is the problem of recovering an expert policy without access to a reward signal. Behavior cloning and GAIL are two widely used methods for performing imitation learning. Behavior cloning converges in a few iterations, but doesn’t achieve peak performance due to its inherent iid assumption about the state-action distribution. GAIL addresses the issue by accounting for the temporal dependencies when performing a state distribution matching between the agent and the expert. Although GAIL is sample efficient in the number of expert trajectories required, it is still not very sample efficient in terms of the environment interactions needed for convergence of the policy. Given the complementary benefits of both methods, we present a simple and elegant method to combine both methods to enable stable and sample efficient learning. Our algorithm is very simple to implement and integrates with different policy gradient algorithms. We demonstrate the effectiveness of the algorithm in low dimensional control tasks, gridworlds and in high dimensional image-based tasks.
[C31] Tolerance-guided Policy Learning for Adaptable and Transferrable Delicate Industrial Insertion
Boshen Niu, Chenxi Wang and Changliu Liu
Conference on Robot Learning, 2020
Citation Formats:
Abstract:
Policy learning for delicate industrial insertion tasks (e.g., PC board assembly) is challenging. This paper considers two major problems: how to learn a diversified policy (instead of just one average policy) that can efficiently handle different workpieces with minimum amount of training data, and how to handle defects of workpieces during insertion. To address the problems, we propose tolerance-guided policy learning. To encourage transferability of the learned policy to different workpieces, we add a task embedding to the policy’s input space using the insertion tolerance. Then we train the policy using generative adversarial imitation learning with reward shaping (RS-GAIL) on a variety of representative situations. To encourage adaptability of the learned policy to handle defects, we build a probabilistic inference model that can output the best inserting pose based on failed insertions using the tolerance model. The best inserting pose is then used as a reference to the learned policy. This proposed method is validated on a sequence of IC socket insertion tasks in simulation. The results show that 1) RS-GAIL can efficiently learn optimal policies under sparse rewards; 2) the tolerance embedding can enhance the transferability of the learned policy; 3) the probabilistic inference makes the policy robust to defects on the workpieces.
[C30] Contact-Rich Trajectory Generation in Confined Environments Using Iterative Convex Optimization
Wei-Ye Zhao, Suqin He, Chengtao Wen and Changliu Liu
Dynamic Systems and Control Conference, 2020
Citation Formats:
Abstract:
Applying intelligent robot arms in dynamic uncertain environments (i.e., flexible production lines) remains challenging, which requires efficient algorithms for real time trajectory generation. The motion planning problem for robot trajectory generation is highly nonlinear and nonconvex, which usually comes with collision avoidance constraints, robot kinematics and dynamics constraints, and task constraints (e.g., following a Cartesian trajectory defined on a surface and maintain the contact). The nonlinear and nonconvex planning problem is computationally expensive to solve, which limits the application of robot arms in the real world. In this paper, for redundant robot arm planning problems with complex constraints, we present a motion planning method using iterative convex optimization that can efficiently handle the constraints and generate optimal trajectories in real time. The proposed planner guarantees the satisfaction of the contact-rich task constraints and avoids collision in confined environments. Extensive experiments on trajectory generation for weld grinding are performed to demonstrate the effectiveness of the proposed method and its applicability in advanced robotic manufacturing.
Video:
[C29] Multi-Car Convex Feasible Set Algorithm in Trajectory Planning
Jing Huang and Changliu Liu
Dynamic Systems and Control Conference, 2020
Citation Formats:
Abstract:
Trajectory planning is an essential module for autonomous driving. To deal with multi-vehicle interactions, existing methods follow the prediction-then-plan approaches which first predict the trajectories of others then plan the trajectory for the ego vehicle given the predictions. However, since the true trajectories of others may deviate from the predictions, frequent re-planning for the ego vehicle is needed, which may cause many issues such as instability or deadlock. These issues can be overcome if all vehicles can form a consensus by solving the same multi-vehicle trajectory planning problem. Then the major challenge is how to efficiently solve the multi-vehicle trajectory planning problem in real time under the curse of dimensionality. We introduce a novel planner for multi-vehicle trajectory planning based on the convex feasible set (CFS) algorithm. The planning problem is formulated as a non-convex optimization. A novel convexification method to obtain the maximal convex feasible set is proposed, which transforms the problem into a quadratic programming. Simulations in multiple typical on-road driving situations are conducted to demonstrate the effectiveness of the proposed planning algorithm in terms of completeness and optimality.
Video:
[C28] Deadlock Analysis and Resolution in Multi-Robot Systems
Jaskaran Grover, Changliu Liu and Katia Sycara
International Workshop on the Algorithmic Foundations of Robotics, 2020
Citation Formats:
Abstract:
Collision avoidance for multirobot systems is a well studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers guarantee collision avoidance and goal stabilization for multiple robots. However, it has been noted reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics causes robots to come to a standstill before reaching their goals. In this paper, we formally derive characteristics of deadlock in a multirobot system uses CBFs. We propose a novel approach to analyze deadlocks resulting from optimization based controllers (CBFs) by bor- rowing tools from duality theory and graph enumeration. Our key insight is system deadlock is characterized by a force-equilibrium on robots and we show how complexity of deadlock analysis increases approximately exponentially with the number of robots. This analysis allows us to interpret deadlock as a subset of the state space, and we prove this set is non-empty, bounded and located on the boundary of the safety set. Finally, we use these properties to develop a provably correct decentralized algorithm for deadlock resolution which ensures robots converge to their goals while avoiding collisions. We show simulation results of the resolution algorithm for two and three robots and experimentally validate this algorithm on Khepera-IV robots.
[C27] A Dynamic Programming Approach to Optimal Lane Merging of Connected and Autonomous Vehicles
Shang-Chien Lin, Hsiang Hsu, Yi-Ting Lin, Chung-Wei Lin, Iris Hui-Ru Jiang and Changliu Liu
IEEE Intelligent Vehicles Symposium, 2020
Citation Formats:
Abstract:
Lane merging is one of the major sources causing traffic congestion and delay. With the help of vehicle-to-vehicle or vehicle-to-infrastructure communication and autonomous driving technology, there are opportunities to alleviate congestion and delay resulting from lane merging. In this paper, we first summarize modern features and requirements for lane merging, along with the advance of vehicular technology. We then formulate and propose a dynamic programming algorithm to find the optimal solution for a two-lane merging scenario. It schedules the passing order for vehicles while minimizing the time needed for all vehicles to go through the merging point (equivalent to the time that the last vehicle goes through the merging point). We further extend the problem to a consecutive lane-merging scenario. We show the difficulty to apply the original dynamic programming to the consecutive lane-merging scenario and propose an improved version to solve it. Experimental results show that our dynamic programming algorithm can efficiently minimize the time needed for all vehicles to go through the merging point and reduce the average delay of all vehicles, compared with some greedy methods.
[C26] Robust Online Model Adaptation by Extended Kalman Filter with Exponential Moving Average and Dynamic Multi-Epoch Strategy
Abulikemu Abuduweili and Changliu Liu
Learning for Dynamics and Control Conference, 2020
Citation Formats:
Abstract:
High-fidelity behavior prediction of intelligent agents is critical in many applications. However, the prediction model trained on the training set may not generalize to the testing set due to domain shift and time variance. The challenge motivates the adoption of online adaptation algorithms to update prediction models in real-time to improve the prediction performance. Inspired by the Extended Kalman Filter (EKF), this paper introduces a series of online adaptation methods, which are applicable to neural network-based models. A base adaptation algorithm, Modified EKF with forgetting factor (MEKF_lambda) is introduced first, followed by exponential moving average filtering techniques. Then, this paper introduces a dynamic multi-epoch update strategy to effectively utilize samples received in real time. With all these extensions, we propose a robust online adaptation algorithm: MEKF with Exponential Moving Average and Dynamic Multi-Epoch strategy (MEKF_EMA-DME). The proposed algorithm outperforms existing methods as demonstrated in experiments.
Video:
[C25] Why Does Symmetry Cause Deadlocks?
Jaskaran Grover, Changliu Liu and Katia Sycara
IFAC-PapersOnLine, 2020
Citation Formats:
Abstract:
Collision avoidance for multirobot systems has been studied thoroughly. Recently, control barrier functions (CBFs) have been proposed to mediate between collision avoidance and goal achievement for multiple robots. However, it has been noted that reactive controllers (such as CBFs) are prone to deadlock, an equilibrium that causes the robots to stall before reaching their goals. In this paper, we formally analyze two and three robot systems and discover circumstances under which CBFs cause deadlocks using duality theory. For the two robot system, we consider mutually heterogeneous robots (such as one more vigorous or closer to its goal than the other) and prove that this heterogeneity does not help in preventing deadlock. We then consider three robots, and conclude from these two scenarios that the geometric symmetry resulting from robots’ initial positions and goals constrains CBFs to generate velocities that render deadlock stable. Thus, conferring skewness to the system can help evade deadlock.
Video:
[C24] Experimental Evaluation of Human Motion Prediction: Toward Safe and Efficient Human Robot Collaboration
Weiye Zhao, Liting Sun, Changliu Liu and Masayoshi Tomizuka
American Control Conference, 2020
Citation Formats:
Abstract:
Human motion prediction is non-trivial in modern industrial settings. Accurate prediction of human motion can not only improve efficiency in human robot collaboration, but also enhance human safety in close proximity to robots. Among existing prediction models, the parameterization and identification methods of those models vary. It remains unclear what is the necessary parameterization of a prediction model, whether online adaptation of the model is necessary, and whether prediction can help improve safety and efficiency during human robot collaboration. These problems result from the difficulty to quantitatively evaluate various prediction models in a closed-loop fashion in real human-robot interaction settings. This paper develops a method to evaluate the closed-loop performance of different prediction models. In particular, we compare models with different parameterizations and models with or without online parameter adaptation. Extensive experiments were conducted on a human robot collaboration platform. The experimental results demonstrated that human motion prediction significantly enhanced the collaboration efficiency and human safety. Adaptable prediction models that were parameterized by neural networks achieved the best performance.
Video:
[W] “Provably Safe” in the Wild: Testing Control Barrier Functions on a Vision-Based Quadrotor in an Outdoor Environment
Cherie Ho, Katherine Shih, Jaskaran Singh Grover, Changliu Liu and Sebastian Scherer
Proceedings of RSS ’20 2nd Workshop on Robust Autonomy: Safe Robot Learning and Control in Uncertain Real-World Environments, 2020
Citation Formats:
[U] A Microscopic Epidemic Model and Pandemic Prediction Using Multi-Agent Reinforcement Learning Changliu Liu
arXiv:2004.12959, 2020
Citation Formats:
Abstract:
This paper introduces a microscopic approach to model epidemics, which can explicitly consider the consequences of individual’s decisions on the spread of the disease. We first formulate a microscopic multi-agent epidemic model where every agent can choose its activity level that affects the spread of the disease. Then by minimizing agents’ cost functions, we solve for the optimal decisions for individual agents in the framework of game theory and multi-agent reinforcement learning. Given the optimal decisions of all agents, we can make predictions about the spread of the disease. We show that there are negative externalities in the sense that infected agents do not have enough incentives to protect others, which then necessitates external interventions to regulate agents’ behaviors. In the discussion section, future directions are pointed out to make the model more realistic.
[J5] Graph-Based Modeling, Scheduling, and Verification for Intersection Management of Intelligent Vehicles
Yi-Ting Lin, Hsiang Hsu, Shang-Chien Lin, Chung-Wei Lin, Iris Hui-Ru Jiang and Changliu Liu
ACM Transactions on Embedded Computing Systems, 2019
Citation Formats:
Abstract:
Intersection management is one of the most representative applications of intelligent vehicles with connected and autonomous functions. The connectivity provides environmental information that a single vehicle cannot sense, and the autonomy supports precise vehicular control that a human driver cannot achieve. Intersection management solves the fundamental conflict resolution problem for vehicles—two vehicles should not appear at the same location at the same time, and, if they intend to do that, an order should be decided to optimize certain objectives such as the traffic throughput or smoothness. In this paper, we first propose a graph-based model for intersection management. The model is general and applicable to different granularities of intersections and other conflicting scenarios. We then derive formal verification approaches which can guarantee deadlock-freeness. Based on the graph-based model and the verification approaches, we develop a centralized cycle removal algorithm for the graph-based model to schedule vehicles to go through the intersection safely (without collisions) and efficiently without deadlocks. Experimental results demonstrate the expressiveness of the proposed model and the effectiveness and efficiency of the proposed algorithm.
[C23] Safe Control Algorithms Using Energy Functions: A Unified Framework, Benchmark, and New Directions
Tianhao Wei and Changliu Liu
IEEE Conference on Decision and Control, 2019
Citation Formats:
Abstract:
Safe autonomy is important in many application domains, especially for applications involving interactions with humans. Existing safe control algorithms are similar to each other in the sense that: they all provide control input to maintain a low value of an energy function that measures safety. In different methods, the energy function is called a potential function, a safety index, or a barrier function. The connections and relative advantages among these methods remain unclear. This paper introduces a unified framework to derive safe control laws using energy functions. We demonstrate how to integrate existing controllers based on potential field method, , barrier function method, and sliding mode algorithm into this unified framework. In addition to theoretical comparison, this paper also introduces a benchmark which implements and compares existing methods on a variety of problems with different system dynamics and interaction modes. Based on the comparison results, a new method, called the sublevel safe set algorithm, is derived under the unified framework by optimizing the hyperparameters. The proposed algorithm achieves the best performance in terms of safety and efficiency on all benchmark problems.
[C22] Toward Modularization of Neural Network Autonomous Driving Policy Using Parallel Attribute Networks
Zhuo Xu, Haonan Chang, Chen Tang, Changliu Liu and Masayoshi Tomizuka
IEEE Intelligent Vehicles Symposium, 2019
Citation Formats:
Abstract:
Neural network autonomous driving policies are widely explored. However, no matter using imitation learning or reinforcement learning, the network policies are generally hard to train, and the learned knowledge encoded in neural network policies are hard to transfer. We propose to modularize the complicated driving policies in terms of the driving attributes, and present the parallel attribute networks (PAN), which can learn to fullfill the requirements of the attributes in the driving tasks separately, and later assemble their knowledge together. Concretely, we first train a policy network that accomplish the base lane tracking attribute. The modules for the add-on attributes such as avoiding obstacles and obeying traffic rules are then trained to map the corresponding state to a satisfactory set of the vehicle action space. Finally the reference action given by the base policy is projected into the satisfactory sets so as to satisfy the requirements of all the attributes. Using the PAN, many complicated tasks that are hard to train from scratch can be easily trained; also unseen driving tasks can be solved in a zero-shot manner by assembling the pretrained attribute modules. We have validated the capability of our model on a class of autonomous driving problems with attributes of obstacle avoidance, traffic light and speed limit in simulation. Experimental results based on an obstacle avoidance task are also presented.
[C21] Agen: Adaptable generative prediction networks for autonomous driving
Wenwen Si, Tianhao Wei and Changliu Liu
IEEE Intelligent Vehicles Symposium, 2019
Citation Formats:
Abstract:
In highly interactive driving scenarios, accurate prediction of other road participants is critical for safe and efficient navigation of autonomous cars. Prediction is challenging due to the difficulty in modeling various driving behavior, or learning such a model. The model should be interactive and reflect individual differences. Imitation learning methods, such as parameter sharing generative adversarial imitation learning (PS-GAIL), are able to learn interactive models. However, the learned models average out individual differences. When used to predict trajectories of individual vehicles, these models are biased. This paper introduces an adaptable generative prediction framework (AGen), which performs online adaptation of the offline learned models to recover individual differences for better prediction. In particular, we combine the recursive least square parameter adaptation algorithm (RLS-PAA) with the offline learned model from PS-GAIL. RLS-PAA has analytical solutions and is able to adapt the model for every single vehicle efficiently online. The proposed method is able to reduce the root mean squared prediction error in a 2.5s time window by 60%, compared with PS-GAIL.
[C20] Human motion prediction using semi-adaptable neural networks
Yujiao Cheng, Weiye Zhao, Changliu Liu and Masayoshi Tomizuka
American Control Conference, 2019
Citation Formats:
Abstract:
Human motion prediction is an important component to facilitate human robot interaction. Robots need to accurately predict human’s future movement in order to efficiently collaborate with humans, as well as to safely plan its own motion trajectories. Many recent approaches predict human’s future movement using deep learning methods, such as recurrent neural networks. However, existing methods lack the ability to adapt to time-varying human behaviors. Moreover, many of them do not quantify uncertainties in the prediction. This paper proposes a new approach that uses an adaptable neural network for human motion prediction, in order to accommodate human’s time-varying behaviors and to provide uncertainty bounds of the predictions in real time. In particular, a neural network is trained offline to represent the human motion transition model. Recursive least square parameter adaptation algorithm (RLS-PAA) is adopted for online parameter adaptation of the neural network and for uncertainty estimation. Experiments on several human motion datasets verify that the proposed method outperforms the state-of-the-art approach with a significant improvement in terms of prediction accuracy and computation efficiency.
[C19] Simulating emergent properties of human driving behavior using multi-agent reward augmented imitation learning
Raunak P Bhattacharyya, Derek J Phillips, Changliu Liu, Jayesh K Gupta, Katherine Driggs-Campbell and Mykel J Kochenderfer
IEEE International Conference on Robotics and Automation, 2019
Citation Formats:
Abstract:
Recent developments in multi-agent imitation learning have shown promising results for modeling the behavior of human drivers. However, it is challenging to capture emergent traffic behaviors that are observed in real-world datasets. Such behaviors arise due to the many local interactions between agents that are not commonly accounted for in imitation learning. This paper proposes Reward Augmented Imitation Learning (RAIL), which integrates reward augmentation into the multi-agent imitation learning framework and allows the designer to specify prior knowledge in a principled fashion. We prove that convergence guarantees for the imitation learning process are preserved under the application of reward augmentation. This method is validated in a driving scenario, where an entire traffic scene is controlled by driving policies learned using our proposed algorithm. Further, we demonstrate improved performance in comparison to traditional imitation learning algorithms both in terms of the local actions of a single agent and the behavior of emergent properties in complex, multi-agent settings.
[W] NeuralVerification.jl: Algorithms for verifying deep neural networks Changliu Liu, Tomer Arnon, Christopher Lazarus and Mykel J Kochenderfer
ICLR 2019 Debugging Machine Learning Models Workshop, 2019
Citation Formats:
[W] Adaptable Human Intention and Trajectory Prediction for Human-Robot Collaboration
Abulikemu Abuduweili, Siyan Li and Changliu Liu
AAAI 2019 Fall Symposium Series, AI for HRI, 2019
Citation Formats:
Abstract:
To engender safe and efficient human-robot collaboration, it is critical to generate high-fidelity predictions of human behavior. The challenges in making accurate predictions lie in the stochasticity and heterogeneity in human behaviors. This paper introduces a method for human trajectory and intention prediction through a multi-task model that is adaptable across different human subjects. We develop a nonlinear recursive least square parameter adaptation algorithm (NRLS-PAA) to achieve online adaptation. The effectiveness and flexibility of the proposed method has been validated in experiments. In particular, online adaptation can reduce the trajectory prediction error by more than 28% for a new human subject. The proposed human prediction method has high flexibility, data efficiency, and generalizability, which can support fast integration of HRC systems for user-specified tasks.
Video:
[B2] Designing robot behavior in human-robot interactions Changliu Liu, Te Tang, Hsien-Chung Lin and Masayoshi Tomizuka
CRC Press, 2019
Citation Formats:
Abstract:
Human-robot interactions (HRI) have been recognized to be a key element of future robots in many application domains such as manufacturing and transportation, which entail huge social and economic impacts. Future robots are envisioned to be acting like human, which are independent entities that make decisions for themselves; intelligent actuators that interact with the physical world; and involved observers that have rich senses and critical judgements. Most importantly, they are entitled social attributions to build relationships with humans. We call these robots co-robots. Technically, it is challenging to design the behavior of co-robots. Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. The book is to study methodologies to ensure that co-robots operate efficiently and safely in dynamic uncertain environments. This book sets up a unified analytical framework for various human-robot systems, which involves peer-peer interactions or hierarchical interactions. Various methods to design the robot behavior through control, planning, decision and learning are proposed. In particular, the following topics are discussed: safety during human-robot interactions, efficiency in real-time robot motion planning, imitation of human behaviors by robot, dexterity of robot to adapt to different environments and tasks, cooperation among robots and humans and conflict resolution. The proposed methods have been applied on various scenarios, such as human-robot collaborative assembly, robot skill learning from human demonstration, interaction between autonomous and human-driven vehicles, and etc.
[P] System and method for planning a vehicle path
Yizhou Wang, Changliu Liu, Xiaoying Chen, Chongyu Wang and Kai Ni
US Patent App. 15/691,617, 2019
[J3] The convex feasible set algorithm for real time optimization in motion planning Changliu Liu, Chung-Yen Lin and Masayoshi Tomizuka
SIAM Journal on Control and optimization, 2018
Citation Formats:
Abstract:
With the development of robotics, there are growing needs for real time motion planning. However, due to obstacles in the environment, the planning problem is highly non-convex, which makes it difficult to achieve real time computation using existing non-convex optimization algorithms. This paper introduces the convex feasible set algorithm (CFS) which is a fast algorithm for non-convex optimization problems that have convex costs and non-convex constraints. The idea is to find a convex feasible set for the original problem and iteratively solve a sequence of subproblems using the convex constraints. The feasibility and the convergence of the proposed algorithm are proved in the paper. The application of this method on motion planning for mobile robots is discussed. The simulations demonstrate the effectiveness of the proposed algorithm.
[C18] Fast Robot Motion Planning with Collision Avoidance and Temporal Optimization
Hsien-Chung Lin, Changliu Liu and Masayoshi Tomizuka
International Conference on Control, Automation, Robotics and Vision, 2018 Best Paper Award
Citation Formats:
Abstract:
Considering the growing demand of real-time motion planning in robot applications, this paper proposes a fast robot motion planner (FRMP) to plan collision-free and time-optimal trajectories, which applies the convex feasible set algorithm (CFS) to solve both the trajectory planning problem and the temporal optimization problem. The performance of CFS in trajectory planning is compared to the sequential quadratic programming (SQP) in simulation, which shows a significant decrease in iteration numbers and computation time to converge a solution. The effectiveness of temporal optimization is shown on the operational time reduction in the experiment on FANUC LR Mate 200iD/7L.
[C17] Analytically Modeling Unmanaged Intersections with Microscopic Vehicle Interactions Changliu Liu and Mykel J Kochenderfer
IEEE International Conference on Intelligent Transportation Systems, 2018
Citation Formats:
Abstract:
With the emergence of autonomous vehicles, it is important to understand their impact on the transportation system. However, conventional traffic simulations are time-consuming. In this paper, we introduce an analytical traffic model for unmanaged intersections accounting for microscopic vehicle interactions. The macroscopic property, i.e., delay at the intersection, is modeled as an event-driven stochastic dynamic process, whose dynamics encode the microscopic vehicle behaviors. The distribution of macroscopic properties can be obtained through either direct analysis or event-driven simulation. They are more efficient than conventional (time-driven) traffic simulation, and capture more microscopic details compared to conventional macroscopic flow models. We illustrate the efficiency of this method by delay analyses under two different policies at a two-lane intersection. The proposed model allows for 1) efficient and effective comparison among different policies, 2) policy optimization, 3) traffic prediction, and 4) system optimization (e.g., infrastructure and protocol).
Video:
[C16] Improving efficiency of autonomous vehicles by V2V communication Changliu Liu, Chung-Wei Lin, Shinichi Shiraishi and Masayoshi Tomizuka
American Control Conference, 2018
Citation Formats:
Abstract:
Autonomous vehicles are widely regarded as a promising technology to improve the safety of transportation systems. However, the efficiency of vehicles may be compromised to ensure safety when there are large uncertainties in perception and prediction of the behaviors of other road participants due to limitations in sensors. To remedy this problem, vehicle to vehicle (V2V) communication is applied to improve efficiency of autonomous vehicles during interactions with other vehicles. By requiring the vehicles to communicate their intentions with one another, the efficiency of the vehicles can be improved in terms of smaller variations in their speed profiles and smaller delay as demonstrated in the simulations.
[C15] FOAD: Fast optimization-based autonomous driving motion planner
Jianyu Chen, Changliu Liu and Masayoshi Tomizuka
American Control Conference, 2018
Citation Formats:
Abstract:
Motion planning is one of the core modules for autonomous driving. Among the current motion planning techniques, optimization-based methods have unique advantages since they allow planning in continuous space and they can evaluate multiple objectives (such as hard constraints) in one formulation. However, it is hard to implement optimization-based methods in real-time in complicated environments due to 1) high computational complexity as the optimization problems are usually non-convex; and 2) difficulty to guarantee closed-loop performance because the low level trajectory tracking controller cannot perform perfect tracking. To solve the first challenge, convex feasible set algorithm (CFS) has been proposed for real time non-convex optimization. To solve the second challenge, a fast optimization-based autonomous driving motion planner (FOAD) is proposed in this paper which implements a soft constrained convex feasible set algorithm (SCCFS) as an enhanced version of CFS. The concept of closed-loop smoothness is defined and analyzed in this paper. Simulations and real vehicle experiments verify the efficiency and capability of the planner.
[U] Analyzing traffic delay at unmanaged intersections Changliu Liu and Mykel J Kochenderfer
arXiv:1806.02660, 2018
Citation Formats:
Abstract:
At an unmanaged intersection, it is important to understand how much traffic delay may be caused as a result of microscopic vehicle interactions. Conventional traffic simulations that explicitly track these interactions are time-consuming. Prior work introduced an analytical traffic model for unmanaged intersections. The traffic delay at the intersection is modeled as an event-driven stochastic process, whose dynamics encode microscopic vehicle interactions. This paper studies the traffic delay in a two-lane intersection using the model. We perform rigorous analyses concerning the distribution of traffic delay under different scenarios. We then discuss the relationships between traffic delay and multiple factors such as traffic flow density, unevenness of traffic flows, temporal gaps between two consecutive vehicles, and the passing order.
Video:
[U] Robot safe interaction system for intelligent industrial co-robots Changliu Liu and Masayoshi Tomizuka
arXiv:1808.03983, 2018
Citation Formats:
Abstract:
Human-robot interactions have been recognized to be a key element of future industrial collaborative robots (co-robots). Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. To ensure that co-robots operate efficiently and safely in dynamic uncertain environments, this paper introduces the robot safe interaction system. In order to address the uncertainties during human-robot interactions, a unique parallel planning and control architecture is proposed, which has a long term global planner to ensure efficiency of robot behavior, and a short term local planner to ensure real time safety under uncertainties. In order for the robot to respond immediately to environmental changes, fast algorithms are used for real-time computation, i.e., the convex feasible set algorithm for the long term optimization, and the for the short term optimization. Several test platforms are introduced for safe evaluation of the developed system in the early phase of deployment. The effectiveness and the efficiency of the proposed method have been verified in experiment with an industrial robot manipulator.
[U] Serocs: Safe and efficient robot collaborative systems for next generation intelligent industrial co-robots Changliu Liu, Te Tang, Hsien-Chung Lin, Yujiao Cheng and Masayoshi Tomizuka
arXiv:1809.08215, 2018
Citation Formats:
Abstract:
Human-robot collaborations have been recognized as an essential component for future factories. It remains challenging to properly design the behavior of those co-robots. Those robots operate in dynamic uncertain environment with limited computation capacity. The design objective is to maximize their task efficiency while guaranteeing safety. This paper discusses a set of design principles of the safe and efficient robot collaboration system (SERoCS) for the next generation co-robots, which consists of robust cognition algorithms for environment monitoring, optimal task planning algorithms for safe human-robot collaborations, and safe motion planning and control algorithms for safe human-robot interactions. The proposed SERoCS will address the design challenges and significantly expand the skill sets of the co-robots to allow them work safely and efficiently with their human counterparts. The development of SERoCS will create a significant advancement toward adoption of co-robots in various industries. The experiments validate the effectiveness of SERoCS.
[J4] Distributed conflict resolution for connected autonomous vehicles Changliu Liu, Chung-Wei Lin, Shinichi Shiraishi and Masayoshi Tomizuka
IEEE Transactions on Intelligent Vehicles, 2017
Citation Formats:
Abstract:
This paper proposes a novel communication-enabled distributed conflict resolution mechanism for interactions among connected autonomous vehicles (CAVs). The environments under consideration are generalized intersections where multiple incoming and outgoing lanes intersect. All vehicles close to the intersection are requested to broadcast their estimated times to occupy the conflict zones where a conflict zone is identified when the extensions of two incoming lanes intersect. The conflict resolution strategy is decoupled temporally for a vehicle. In a decision maker, the vehicle computes the desired time slot to pass the conflict zones based on the broadcasted information by solving a conflict graph locally. Then in a motion planner, the vehicle computes the desired speed profile by solving a temporal optimization problem constrained in the desired time slot. The estimated time to occupy the conflict zones given the new speed profile is then broadcasted again. It is shown theoretically that the aggregation of these local decisions solves the conflicts globally. Moreover, this mechanism increases the efficiency of autonomous vehicles, and outperforms conventional mechanisms such as traffic light or stop sign in a four-way intersection in terms of delay time and throughput as demonstrated by simulation.
[J2] Real time trajectory optimization for nonlinear robotic systems: Relaxation and convexification Changliu Liu and Masayoshi Tomizuka
Systems & Control Letters, 2017
Citation Formats:
Abstract:
Real time trajectory optimization is critical for robotic systems. Due to nonlinear system dynamics and obstacles in the environment, the trajectory optimization problems are highly nonlinear and non convex, hence hard to be computed online. Liu, Lin and Tomizuka proposed the convex feasible set algorithm (CFS) to handle the non convex optimization in real time by convexification. However, one limitation of CFS is that it will not converge to local optima when there are nonlinear equality constraints. In this paper, the slack convex feasible set algorithm (SCFS) is proposed to handle the nonlinear equality constraints, e.g. nonlinear system dynamics, by introducing slack variables to relax the constraints. The geometric interpretation of the method is discussed. The feasibility and convergence of the SCFS algorithm is proved. It is demonstrated that SCFS performs better than existing non convex optimization methods such as interior-point, active set and sequential quadratic programming, as it requires less computation time and converges faster.
[J1] Safe Robot Navigation Among Moving and Steady Obstacles [Bookshelf] Changliu Liu
IEEE Control Systems Magazine, 2017
Citation Formats:
[C9] The robustly-safe automated driving system for enhanced active safety Changliu Liu, Jianyu Chen, Trong-Duy Nguyen and Masayoshi Tomizuka
SAE World Congress, 2017
Citation Formats:
Abstract:
The development of automated vehicles brings new challenges to road safety. The behavior of the automated vehicles should be carefully designed in order to interact with the environment and other vehicles efficiently and safely. The robustly-safe automated driving system (ROAD) prevents or minimizes occurrences of collisions of the automated vehicles with surrounding vehicles and moving objects while maintaining efficiency. A set of design principles are elaborated based on the previous work, including robust perception and cognition algorithms for environment monitoring and high level decision making and low level control algorithms for safe maneuvering of the automated vehicle. The autonomous driving problem in mixed traffic is posed as a stochastic optimization problem, which is solved by 1) behavior classification and trajectory prediction of the surrounding vehicles, and 2) an unique parallel planner architecture which addresses the efficiency goal and the safety goal separately. Extensive simulations are performed to validate the effectiveness of the proposed algorithm, which evaluate both high level decision making and low level vehicle regulation. Two typical scenarios are considered, driving on freeway and driving in unstructured environments such as parking lots. In the simulation, multiple moving agents representing surrounding vehicles and pedestrians are added to the environment, some of which are controlled by human users in order to test the real time response of the automated vehicle.
[C14] Real-time collision avoidance algorithm on industrial manipulators
Hsien-Chung Lin, Changliu Liu, Yongxiang Fan and Masayoshi Tomizuka
IEEE Conference on Control Technology and Applications, 2017
Citation Formats:
Abstract:
Safety is a fundamental issue in robotics, especially in the growing application of human-robot interaction (HRI), where collision avoidance is an important consideration. In this paper, a novel real-time velocity based collision avoidance planner is presented to address this problem. The proposed algorithm provides a solution to deal with both collision avoidance and reference tracking simultaneously. An invariant safe set is introduced to exclude the dangerous states that may lead to collision, and a smoothing function is introduced to adapt different reference commands and to preserve the invariant property of the safe set. A real-time experiment with a moving obstacle is conducted on FANUC LR Mate 200iD/7L.
[C13] Boundary layer heuristic for search-based nonholonomic path planning in maze-like environments Changliu Liu, Yizhou Wang and Masayoshi Tomizuka
IEEE Intelligent Vehicles Symposium, 2017
Citation Formats:
Abstract:
Automatic valet parking is widely viewed as a milestone towards fully autonomous driving. One of the key problems is nonholonomic path planning in maze-like environments (e.g. parking lots). To balance efficiency and passenger comfort, the planner needs to minimize the length of the path as well as the number of gear shifts. Lattice A* search is widely adopted for optimal path planning. However, existing heuristics do not evaluate the nonholonomic dynamic constraint and the collision avoidance constraint simultaneously, which may mislead the search. To efficiently search the environment, the boundary layer heuristic is proposed which puts large cost in the area that the vehicle must shift gear to escape. Such area is called the boundary layer. A simple and efficient geometric method to compute the boundary layer is proposed. The admissibility and consistency of the additive combination of the boundary layer heuristic and existing heuristics are proved in the paper. The simulation results verify that the introduction of the boundary layer heuristic improves the search performance by reducing the computation time by 56.1%.
[C12] Speed profile planning in dynamic environments via temporal optimization Changliu Liu, Wei Zhan and Masayoshi Tomizuka
IEEE Intelligent Vehicles Symposium, 2017
Citation Formats:
Abstract:
To generate safe and efficient trajectories for an automated vehicle in dynamic environments, a layered approach is usually considered, which separates path planning and speed profile planning. This paper is focused on speed profile planning for a given path that is represented by a set of waypoints. The speed profile will be generated using temporal optimization which optimizes the time stamps for all waypoints along the given path. The formulation of the problem under urban driving scenarios is discussed. To speed up the computation, the non-convex temporal optimization is approximated by a set of quadratic programs which are solved iteratively using the slack convex feasible set (SCFS) algorithm. The simulations in various urban driving scenarios validate the effectiveness of the method.
[C11] Spatially-partitioned environmental representation and planning architecture for on-road autonomous driving
Wei Zhan, Jianyu Chen, Ching-Yao Chan, Changliu Liu and Masayoshi Tomizuka
IEEE Intelligent Vehicles Symposium, 2017
Citation Formats:
Abstract:
Conventional layered planning architecture temporally partitions the spatiotemporal motion planning by the path and speed, which is not suitable for lane change and overtaking scenarios with moving obstacles. In this paper, we propose to spatially partition the motion planning by longitudinal and lateral motions along the rough reference path in the Frenét Frame, which makes it possible to create linearized safety constraints for each layer in a variety of on-road driving scenarios. A generic environmental representation methodology is proposed with three topological elements and corresponding longitudinal constraints to compose all driving scenarios mentioned in this paper according to the overlap between the potential path of the autonomous vehicle and predicted path of other road users. Planners combining A* search and quadratic programming (QP) are designed to plan both rough long-term longitudinal motions and short-term trajectories to exploit the advantages of both search-based and optimization-based methods. Limits of vehicle kinematics and dynamics are considered in the planners to handle extreme cases. Simulation results show that the proposed framework can plan collision-free motions with high driving quality under complicated scenarios and emergency situations.
[C10] Convex feasible set algorithm for constrained trajectory smoothing Changliu Liu, Chung-Yen Lin, Yizhou Wang and Masayoshi Tomizuka
American Control Conference, 2017
Citation Formats:
Abstract:
Trajectory smoothing is an important step in robot motion planning, where optimization methods are usually employed. However, the optimization problem for trajectory smoothing in a clustered environment is highly non-convex, and is hard to solve in real time using conventional non-convex optimization solvers. This paper discusses a fast online optimization algorithm for trajectory smoothing, which transforms the original non-convex problem to a convex problem so that it can be solved efficiently online. The performance of the algorithm is illustrated in various cases, and is compared to that of conventional sequential quadratic programming (SQP). It is shown that the computation time is greatly reduced using the proposed algorithm.
[T0] Designing robot behavior in human-robot interactions Changliu Liu
PhD Thesis, 2017
Citation Formats:
[B1] Designing the Robot Behavior for Safe Human–Robot Interactions Changliu Liu and Masayoshi Tomizuka
Trends in Control and Decision-Making for Human–Robot Collaboration Systems, 2017
Citation Formats:
Abstract:
Recent advances in robotics suggest that human robot interaction (HRI) is no longer a fantasy, but is happening in various fields such as industrial robots, autonomous vehicles and medical robots. Human safety is one of the biggest concerns in HRI. As humans will respond to the robot’s movement, interactions need to be considered explicitly by the robot. A systematic approach to design the robot behavior towards safe HRI is discussed in this chapter. By modeling the interactions in a multi-agent framework, the safety issues are understood as conflicts in the multi-agent system. By mimicking human’s social behavior, the robot’s behavior is constrained by the "no-collision" social norm and the uncertainties it perceives for human motions. An efficient action is then found within the constraints. Both analysis and human-involved simulation verify the effectiveness of the method.
[C8] A non-conservatively defensive strategy for urban autonomous driving
Wei Zhan, Changliu Liu, Ching-Yao Chan and Masayoshi Tomizuka
IEEE International Conference on Intelligent Transportation Systems, 2016
Citation Formats:
Abstract:
From the driving strategy point of view, a major challenge for autonomous vehicles in urban environment is to behave defensively to potential dangers, yet to not overreact to threats with low probability. As it is overwhelming to program the action rules case-by-case, a unified planning framework under uncertainty is proposed in this paper, which achieves a non-conservatively defensive strategy (NCDS) in various kinds of scenarios for urban autonomous driving. First, uncertainties in urban scenarios are simplified to two probabilistic cases, namely passing and yielding. Two-way-stop intersection is used as an exemplar scenario to illustrate the derivation of probabilities for different intentions of others via a logistic regression model. Then a deterministic planner is designed as the baseline. Also, a safe set is defined, which considers both current and preview safety. The planning framework under uncertainty is then proposed, in which safety is guaranteed and overcautious behavior is prevented. Finally, the proposed planning framework is tested by simulation in the exemplar scenario, which demonstrates that an NCDS can be realistically achieved by employing the proposed framework.
[C7] Robotic manipulation of deformable objects by tangent space mapping and non-rigid registration
Te Tang, Changliu Liu, Wenjie Chen and Masayoshi Tomizuka
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016
Citation Formats:
Abstract:
Recent works of non-rigid registration have shown promising applications on tasks of deformable manipulation. Those approaches use thin plate spline-robust point matching (TPS-RPM) algorithm to regress a transformation function, which could generate a corresponding manipulation trajectory given a new pose/shape of the object. However, this method regards the object as a bunch of discrete and independent points. Structural information, such as shape and length, is lost during the transformation. This limitation makes the object’s final shape to differ from training to test, and can sometimes cause damage to the object because of excessive stretching. To deal with these problems, this paper introduces a tangent space mapping (TSM) algorithm, which maps the deformable object in the tangent space instead of the Cartesian space to maintain structural information. The new algorithm is shown to be robust to the changes in the object’s pose/shape, and the object’s final shape is similar to that of training. It is also guaranteed not to overstretch the object during manipulation. A series of rope manipulation tests are performed to validate the effectiveness of the proposed algorithm.
[C6] Enabling safe freeway driving for automated vehicles Changliu Liu and Masayoshi Tomizuka
American Control Conference, 2016
Citation Formats:
Abstract:
The development of automated vehicles brings new challenges to road safety. The behavior of the automated vehicles should be carefully designed in order to interact with the environment and other vehicles efficiently and safely. This paper is focused on the learning and decision making methods for the automated vehicles towards safe freeway driving. Based on a multi-agent traffic model, the decision making problem is posed as an optimal control problem, which is solved by 1) behavior classification and trajectory prediction of the surrounding vehicles, and 2) a unique parallel planner architecture which addresses the efficiency goal and the safety goal separately. The simulation results demonstrate the effectiveness of the algorithm.
[C5] Who to blame? learning and control strategies with information asymmetry Changliu Liu, Wenlong Zhang and Masayoshi Tomizuka
American Control Conference, 2016
Citation Formats:
Abstract:
The rise of robot-robot interactions (RRI) is pushing for novel controller design techniques. Instead of using fixed control laws, robots should choose actions to minimize some cost functions specified by the designer. However, since the cost function of one robot may not be known to other robots (information asymmetry), special reasoning strategies are needed for multiple robots to learn to cooperate. Analysis shows that conventional learning and control strategies can lead to instability in a multi-agent system since the imperfection of other agents is not considered. In this paper, a new learning and control strategy that deals with interactions among imperfect agents is proposed. Analysis and simulation results show that the proposed strategy improves the performance of the system.
[C4] Algorithmic safety measures for intelligent industrial co-robots Changliu Liu and Masayoshi Tomizuka
IEEE International Conference on Robotics and Automation, 2016
Citation Formats:
Abstract:
In factories of the future, humans and robots are expected to be co-workers and co-inhabitants in the flexible production lines. It is important to ensure that humans and robots do not harm each other. This paper is concerned with functional issues to ensure safe and efficient interactions among human workers and the next generation intelligent industrial co-robots. The robot motion planning and control problem in a human involved environment is posed as a constrained optimal control problem. A modularized parallel controller structure is proposed to solve the problem online, which includes a baseline controller that ensures efficiency, and a safety controller that addresses real time safety by making a safe set invariant. Capsules are used to represent the complicated geometry of humans and robots. The design considerations of each module are discussed. Simulation studies which reproduce realistic scenarios are performed on a planar robot arm and a 6 DoF robot arm. The simulation results confirm the effectiveness of the method.
[C3] Safe exploration: Addressing various uncertainty levels in human robot interactions Changliu Liu and Masayoshi Tomizuka
American Control Conference, 2015
Citation Formats:
Abstract:
To address the safety issues in human robot interactions (HRI), a (SSA) was developed previously. However, during HRI, the uncertainty levels are changing in different phases of the interaction, which is not captured by SSA. A safe exploration algorithm (SEA) is proposed in this paper to address the uncertainty levels in the robot control. To estimate the uncertainty levels online, a learning method in the belief space is developed. A comparative study between SSA and SEA is conducted. The simulation results confirm that SEA can capture the uncertainty reduction behavior which is observed in human-human interactions.
[C2] Control in a safe set: Addressing safety in human-robot interactions Changliu Liu and Masayoshi Tomizuka
Dynamic Systems and Control Conference, 2014 Best Student Paper Finalist
Citation Formats:
Abstract:
Human-robot interactions (HRI) happen in a wide range of situations. Safety is one of the biggest concerns in HRI. This paper proposes a safe set method for designing the robot controller and offers theoretical guarantees of safety. The interactions are modeled in a multi-agent system framework. To deal with humans in the loop, we design a parameter adaptation algorithm (PAA) to learn the closed loop behavior of humans online. Then a safe set (a subset of the state space) is constructed and the optimal control law is mapped to the set of control which can make the safe set invariant. This algorithm is applied with different safety constraints to both mobile robots and robot arms. The simulation results confirm the effectiveness of the algorithm.
[C1] Modeling and controller design of cooperative robots in workspace sharing human-robot assembly teams Changliu Liu and Masayoshi Tomizuka
IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014
Citation Formats:
Abstract:
Human workers and robots are two major workforces in modern factories. For safety reasons, they are separated, which limits the productive potentials of both parties. It is promising if we can combine human’s flexibility and robot’s productivity in manufacturing. This paper investigates the modeling and controller design method of workspace sharing human-robot assembly teams and adopts a two-layer interaction model between the human and the robot. In theoretical analysis, enforcing invariance in a safe set guarantees safety. In imple- mentation, an integrated method concerning online learning of closed loop human behavior and receding horizon control in the safe set is proposed. Simulation results in a 2D setup confirm the safety and efficiency of the algorithm.
