Publications

Control barrier functions (CBFs) are important in safety-critical systems and robot control applications. Neural networks have been used to parameterize and synthesize CBFs with bounded control input for complex systems. However, it is still challenging to verify pre-trained neural networks CBFs (neural CBFs) in an efficient symbolic manner. To this end, we propose a new efficient verification framework for ReLU-based neural CBFs through symbolic derivative bound propagation by combining the linearly bounded nonlinear dynamic system and the gradient bounds of neural CBFs. Specifically, with Heaviside step function form for derivatives of activation functions, we show that the symbolic bounds can be propagated through the inner product of neural CBF Jacobian and nonlinear system dynamics. Through extensive experiments on different robot dynamics, our results outperform the interval arithmetic-based baselines in verified rate and verification time along the CBF boundary, validating the effectiveness and efficiency of the proposed method with different model complexity. The code can be found at https://github.com/intelligent-control-lab/verify-neural-CBF.

The requirements for real-world manipulation tasks are diverse and often conflicting; some tasks require precise motion while others require force compliance; some tasks require avoidance of certain regions while others require convergence to certain states. Satisfying these varied requirements with a fixed state-action representation and control strategy is challenging, impeding the development of a universal robotic foundation model. In this work, we propose Meta-Control, the first LLM-enabled automatic control synthesis approach that creates customized state representations and control strategies tailored to specific tasks. Our core insight is that a meta-control system can be built to automate the thought process that human experts use to design control systems. Specifically, human experts heavily use a model-based, hierarchical (from abstract to concrete) thought model, then compose various dynamic models and controllers together to form a control system. Meta-Control mimics the thought model and harnesses LLM’s extensive control knowledge with Socrates’ "art of midwifery" to automate the thought process. Meta-Control stands out for its fully model-based nature, allowing rigorous analysis, generalizability, robustness, efficient parameter tuning, and reliable real-time execution.

Deep Neural Networks (DNN) are crucial in approximating nonlinear functions across diverse applications, ranging from image classification to control. Verifying specific input-output properties can be a highly challenging task due to the lack of a single, self-contained framework that allows a complete range of verification types. To this end, we present \textttModelVerification.jl (MV), the first comprehensive, cutting-edge toolbox that contains a suite of state-of-the-art methods for verifying different types of DNNs and safety specifications. This versatile toolbox is designed to empower developers and machine learning practitioners with robust tools for verifying and ensuring the trustworthiness of their DNN models.

Hamilton-Jacobi (HJ) reachability analysis provides a formal method for guaranteeing safety in constrained control problems. It synthesizes a value function to represent a long-term safe set called feasible region. Early synthesis methods based on state space discretization cannot scale to high-dimensional problems, while recent methods that use neural networks to approximate value functions result in unverifiable feasible regions. To achieve both scalability and verifiability, we propose a framework for synthesizing verified neural value functions for HJ reachability analysis. Our framework consists of three stages: pre-training, adversarial training, and verification-guided training. We design three techniques to address three challenges to improve scalability respectively: boundary-guided backtracking (BGB) to improve counterexample search efficiency, entering state regularization (ESR) to enlarge feasible region, and activation pattern alignment (APA) to accelerate neural network verification. We also provide a neural safety certificate synthesis and verification benchmark called Cersyve-9, which includes nine commonly used safe control tasks and supplements existing neural network verification benchmarks. Our framework successfully synthesizes verified neural value functions on all tasks, and our proposed three techniques exhibit superior scalability and efficiency compared with existing methods.

This work addresses the certification of the local robustness of vision-based two-stage 6D object pose estimation. The two-stage method for object pose estimation achieves superior accuracy by first employing deep neural network-driven keypoint regression and then applying a Perspective-n-Point (PnP) technique. Despite advancements, the certification of these methods’ robustness remains scarce. This research aims to fill this gap with a focus on their local robustness on the system level–the capacity to maintain robust estimations amidst semantic input perturbations. The core idea is to transform the certification of local robustness into neural network verification for classification tasks. The challenge is to develop model, input, and output specifications that align with off-the-shelf verification tools. To facilitate verification, we modify the keypoint detection model by substituting nonlinear operations with those more amenable to the verification processes. Instead of injecting random noise into images, as is common, we employ a convex hull representation of images as input specifications to more accurately depict semantic perturbations. Furthermore, by conducting a sensitivity analysis, we propagate the robustness criteria from pose to keypoint accuracy, and then formulating an optimal error threshold allocation problem that allows for the setting of a maximally permissible keypoint deviation thresholds. Viewing each pixel as an individual class, these thresholds result in linear, classification-akin output specifications. Under certain conditions, we demonstrate that the main components of our certification framework are both sound and complete, and validate its effects through extensive evaluations on realistic perturbations. To our knowledge, this is the first study to certify the robustness of large-scale, keypoint-based pose estimation given images in real-world scenarios.

Safety is critical in autonomous robotic systems. A safe control law should ensure forward invariance of a safe set (a subset in the state space). It has been extensively studied regarding how to derive a safe control law with a control-affine analytical dynamic model. However, how to formally derive a safe control law with Neural Network Dynamic Models (NNDM) remains unclear due to the lack of computationally tractable methods to deal with these black-box functions. In fact, even finding the control that minimizes an objective for NNDM without any safety constraint is still challenging. In this work, we propose MIND-SIS (Mixed Integer for Neural network Dynamic model with Safety Index Synthesis), the first method to synthesize safe control for NNDM. The method includes two parts: 1) SIS: an algorithm for the offline synthesis of the safety index (also called as a barrier function), which uses evolutionary methods and 2) MIND: an algorithm for online computation of the optimal and safe control signal, which solves a constrained optimization using a computationally efficient encoding of neural networks. It has been theoretically proved that MIND-SIS guarantees forward invariance and finite convergence to a subset of the user-defined safe set. And it has been numerically validated that MIND-SIS achieves safe and optimal control of NNDM. The optimality gap is less than 10−8, and the safety constraint violation is 0.

Safety is the major consideration in controlling complex dynamical systems using reinforcement learning (RL), where the safety certificates can provide provable safety guarantees. A valid safety certificate is an energy function indicating that safe states are with low energy, and there exists a corresponding safe control policy that allows the energy function to always dissipate. The safety certificates and the safe control policies are closely related to each other and both challenging to synthesize. Therefore, existing learning-based studies treat either of them as prior knowledge to learn the other, limiting their applicability to general systems with unknown dynamics. This paper proposes a novel approach that simultaneously synthesizes the energy-function-based safety certificates and learns the safe control policies with constrained reinforcement learning (CRL). We do not rely on prior knowledge about either a prior control law or a perfect safety certificate. In particular, we formulate a loss function to optimize the safety certificate parameters by minimizing the occurrence of energy increases. By adding this optimization procedure as an outer loop to the Lagrangian-based CRL, we jointly update the policy and safety certificate parameters, and prove that they will converge to their respective local optima, the optimal safe policies and valid safety certificates. Finally, we evaluate our algorithms on multiple safety-critical benchmark environments. The results show that the proposed algorithm learns solidly safe policies with no constraint violation. The validity, or feasibility of synthesized safety certificates is also verified numerically.

The need to increase the flexibility of production lines is calling for robots to collaborate with human workers. However, existing interactive industrial robots only guarantee intrinsic safety (reduce collision impact), but not interactive safety (collision avoidance), which greatly limited their flexibility. The issue arises from two limitations in existing control software for industrial robots: 1) lack of support for real-time trajectory modification; 2) lack of intelligent safe control algorithms with guaranteed collision avoidance under robot dynamics constraints. To address the first issue, a jerk-bounded position controller (JPC) was developed previously. This paper addresses the second limitation, on top of the JPC. Specifically, we introduce a jerk-based safe set algorithm (JSSA) to ensure collision avoidance while considering the robot dynamics constraints. The JSSA greatly extends the scope of the original safe set algorithm, which has only been applied for second-order systems with unbounded accelerations. The JSSA is implemented on the FANUC LR Mate 200id/7L robot and validated with HRI tasks. Experiments show that the JSSA can consistently keep the robot at a safe distance from the human while executing the designated task.

Efficient and robust task planning for a human-robot collaboration (HRC) system remains challenging. The human-aware task planner needs to assign jobs to both robots and human workers so that they can work collaboratively to achieve better time efficiency. However, the complexity of the tasks and the stochastic nature of the human collaborators bring challenges to such task planning. To reduce the complexity of the planning problem, we utilize the hierarchical task model, which explicitly captures the sequential and parallel relationships of the task. We model human movements with the sigma-lognormal functions to account for human-induced uncertainties. A human action model adaptation scheme is applied during run-time, and it provides a measure for modeling the human-induced uncertainties. We propose a sampling-based method to estimate human job completion time uncertainties. Next, we propose a robust task planner, which formulates the planning problem as a robust optimization problem by considering the task structure and the uncertainties. We conduct simulations of a robot arm collaborating with a human worker in an electronics assembly setting. The results show that our proposed planner can reduce task completion time when human-induced uncertainties occur compared to the baseline planner.

Industrial robots are widely used in many applications with structured and deterministic environments. However, the contemporary need requires industrial robots to intelligently operate in dynamic environments. It is challenging to design a safe and efficient robotic system with industrial robots in a dynamic environment for several reasons. First, most industrial robots require the input to have specific formats, which takes additional efforts to convert from task-level user commands. Second, existing robot drivers do not support overwriting ongoing tasks in real-time, which hinders the robot from responding to the dynamic environment. Third, most industrial robots only expose motion-level control, making it challenging to enforce dynamic constraints during trajectory tracking. To resolve the above challenges, this paper presents a jerk-bounded position control driver (JPC) for industrial robots. JPC provides a unified interface for tracking complex trajectories and is able to enforce dynamic constraints using motion-level control, without accessing servo-level control. Most importantly, JPC enables real-time trajectory modification. Users can overwrite the ongoing task with a new one without violating dynamic constraints. The proposed JPC is implemented and tested on the FANUC LR Mate 200id/7L robot with both artificially generated data and an interactive robot handover task. Experiments show that the proposed JPC can track complex trajectories accurately within dynamic limits and seamlessly switch to new trajectory references before the ongoing task ends.

Delicate industrial insertion tasks (e.g., PC board assembly) remain challenging for industrial robots. The challenges include low error tolerance, delicacy of the components, and large task variations with respect to the components to be inserted. To deliver a feasible robotic solution for these insertion tasks, we also need to account for hardware limits of existing robotic systems and minimize the integration effort. This paper proposes a composable framework for efficient integration of a safe insertion policy on existing robotic platforms to accomplish these insertion tasks. The policy has an interpretable modularized design and can be learned efficiently on hardware and transferred to new tasks easily. In particular, the policy includes a safe insertion agent as a baseline policy for insertion, an optimal configurable Cartesian tracker as an interface to robot hardware, a probabilistic inference module to handle component variety and insertion errors, and a safe learning module to optimize the parameters in the aforementioned modules to achieve the best performance on designated hardware. The experiment results on a UR10 robot show that the proposed framework achieves safety (for the delicacy of components), accuracy (for low tolerance), robustness (against perception error and component defection), adaptability and transferability (for task variations), as well as task efficiency during execution plus data and time efficiency during learning.

Many collaborative human-robot tasks require the robot to stay safe and work efficiently around humans. Since the robot can only stay safe with respect to its own model of the human, we want the robot to learn a good model of the human in order to act both safely and efficiently. This paper studies methods that enable a robot to safely explore the space of a human-robot system to improve the robot’s model of the human, which will consequently allow the robot to access a larger state space and better work with the human. In particular, we introduce active exploration under the framework of energy-function based safe control, investigate the effect of different active exploration strategies, and finally analyze the effect of safe active exploration on both analytical and neural network human models.

In this paper, we consider the problem of protecting a high-value area from being breached by sheep agents by crafting motions for dog robots. We use control barrier functions to pose constraints on the dogs’ velocities that induce repulsion in the sheep relative to the high-value area. This paper extends the results developed in our prior work on the same topic in three ways. Firstly, we implement and validate our previously developed centralized herding algorithm on many robots. We show herding of up to five sheep agents using three dog robots. Secondly, as an extension to the centralized approach, we develop two distributed herding algorithms, one favoring feasibility while the other favoring optimality. In the first algorithm, we allocate a unique sheep to a unique dog, making that dog responsible for herding its allocated sheep away from the protected zone. We provide feasibility proof for this approach, along with numerical simulations. In the second algorithm, we develop an iterative distributed reformulation of the centralized algorithm, which inherits the optimality (i.e. budget efficiency) from the centralized approach. Lastly, we conduct real-world experiments of these distributed algorithms and demonstrate herding of up to five sheep agents using five dog robots.

Collision avoidance for multirobot systems is a well-studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers that guarantee collision avoidance and goal stabilization for multiple robots. However, it has been noted that reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics that causes the robots to stall before reaching their goals. In this paper, we analyze the closed-loop dynamics of robots using CBFs, to characterize controller parameters, initial conditions, and goal locations that invariably lead the system to deadlock. Using tools from duality theory, we derive geometric properties of robot configurations of an N robot system once it is in deadlock and we justify them using the mechanics interpretation of KKT conditions. Our key deductions are that 1) system deadlock is characterized by a force-equilibrium on robots and 2) deadlock occurs to ensure safety when safety is on the brink of being violated. These deductions allow us to interpret deadlock as a subset of the state space, and we show that this set is non-empty and located on the boundary of the safe set. By exploiting these properties, we analyze the number of admissible robot configurations in deadlock and develop a provably-correct decentralized algorithm for deadlock resolution to safely deliver the robots to their goals. This algorithm is validated in simulations as well as experimentally on Khepera-IV robots.

Decentralized conflict resolution for autonomous vehicles is needed in many places where a centralized method is not feasible, e.g., parking lots, rural roads, merge lanes, etc. However, existing methods generally do not fully utilize optimization in decentralized conflict resolution. We propose a decentralized conflict resolution method for autonomous vehicles based on a novel extension to the Alternating Direc- tions Method of Multipliers (ADMM), called Online Adaptive ADMM (OA-ADMM), and on Model Predictive Control (MPC). OA-ADMM is tailored to online systems, where fast and adaptive real-time optimization is crucial, and allows the use of safety information about the physical system to improve safety in real-time control. We prove convergence in the static case and give requirements for online convergence. Combining OA-ADMM and MPC allows for robust decentralized motion planning and control that seamlessly integrates decentralized conflict resolution. The effectiveness of our proposed method is shown through simulations in CARLA, an open-source vehicle simulator, resulting in a reduction of 47.93% in mean added delay compared with the next best method.

We consider the problem of estimating bounds on parameters representing tasks being performed by individual robots in a multirobot system. In our previous work, we derived necessary conditions based on persistency of excitation analysis for exact identification of these parameters. We concluded that depending on the robot’s task, the dynamics of individual robots may fail to satisfy these conditions, thereby preventing exact inference. As an extension to that work, this paper focuses on estimating bounds on task parameters when such conditions are not satisfied. Each robot in the team uses optimization-based controllers for mediating between task satisfaction and collision avoidance. We use KKT conditions of this optimization and SVD of active collision avoidance constraints to derive explicit relations between Lagrange multipliers, robot dynamics and task parameters. Using these relations, we are able to derive bounds on each robot’s task parameters. Through numerical simulations we show how our proposed region based identification approach generates feasible regions for parameters when a conventional estimator such as a UKF fails. Additionally, empirical evidence shows that this approach generates contracting sets which converge to the true parameters much faster than the rate at which a UKF based estimate converges. Videos of these results are available at https://bit.ly/2JDMgeJ.

The implementation of optimization-based motion coordination approaches in real world multi-agent systems remains challenging due to their high computational complexity and potential deadlocks. This paper presents a distributed model predictive control (MPC) approach based on convex feasible set (CFS) algorithm for multi-vehicle motion coordination in autonomous driving. By using CFS to convexify the collision avoidance constraints, collision-free trajectories can be computed in real time. We analyze the potential deadlocks and show that a deadlock can be resolved by changing vehicles’ desired speeds. The MPC structure ensures that our algorithm is robust to low-level tracking errors. The proposed distributed method has been tested in multiple challenging multi-vehicle environments, including unstructured road, intersection, crossing, platoon formation, merging, and overtaking scenarios. The numerical results and comparison with other approaches (including a centralized MPC and reciprocal velocity obstacles) show that the proposed method is computationally efficient and robust, and avoids deadlocks.

This paper presents algorithms for learning parameters of optimization-based controllers used in multiagent systems based on their position-velocity measurements. The motivation to learn these parameters stems from the need to infer an agent’s intent (human or robot) to facilitate accurate predictions of motion as well as efficient interactions in a multiagent system. In this work, we demonstrate how to perform inference using algorithms based on the theory of inverse optimization (IO). We propose QP-based reformulations of IO algorithms for faster processing of batch-data to facilitate quicker inference. In our prior work, we used persistency of excitation analysis for deriving conditions under which conventional estimators such as a Kalman filter can successfully perform such inference. In this work, we demonstrate that whenever these conditions are violated, inference of parameters will fail, be it using IO-based algorithms or a UKF. We provide numerical simulations to infer desired goal locations and controller gains of each robot in a multirobot system and compare performance of IO-based algorithms with a UKF and an adaptive observer. In addition to these, we also conduct experiments with Khepera-4 robots and demonstrate the power of IO-based algorithms in inferring goals in the presence of perception noise.

Maintaining safety under adaptation has long been considered to be an important capability for autonomous systems. As these systems estimate and change the ego-model of the system dynamics, questions regarding how to develop safety guarantees for such systems continue to be of interest. We propose a novel robust safe control methodology that uses set-based safety constraints to make a robotic system with dynamical uncertainties safely adapt and operate in its environment. The method consists of designing a scalar energy function (safety index) for an adaptive system with parametric uncertainty and an optimization-based approach for control synthesis. Simulation studies on a two-link manipulator are conducted and the results demonstrate the effectiveness of our proposed method in terms of generating provably safe control for adaptive systems with parametric uncertainty.

Maintaining safety under adaptation has long been considered to be an important capability for autonomous systems. As these systems estimate and change the ego-model of the system dynamics, questions regarding how to develop safety guarantees for such systems continue to be of interest. We propose a novel robust safe control methodology that uses set-based safety constraints to make a robotic system with dynamical uncertainties safely adapt and operate in its environment. The method consists of designing a scalar energy function (safety index) for an adaptive system with parametric uncertainty and an optimization-based approach for control synthesis. Simulation studies on a two-link manipulator are conducted and the results demonstrate the effectiveness of our proposed method in terms of generating provably safe control for adaptive systems with parametric uncertainty.

This paper considers the problem of parameter identification for a multirobot system. We wish to understand when is it feasible for an adversarial observer to reverse-engineer the parameters of tasks being performed by a team of robots by simply observing their positions. We address this question by using the concept of persistency of excitation from system identification. Each robot in the team uses optimization-based controllers for mediating between task satisfaction and collision avoidance. These controllers exhibit an implicit dependence on the task’s parameters which poses a hurdle for deriving necessary conditions for parameter identification, since such conditions usually require an explicit relation. We address this bottleneck by using duality theory and SVD of active collision avoidance constraints and derive an explicit relation between each robot’s task parameters and its control inputs. This allows us to derive the main necessary conditions for successful identification which agree with our intuition. We demonstrate the importance of these conditions through numerical simulations by using (a) an adaptive observer and (b) an unscented Kalman filter for goal estimation in various geometric settings. These simulations show that under circumstances where parameter inference is supposed to be infeasible per our conditions, both these estimators fail and likewise when it is feasible, both converge to the true parameters.

High fidelity behavior prediction of intelligent agents is critical in many applications,which is challenging due to the stochasticity, heterogeneity and time-varying natureof agent behaviors. Prediction models that work for one individual may not be appli-cable to another. Besides, the prediction model trained on the training set may notgeneralize to the testing set. These challenges motivate the adoption of online adap-tation algorithms to update prediction models in real-time to improve the predictionperformance. This paper considers online adaptable multi-task prediction for bothintention and trajectory. The goal of online adaptation is to improve the performanceof both intention and trajectory predictions with only the feedback of the observedtrajectory. We first introduce a generic tau-step adaptation algorithm of the multi-taskprediction model that updates the model parameters with the trajectory predictionerror in recent tau steps. Inspired by Extended Kalman Filter (EKF), a base adaptationalgorithm Modified EKF with forgetting factor (MEKFtau) is introduced. In order toimprove the performance of MEKFtau, generalized exponential moving average filter-ing techniques are adopted. Then this paper introduces a dynamic multi-epoch updatestrategy to effectively utilize samples received in real time. With all these exten-sions, we propose a robust online adaptation algorithm: MEKF with Moving Averageand dynamic Multi-Epoch strategy (MEKFMA−ME). We empirically study the bestset of parameters to adapt in the multi-task prediction model and demonstrate theeffectiveness of the proposed adaptation algorithms to reduce the prediction error.

Human motion prediction, especially arm prediction, is critical to facilitate safe and efficient human-robot collaboration (HRC). This letter proposes a novel human motion prediction framework that combines a recurrent neural network (RNN) and inverse kinematics (IK) to predict human arm motion. A modified Kalman filter (MKF) is applied to adapt the model online. The proposed framework is tested on collected human motion data with up to 2 s prediction horizon. The experiments demonstrate that the proposed method improves the prediction accuracy by approximately 14% comparing to the state-of-art on seen situations. It stably adapts to unseen situations by keeping the maximum prediction error under 4 cm, which is 70% lower than other methods. Moreover, it is robust when the arm is partially occluded. The wrist prediction remains the same, while the elbow prediction has 20% less variation.

Deep neural networks are widely used for nonlinear function approximation with applications spanning from computer vision to control. Although these networks involve the composition of simple arithmetic operations, it can be very challenging to verify whether a particular network satisfies certain input-output properties. This article surveys methods that have emerged recently for soundly verifying such properties. These methods borrow insights from reachability analysis, optimization, and search. We discuss fundamental differences and connections between existing algorithms. In addition, we provide pedagogical implementations of existing methods and compare them on a set of benchmark problems.

We present an algorithm for learning constraint and objective function parameters of optimization-based controllers used in multirobot systems. Our proposed approach uses position-velocity measurements of each robot in the team to perform this inference. The motivation to learn these parameters stems from the need to infer an agent’s intent for accurate predictions of motion in a multiagent system. We develop an extension of our prior work in which we performed task learning assuming constraint parameters were known. In this work, we perform simultaneous learning of constraint and cost function parameters by posing it as a constrained nonconvex optimization problem. The cost function parameters that we learn encode information of the task being performed by each robot in the team whereas the constraint parameters encode information about individual safety margin distances and size of the safe control set for each robot. Our simulation results show the accurate reconstruction of both the constraint and cost function parameters and we analyze some failure cases.

Although neural networks are widely used, it remains challenging to formally verify the safety and robustness of neural networks in real-world applications. Existing methods are designed to verify the network before deployment, which are limited to relatively simple specifications and fixed networks. These methods are not ready to be applied to real-world problems with complex and/or dynamically changing specifications and networks. To effectively handle such problems, verification needs to be performed online when these changes take place. However, it is still challenging to run existing verification algorithms online. Our key insight is that we can leverage the temporal dependencies of these changes to accelerate the verification process. This paper establishes a novel framework for scalable online verification to solve real-world verification problems with dynamically changing specifications and/or networks. We propose three types of acceleration algorithms: Branch Management to reduce repetitive computation, Perturbation Tolerance to tolerate changes, and Incremental Computation to reuse previous results. Experiment results show that our algorithms achieve up to 100× acceleration, and thus show a promising way to extend neural network verification to real-world applications.

Neural networks (NNs) are widely used for classification tasks for their remarkable performance. However, the robustness and accuracy of NNs heavily depend on the training data. In many applications, massive training data is usually not available. To address the challenge, this paper proposes an iterative adversarial data augmentation (IADA) framework to learn neural network models from an insufficient amount of training data. The method uses formal verification to identify the most "confusing" input samples, and leverages human guidance to safely and iteratively augment the training data with these samples. The proposed framework is applied to an artificial 2D dataset, the MNIST dataset, and a human motion dataset. By applying IADA to fully-connected NN classifiers, we show that our training method can improve the robustness and accuracy of the learned model. By comparing to regular supervised training, on the MNIST dataset, the average perturbation bound improved 107.4%. The classification accuracy improved 1.77%, 3.76%, 10.85% on the 2D dataset, the MNIST dataset, and the human motion dataset respectively.

When autonomous vehicles still struggle to solve challenging situations during on-road driving, humans have long mastered the essence of driving with efficient transferable and adaptable driving capability. By mimicking humans’ cognition model and semantic understanding during driving, we present HATN, a hierarchical framework to generate high-quality driving behaviors in multi-agent dense-traffic environments. Our method hierarchically consists of a high-level intention identification and low-level action generation policy. With the semantic sub-task definition and generic state representation, the hierarchical framework is transferable across different driving scenarios. Besides, our model is also able to capture variations of driving behaviors among individuals and scenarios by an online adaptation module. We demonstrate our algorithms in the task of trajectory prediction for real traffic data at intersections and roundabouts, where we conducted extensive studies of the proposed method and demonstrated how our method outperformed other methods in terms of prediction accuracy and transferability.

High fidelity behavior prediction of human drivers is crucial for efficient and safe deployment of autonomous vehicles, which is challenging due to the stochasticity, heterogeneity, and time-varying nature of human behaviors. On one hand, the trained prediction model can only capture the motion pattern in an average sense, while the nuances among individuals can hardly be reflected. On the other hand, the prediction model trained on the training set may not generalize to the testing set which may be in a different scenario or data distribution, resulting in low transferability and generalizability. In this paper, we applied a τ-step modified Extended Kalman Filter parameter adaptation algorithm (MEKFλ) to the driving behavior prediction task, which has not been studied before in literature. With the feedback of the observed trajectory, the algorithm is applied to neural-network-based models to improve the performance of driving behavior predictions across different human subjects and scenarios. A new set of metrics is proposed for systematic evaluation of online adaptation performance in reducing the prediction error for different individuals and scenarios. Empirical studies on the best layer in the model and steps of observation to adapt are also provided.

This study proposes a safe and sample-efficient reinforcement learning (RL) framework to address two major challenges in developing applicable RL algorithms: satisfying safety constraints and efficiently learning with limited samples. To guarantee safety in real-world complex environments, we use the safe set algorithm (SSA) to monitor and modify the nominal controls, and evaluate SSA+RL in a clustered dynamic environment which is challenging to be solved by existing RL algorithms. However, the SSA+RL framework is usually not sample-efficient especially in reward-sparse environments, which has not been addressed in previous safe RL works. To improve the learning efficiency, we propose three techniques: (1) avoiding behaving overly conservative by adapting the SSA; (2) encouraging safe exploration using random network distillation with safety constraints; (3) improving policy convergence by treating SSA as expert demonstrations and directly learn from that. The experimental results show that our framework can achieve better safety performance compare to other safe RL methods during training and solve the task with substantially fewer episodes.

Microscopic epidemic models are powerful tools for government policy makers to predict and simulate epidemic outbreaks, which can capture the impact of individual behaviors on the macroscopic phenomenon. However, existing models only consider simple rule-based individual behaviors, limiting their applicability. This paper proposes a deep-reinforcement-learning-powered microscopic model named Microscopic Pandemic Simulator (MPS). By replacing rule-based agents with rational agents whose behaviors are driven to maximize rewards, the MPS provides a better approximation of real world dynamics. To efficiently simulate with massive amounts of agents in MPS, we propose Scalable Million-Agent DQN (SMADQN). The MPS allows us to efficiently evaluate the impact of different government strategies. This paper first calibrates the MPS against real-world data in Allegheny, US, then demonstratively evaluates two government strategies: information disclosure and quarantine. The results validate the effectiveness of the proposed method. As a broad impact, this paper provides novel insights for the application of DRL in large scale agent-based networks such as economic and social networks.

This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021), held as a part of the 4th Workshop on Formal Methods for ML-Enabled Autonomous Systems that was collocated with the 33rd International Conference on Computer-Aided Verification (CAV). Twelve teams participated in this competition. The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification, in terms of scalability and speed. Along this line, we used standard formats (ONNX for neural networks and VNNLIB for specifications), standard hardware (all tools are run by the organizers on AWS), and tool parameters provided by the tool authors. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.

Human motion prediction is non-trivial in modern industrial settings. Accurate prediction of human motion can not only improve efficiency in human robot collaboration, but also enhance human safety in close proximity to robots. Among existing prediction models, the parameterization and identification methods of those models vary. It remains unclear what is the necessary parameterization of a prediction model, whether online adaptation of the model is necessary, and whether prediction can help improve safety and efficiency during human robot collaboration. These problems result from the difficulty to quantitatively evaluate various prediction models in a closed-loop fashion in real human-robot interaction settings. This paper develops a method to evaluate the closed-loop performance of different prediction models. In particular, we compare models with different parameterizations and models with or without online parameter adaptation. Extensive experiments were conducted on a human robot collaboration platform. The experimental results demonstrated that human motion prediction significantly enhanced the collaboration efficiency and human safety. Adaptable prediction models that were parameterized by neural networks achieved the best performance.

Collision avoidance for multirobot systems has been studied thoroughly. Recently, control barrier functions (CBFs) have been proposed to mediate between collision avoidance and goal achievement for multiple robots. However, it has been noted that reactive controllers (such as CBFs) are prone to deadlock, an equilibrium that causes the robots to stall before reaching their goals. In this paper, we formally analyze two and three robot systems and discover circumstances under which CBFs cause deadlocks using duality theory. For the two robot system, we consider mutually heterogeneous robots (such as one more vigorous or closer to its goal than the other) and prove that this heterogeneity does not help in preventing deadlock. We then consider three robots, and conclude from these two scenarios that the geometric symmetry resulting from robots’ initial positions and goals constrains CBFs to generate velocities that render deadlock stable. Thus, conferring skewness to the system can help evade deadlock.

Human-robot collaboration (HRC) is becoming increasingly important as the paradigm of manufacturing is shifting from mass production to mass customization. The introduction of HRC can significantly improve the flexibility and intelligence of automation. To efficiently finish tasks in HRC systems, the robots need to not only predict the future movements of human, but also more high-level plans, i.e., the sequence of actions to finish the tasks. However, due to the stochastic and time-varying nature of human collaborators, it is quite challenging for the robot to efficiently and accurately identify such task plans and respond in a safe manner. To address this challenge, we propose an integrated human-robot collaboration framework. Both plan recognition and trajectory prediction modules are included for the generation of safe and efficient robotic motions. Such a framework enables the robots to perceive, predict and adapt their actions to the human’s work plan and intelligently avoid collisions with the human. Moreover, by explicitly leveraging the hierarchical relationship between plans and trajectories, more robust plan recognition performance can be achieved. Physical experiments were conducted on an industrial robot to verify the proposed framework. The results show that the proposed framework could accurately recognize the human workers’ plans and thus significantly improve the time efficiency of the HRC team even in the presence of motion classification noises.

This paper introduces a microscopic approach to model epidemics, which can explicitly consider the consequences of individual’s decisions on the spread of the disease. We first formulate a microscopic multi-agent epidemic model where every agent can choose its activity level that affects the spread of the disease. Then by minimizing agents’ cost functions, we solve for the optimal decisions for individual agents in the framework of game theory and multi-agent reinforcement learning. Given the optimal decisions of all agents, we can make predictions about the spread of the disease. We show that there are negative externalities in the sense that infected agents do not have enough incentives to protect others, which then necessitates external interventions to regulate agents’ behaviors. In the discussion section, future directions are pointed out to make the model more realistic.

High fidelity behavior prediction of intelligent agents is critical in many applications. However, the prediction model trained on the training set may not generalize to the testing set due to domain shift and time variance. The challenge motivates the adoption of online adaptation algorithms to update prediction models in real-time to improve the prediction performance. Inspired by Extended Kalman Filter (EKF), this paper introduces a series of online adaptation methods, which are applicable to neural network-based models. A base adaptation algorithm Modified EKF with forgetting factor (MEKF_lambda) is introduced first, followed by exponential moving average filtering techniques. Then this paper introduces a dynamic multi-epoch update strategy to effectively utilize samples received in real time. With all these extensions, we propose a robust online adaptation algorithm: MEKF with Exponential Moving Average and Dynamic Multi-Epoch strategy (MEKF_EMA-DME). The proposed algorithm outperforms existing methods as demonstrated in experiments.

Lane merging is one of the major sources causing traffic congestion and delay. With the help of vehicle-to-vehicle or vehicle-to-infrastructure communication and autonomous driving technology, there are opportunities to alleviate congestion and delay resulting from lane merging. In this paper, we first summarize modern features and requirements for lane merging, along with the advance of vehicular technology. We then formulate and propose a dynamic programming algorithm to find the optimal solution for a two-lane merging scenario. It schedules the passing order for vehicles while minimizing the time needed for all vehicles to go through the merging point (equivalent to the time that the last vehicle goes through the merging point). We further extend the problem to a consecutive lane-merging scenario. We show the difficulty to apply the original dynamic programming to the consecutive lane-merging scenario and propose an improved version to solve it. Experimental results show that our dynamic programming algorithm can efficiently minimize the time needed for all vehicles to go through the merging point and reduce the average delay of all vehicles, compared with some greedy methods.

Collision avoidance for multirobot systems is a well studied problem. Recently, control barrier functions (CBFs) have been proposed for synthesizing controllers guarantee collision avoidance and goal stabilization for multiple robots. However, it has been noted reactive control synthesis methods (such as CBFs) are prone to deadlock, an equilibrium of system dynamics causes robots to come to a standstill before reaching their goals. In this paper, we formally derive characteristics of deadlock in a multirobot system uses CBFs. We propose a novel approach to analyze deadlocks resulting from optimization based controllers (CBFs) by bor- rowing tools from duality theory and graph enumeration. Our key insight is system deadlock is characterized by a force-equilibrium on robots and we show how complexity of deadlock analysis increases approximately exponentially with the number of robots. This analysis allows us to interpret deadlock as a subset of the state space, and we prove this set is non-empty, bounded and located on the boundary of the safety set. Finally, we use these properties to develop a provably correct decentralized algorithm for deadlock resolution which ensures robots converge to their goals while avoiding collisions. We show simulation results of the resolution algorithm for two and three robots and experimentally validate this algorithm on Khepera-IV robots.

Trajectory planning is an essential module for autonomous driving. To deal with multi-vehicle interactions, existing methods follow the prediction-then-plan approaches which first predict the trajectories of others then plan the trajectory for the ego vehicle given the predictions. However, since the true trajectories of others may deviate from the predictions, frequent re-planning for the ego vehicle is needed, which may cause many issues such as instability or deadlock. These issues can be overcome if all vehicles can form a consensus by solving the same multi-vehicle trajectory planning problem. Then the major challenge is how to efficiently solve the multi-vehicle trajectory planning problem in real time under the curse of dimensionality. We introduce a novel planner for multi-vehicle trajectory planning based on the convex feasible set (CFS) algorithm. The planning problem is formulated as a non-convex optimization. A novel convexification method to obtain the maximal convex feasible set is proposed, which transforms the problem into a quadratic programming. Simulations in multiple typical on-road driving situations are conducted to demonstrate the effectiveness of the proposed planning algorithm in terms of completeness and optimality.

Applying intelligent robot arms in dynamic uncertain environments (i.e., flexible production lines) remains challenging, which requires efficient algorithms for real time trajectory generation. The motion planning problem for robot trajectory generation is highly nonlinear and nonconvex, which usually comes with collision avoidance constraints, robot kinematics and dynamics constraints, and task constraints (e.g., following a Cartesian trajectory defined on a surface and maintain the contact). The nonlinear and nonconvex planning problem is computationally expensive to solve, which limits the application of robot arms in the real world. In this paper, for redundant robot arm planning problems with complex constraints, we present a motion planning method using iterative convex optimization that can efficiently handle the constraints and generate optimal trajectories in real time. The proposed planner guarantees the satisfaction of the contact-rich task constraints and avoids collision in confined environments. Extensive experiments on trajectory generation for weld grinding are performed to demonstrate the effectiveness of the proposed method and its applicability in advanced robotic manufacturing.

Policy learning for delicate industrial insertion tasks (e.g., PC board assembly) is challenging. This paper considers two major problems: how to learn a diversified policy (instead of just one average policy) that can efficiently handle different workpieces with minimum amount of training data, and how to handle defects of workpieces during insertion. To address the problems, we propose tolerance-guided policy learning. To encourage transferability of the learned policy to different workpieces, we add a task embedding to the policy’s input space using the insertion tolerance. Then we train the policy using generative adversarial imitation learning with reward shaping (RS-GAIL) on a variety of representative situations. To encourage adaptability of the learned policy to handle defects, we build a probabilistic inference model that can output the best inserting pose based on failed insertions using the tolerance model. The best inserting pose is then used as a reference to the learned policy. This proposed method is validated on a sequence of IC socket insertion tasks in simulation. The results show that 1) RS-GAIL can efficiently learn optimal policies under sparse rewards; 2) the tolerance embedding can enhance the transferability of the learned policy; 3) the probabilistic inference makes the policy robust to defects on the workpieces.

Imitation learning is the problem of recovering an expert policy without access to a reward signal. Behavior cloning and GAIL are two widely used methods for performing imitation learning. Behavior cloning converges in a few iterations, but doesn’t achieve peak performance due to its inherent iid assumption about the state-action distribution. GAIL addresses the issue by accounting for the temporal dependencies when performing a state distribution matching between the agent and the expert. Although GAIL is sample efficient in the number of expert trajectories required, it is still not very sample efficient in terms of the environment interactions needed for convergence of the policy. Given the complementary benefits of both methods, we present a simple and elegant method to combine both methods to enable stable and sample efficient learning. Our algorithm is very simple to implement and integrates with different policy gradient algorithms. We demonstrate the effectiveness of the algorithm in low dimensional control tasks, gridworlds and in high dimensional image-based tasks.

Recent developments in multi-agent imitation learning have shown promising results for modeling the behavior of human drivers. However, it is challenging to capture emergent traffic behaviors that are observed in real-world datasets. Such behaviors arise due to the many local interactions between agents that are not commonly accounted for in imitation learning. This paper proposes Reward Augmented Imitation Learning (RAIL), which integrates reward augmentation into the multi-agent imitation learning framework and allows the designer to specify prior knowledge in a principled fashion. We prove that convergence guarantees for the imitation learning process are preserved under the application of reward augmentation. This method is validated in a driving scenario, where an entire traffic scene is controlled by driving policies learned using our proposed algorithm. Further, we demonstrate improved performance in comparison to traditional imitation learning algorithms both in terms of the local actions of a single agent and the behavior of emergent properties in complex, multi-agent settings.

Human motion prediction is an important component to facilitate human robot interaction. Robots need to accurately predict human’s future movement in order to efficiently collaborate with humans, as well as to safely plan its own motion trajectories. Many recent approaches predict human’s future movement using deep learning methods, such as recurrent neural networks. However, existing methods lack the ability to adapt to time-varying human behaviors. Moreover, many of them do not quantify uncertainties in the prediction. This paper proposes a new approach that uses an adaptable neural network for human motion prediction, in order to accommodate human’s time-varying behaviors and to provide uncertainty bounds of the predictions in real time. In particular, a neural network is trained offline to represent the human motion transition model. Recursive least square parameter adaptation algorithm (RLS-PAA) is adopted for online parameter adaptation of the neural network and for uncertainty estimation. Experiments on several human motion datasets verify that the proposed method outperforms the state-of-the-art approach with a significant improvement in terms of prediction accuracy and computation efficiency.

In highly interactive driving scenarios, accurate prediction of other road participants is critical for safe and efficient navigation of autonomous cars. Prediction is challenging due to the difficulty in modeling various driving behavior, or learning such a model. The model should be interactive and reflect individual differences. Imitation learning methods, such as parameter sharing generative adversarial imitation learning (PS-GAIL), are able to learn interactive models. However, the learned models average out individual differences. When used to predict trajectories of individual vehicles, these models are biased. This paper introduces an adaptable generative prediction framework (AGen), which performs online adaptation of the offline learned models to recover individual differences for better prediction. In particular, we combine the recursive least square parameter adaptation algorithm (RLS-PAA) with the offline learned model from PS-GAIL. RLS-PAA has analytical solutions and is able to adapt the model for every single vehicle efficiently online. The proposed method is able to reduce the root mean squared prediction error in a 2.5s time window by 60%, compared with PS-GAIL.

Neural network autonomous driving policies are widely explored. However, no matter using imitation learning or reinforcement learning, the network policies are generally hard to train, and the learned knowledge encoded in neural network policies are hard to transfer. We propose to modularize the complicated driving policies in terms of the driving attributes, and present the parallel attribute networks (PAN), which can learn to fullfill the requirements of the attributes in the driving tasks separately, and later assemble their knowledge together. Concretely, we first train a policy network that accomplish the base lane tracking attribute. The modules for the add-on attributes such as avoiding obstacles and obeying traffic rules are then trained to map the corresponding state to a satisfactory set of the vehicle action space. Finally the reference action given by the base policy is projected into the satisfactory sets so as to satisfy the requirements of all the attributes. Using the PAN, many complicated tasks that are hard to train from scratch can be easily trained; also unseen driving tasks can be solved in a zero-shot manner by assembling the pretrained attribute modules. We have validated the capability of our model on a class of autonomous driving problems with attributes of obstacle avoidance, traffic light and speed limit in simulation. Experimental results based on an obstacle avoidance task are also presented.

Safe autonomy is important in many application domains, especially for applications involving interactions with humans. Existing safe control algorithms are similar to each other in the sense that: they all provide control input to maintain a low value of an energy function that measures safety. In different methods, the energy function is called a potential function, a safety index, or a barrier function. The connections and relative advantages among these methods remain unclear. This paper introduces a unified framework to derive safe control laws using energy functions. We demonstrate how to integrate existing controllers based on potential field method, safe set algorithm, barrier function method, and sliding mode algorithm into this unified framework. In addition to theoretical comparison, this paper also introduces a benchmark which implements and compares existing methods on a variety of problems with different system dynamics and interaction modes. Based on the comparison results, a new method, called the sublevel safe set algorithm, is derived under the unified framework by optimizing the hyperparameters. The proposed algorithm achieves the best performance in terms of safety and efficiency on all benchmark problems.

Intersection management is one of the most representative applications of intelligent vehicles with connected and autonomous functions. The connectivity provides environmental information that a single vehicle cannot sense, and the autonomy supports precise vehicular control that a human driver cannot achieve. Intersection management solves the fundamental conflict resolution problem for vehicles—two vehicles should not appear at the same location at the same time, and, if they intend to do that, an order should be decided to optimize certain objectives such as the traffic throughput or smoothness. In this paper, we first propose a graph-based model for intersection management. The model is general and applicable to different granularities of intersections and other conflicting scenarios. We then derive formal verification approaches which can guarantee deadlock-freeness. Based on the graph-based model and the verification approaches, we develop a centralized cycle removal algorithm for the graph-based model to schedule vehicles to go through the intersection safely (without collisions) and efficiently without deadlocks. Experimental results demonstrate the expressiveness of the proposed model and the effectiveness and efficiency of the proposed algorithm.

Human-robot interactions (HRI) have been recognized to be a key element of future robots in many application domains such as manufacturing and transportation, which entail huge social and economic impacts. Future robots are envisioned to be acting like human, which are independent entities that make decisions for themselves; intelligent actuators that interact with the physical world; and involved observers that have rich senses and critical judgements. Most importantly, they are entitled social attributions to build relationships with humans. We call these robots co-robots. Technically, it is challenging to design the behavior of co-robots. Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. The book is to study methodologies to ensure that co-robots operate efficiently and safely in dynamic uncertain environments. This book sets up a unified analytical framework for various human-robot systems, which involves peer-peer interactions or hierarchical interactions. Various methods to design the robot behavior through control, planning, decision and learning are proposed. In particular, the following topics are discussed: safety during human-robot interactions, efficiency in real-time robot motion planning, imitation of human behaviors by robot, dexterity of robot to adapt to different environments and tasks, cooperation among robots and humans and conflict resolution. The proposed methods have been applied on various scenarios, such as human-robot collaborative assembly, robot skill learning from human demonstration, interaction between autonomous and human-driven vehicles, and etc.

With the development of robotics, there are growing needs for real time motion planning. However, due to obstacles in the environment, the planning problem is highly non-convex, which makes it difficult to achieve real time computation using existing non-convex optimization algorithms. This paper introduces the convex feasible set algorithm (CFS) which is a fast algorithm for non-convex optimization problems that have convex costs and non-convex constraints. The idea is to find a convex feasible set for the original problem and iteratively solve a sequence of subproblems using the convex constraints. The feasibility and the convergence of the proposed algorithm are proved in the paper. The application of this method on motion planning for mobile robots is discussed. The simulations demonstrate the effectiveness of the proposed algorithm.

Motion planning is one of the core modules for autonomous driving. Among the current motion planning techniques, optimization-based methods have unique advantages since they allow planning in continuous space and they can evaluate multiple objectives (such as hard constraints) in one formulation. However, it is hard to implement optimization-based methods in real-time in complicated environments due to 1) high computational complexity as the optimization problems are usually non-convex; and 2) difficulty to guarantee closed-loop performance because the low level trajectory tracking controller cannot perform perfect tracking. To solve the first challenge, convex feasible set algorithm (CFS) has been proposed for real time non-convex optimization. To solve the second challenge, a fast optimization-based autonomous driving motion planner (FOAD) is proposed in this paper which implements a soft constrained convex feasible set algorithm (SCCFS) as an enhanced version of CFS. The concept of closed-loop smoothness is defined and analyzed in this paper. Simulations and real vehicle experiments verify the efficiency and capability of the planner.

Autonomous vehicles are widely regarded as a promising technology to improve the safety of transportation systems. However, the efficiency of vehicles may be compromised to ensure safety when there are large uncertainties in perception and prediction of the behaviors of other road participants due to limitations in sensors. To remedy this problem, vehicle to vehicle (V2V) communication is applied to improve efficiency of autonomous vehicles during interactions with other vehicles. By requiring the vehicles to communicate their intentions with one another, the efficiency of the vehicles can be improved in terms of smaller variations in their speed profiles and smaller delay as demonstrated in the simulations.

With the emergence of autonomous vehicles, it is important to understand their impact on the transportation system. However, conventional traffic simulations are time-consuming. In this paper, we introduce an analytical traffic model for unmanaged intersections accounting for microscopic vehicle interactions. The macroscopic property, i.e., delay at the intersection, is modeled as an event-driven stochastic dynamic process, whose dynamics encode the microscopic vehicle behaviors. The distribution of macroscopic properties can be obtained through either direct analysis or event-driven simulation. They are more efficient than conventional (time-driven) traffic simulation, and capture more microscopic details compared to conventional macroscopic flow models. We illustrate the efficiency of this method by delay analyses under two different policies at a two-lane intersection. The proposed model allows for 1) efficient and effective comparison among different policies, 2) policy optimization, 3) traffic prediction, and 4) system optimization (e.g., infrastructure and protocol).

At an unmanaged intersection, it is important to understand how much traffic delay may be caused as a result of microscopic vehicle interactions. Conventional traffic simulations that explicitly track these interactions are time-consuming. Prior work introduced an analytical traffic model for unmanaged intersections. The traffic delay at the intersection is modeled as an event-driven stochastic process, whose dynamics encode microscopic vehicle interactions. This paper studies the traffic delay in a two-lane intersection using the model. We perform rigorous analyses concerning the distribution of traffic delay under different scenarios. We then discuss the relationships between traffic delay and multiple factors such as traffic flow density, unevenness of traffic flows, temporal gaps between two consecutive vehicles, and the passing order.

Human-robot interactions have been recognized to be a key element of future industrial collaborative robots (co-robots). Unlike traditional robots that work in structured and deterministic environments, co-robots need to operate in highly unstructured and stochastic environments. To ensure that co-robots operate efficiently and safely in dynamic uncertain environments, this paper introduces the robot safe interaction system. In order to address the uncertainties during human-robot interactions, a unique parallel planning and control architecture is proposed, which has a long term global planner to ensure efficiency of robot behavior, and a short term local planner to ensure real time safety under uncertainties. In order for the robot to respond immediately to environmental changes, fast algorithms are used for real-time computation, i.e., the convex feasible set algorithm for the long term optimization, and the safe set algorithm for the short term optimization. Several test platforms are introduced for safe evaluation of the developed system in the early phase of deployment. The effectiveness and the efficiency of the proposed method have been verified in experiment with an industrial robot manipulator.

Human-robot collaborations have been recognized as an essential component for future factories. It remains challenging to properly design the behavior of those co-robots. Those robots operate in dynamic uncertain environment with limited computation capacity. The design objective is to maximize their task efficiency while guaranteeing safety. This paper discusses a set of design principles of the safe and efficient robot collaboration system (SERoCS) for the next generation co-robots, which consists of robust cognition algorithms for environment monitoring, optimal task planning algorithms for safe human-robot collaborations, and safe motion planning and control algorithms for safe human-robot interactions. The proposed SERoCS will address the design challenges and significantly expand the skill sets of the co-robots to allow them work safely and efficiently with their human counterparts. The development of SERoCS will create a significant advancement toward adoption of co-robots in various industries. The experiments validate the effectiveness of SERoCS.

Considering the growing demand of real-time motion planning in robot applications, this paper proposes a fast robot motion planner (FRMP) to plan collision-free and time-optimal trajectories, which applies the convex feasible set algorithm (CFS) to solve both the trajectory planning problem and the temporal optimization problem. The performance of CFS in trajectory planning is compared to the sequential quadratic programming (SQP) in simulation, which shows a significant decrease in iteration numbers and computation time to converge a solution. The effectiveness of temporal optimization is shown on the operational time reduction in the experiment on FANUC LR Mate 200iD/7L.

Recent advances in robotics suggest that human robot interaction (HRI) is no longer a fantasy, but is happening in various fields such as industrial robots, autonomous vehicles and medical robots. Human safety is one of the biggest concerns in HRI. As humans will respond to the robot’s movement, interactions need to be considered explicitly by the robot. A systematic approach to design the robot behavior towards safe HRI is discussed in this chapter. By modeling the interactions in a multi-agent framework, the safety issues are understood as conflicts in the multi-agent system. By mimicking human’s social behavior, the robot’s behavior is constrained by the "no-collision" social norm and the uncertainties it perceives for human motions. An efficient action is then found within the constraints. Both analysis and human-involved simulation verify the effectiveness of the method.

The development of automated vehicles brings new challenges to road safety. The behavior of the automated vehicles should be carefully designed in order to interact with the environment and other vehicles efficiently and safely. The robustly-safe automated driving system (ROAD) prevents or minimizes occurrences of collisions of the automated vehicles with surrounding vehicles and moving objects while maintaining efficiency. A set of design principles are elaborated based on the previous work, including robust perception and cognition algorithms for environment monitoring and high level decision making and low level control algorithms for safe maneuvering of the automated vehicle. The autonomous driving problem in mixed traffic is posed as a stochastic optimization problem, which is solved by 1) behavior classification and trajectory prediction of the surrounding vehicles, and 2) an unique parallel planner architecture which addresses the efficiency goal and the safety goal separately. Extensive simulations are performed to validate the effectiveness of the proposed algorithm, which evaluate both high level decision making and low level vehicle regulation. Two typical scenarios are considered, driving on freeway and driving in unstructured environments such as parking lots. In the simulation, multiple moving agents representing surrounding vehicles and pedestrians are added to the environment, some of which are controlled by human users in order to test the real time response of the automated vehicle.

Trajectory smoothing is an important step in robot motion planning, where optimization methods are usually employed. However, the optimization problem for trajectory smoothing in a clustered environment is highly non-convex, and is hard to solve in real time using conventional non-convex optimization solvers. This paper discusses a fast online optimization algorithm for trajectory smoothing, which transforms the original non-convex problem to a convex problem so that it can be solved efficiently online. The performance of the algorithm is illustrated in various cases, and is compared to that of conventional sequential quadratic programming (SQP). It is shown that the computation time is greatly reduced using the proposed algorithm.

Conventional layered planning architecture temporally partitions the spatiotemporal motion planning by the path and speed, which is not suitable for lane change and overtaking scenarios with moving obstacles. In this paper, we propose to spatially partition the motion planning by longitudinal and lateral motions along the rough reference path in the Frenét Frame, which makes it possible to create linearized safety constraints for each layer in a variety of on-road driving scenarios. A generic environmental representation methodology is proposed with three topological elements and corresponding longitudinal constraints to compose all driving scenarios mentioned in this paper according to the overlap between the potential path of the autonomous vehicle and predicted path of other road users. Planners combining A* search and quadratic programming (QP) are designed to plan both rough long-term longitudinal motions and short-term trajectories to exploit the advantages of both search-based and optimization-based methods. Limits of vehicle kinematics and dynamics are considered in the planners to handle extreme cases. Simulation results show that the proposed framework can plan collision-free motions with high driving quality under complicated scenarios and emergency situations.

To generate safe and efficient trajectories for an automated vehicle in dynamic environments, a layered approach is usually considered, which separates path planning and speed profile planning. This paper is focused on speed profile planning for a given path that is represented by a set of waypoints. The speed profile will be generated using temporal optimization which optimizes the time stamps for all waypoints along the given path. The formulation of the problem under urban driving scenarios is discussed. To speed up the computation, the non-convex temporal optimization is approximated by a set of quadratic programs which are solved iteratively using the slack convex feasible set (SCFS) algorithm. The simulations in various urban driving scenarios validate the effectiveness of the method.

Automatic valet parking is widely viewed as a milestone towards fully autonomous driving. One of the key problems is nonholonomic path planning in maze-like environments (e.g. parking lots). To balance efficiency and passenger comfort, the planner needs to minimize the length of the path as well as the number of gear shifts. Lattice A* search is widely adopted for optimal path planning. However, existing heuristics do not evaluate the nonholonomic dynamic constraint and the collision avoidance constraint simultaneously, which may mislead the search. To efficiently search the environment, the boundary layer heuristic is proposed which puts large cost in the area that the vehicle must shift gear to escape. Such area is called the boundary layer. A simple and efficient geometric method to compute the boundary layer is proposed. The admissibility and consistency of the additive combination of the boundary layer heuristic and existing heuristics are proved in the paper. The simulation results verify that the introduction of the boundary layer heuristic improves the search performance by reducing the computation time by 56.1%.

Real time trajectory optimization is critical for robotic systems. Due to nonlinear system dynamics and obstacles in the environment, the trajectory optimization problems are highly nonlinear and non convex, hence hard to be computed online. Liu, Lin and Tomizuka proposed the convex feasible set algorithm (CFS) to handle the non convex optimization in real time by convexification. However, one limitation of CFS is that it will not converge to local optima when there are nonlinear equality constraints. In this paper, the slack convex feasible set algorithm (SCFS) is proposed to handle the nonlinear equality constraints, e.g. nonlinear system dynamics, by introducing slack variables to relax the constraints. The geometric interpretation of the method is discussed. The feasibility and convergence of the SCFS algorithm is proved. It is demonstrated that SCFS performs better than existing non convex optimization methods such as interior-point, active set and sequential quadratic programming, as it requires less computation time and converges faster.

Safety is a fundamental issue in robotics, especially in the growing application of human-robot interaction (HRI), where collision avoidance is an important consideration. In this paper, a novel real-time velocity based collision avoidance planner is presented to address this problem. The proposed algorithm provides a solution to deal with both collision avoidance and reference tracking simultaneously. An invariant safe set is introduced to exclude the dangerous states that may lead to collision, and a smoothing function is introduced to adapt different reference commands and to preserve the invariant property of the safe set. A real-time experiment with a moving obstacle is conducted on FANUC LR Mate 200iD/7L.

This paper proposes a novel communication-enabled distributed conflict resolution mechanism for interactions among connected autonomous vehicles (CAVs). The environments under consideration are generalized intersections where multiple incoming and outgoing lanes intersect. All vehicles close to the intersection are requested to broadcast their estimated times to occupy the conflict zones where a conflict zone is identified when the extensions of two incoming lanes intersect. The conflict resolution strategy is decoupled temporally for a vehicle. In a decision maker, the vehicle computes the desired time slot to pass the conflict zones based on the broadcasted information by solving a conflict graph locally. Then in a motion planner, the vehicle computes the desired speed profile by solving a temporal optimization problem constrained in the desired time slot. The estimated time to occupy the conflict zones given the new speed profile is then broadcasted again. It is shown theoretically that the aggregation of these local decisions solves the conflicts globally. Moreover, this mechanism increases the efficiency of autonomous vehicles, and outperforms conventional mechanisms such as traffic light or stop sign in a four-way intersection in terms of delay time and throughput as demonstrated by simulation.

In factories of the future, humans and robots are expected to be co-workers and co-inhabitants in the flexible production lines. It is important to ensure that humans and robots do not harm each other. This paper is concerned with functional issues to ensure safe and efficient interactions among human workers and the next generation intelligent industrial co-robots. The robot motion planning and control problem in a human involved environment is posed as a constrained optimal control problem. A modularized parallel controller structure is proposed to solve the problem online, which includes a baseline controller that ensures efficiency, and a safety controller that addresses real time safety by making a safe set invariant. Capsules are used to represent the complicated geometry of humans and robots. The design considerations of each module are discussed. Simulation studies which reproduce realistic scenarios are performed on a planar robot arm and a 6 DoF robot arm. The simulation results confirm the effectiveness of the method.

The rise of robot-robot interactions (RRI) is pushing for novel controller design techniques. Instead of using fixed control laws, robots should choose actions to minimize some cost functions specified by the designer. However, since the cost function of one robot may not be known to other robots (information asymmetry), special reasoning strategies are needed for multiple robots to learn to cooperate. Analysis shows that conventional learning and control strategies can lead to instability in a multi-agent system since the imperfection of other agents is not considered. In this paper, a new learning and control strategy that deals with interactions among imperfect agents is proposed. Analysis and simulation results show that the proposed strategy improves the performance of the system.

The development of automated vehicles brings new challenges to road safety. The behavior of the automated vehicles should be carefully designed in order to interact with the environment and other vehicles efficiently and safely. This paper is focused on the learning and decision making methods for the automated vehicles towards safe freeway driving. Based on a multi-agent traffic model, the decision making problem is posed as an optimal control problem, which is solved by 1) behavior classification and trajectory prediction of the surrounding vehicles, and 2) a unique parallel planner architecture which addresses the efficiency goal and the safety goal separately. The simulation results demonstrate the effectiveness of the algorithm.

Recent works of non-rigid registration have shown promising applications on tasks of deformable manipulation. Those approaches use thin plate spline-robust point matching (TPS-RPM) algorithm to regress a transformation function, which could generate a corresponding manipulation trajectory given a new pose/shape of the object. However, this method regards the object as a bunch of discrete and independent points. Structural information, such as shape and length, is lost during the transformation. This limitation makes the object’s final shape to differ from training to test, and can sometimes cause damage to the object because of excessive stretching. To deal with these problems, this paper introduces a tangent space mapping (TSM) algorithm, which maps the deformable object in the tangent space instead of the Cartesian space to maintain structural information. The new algorithm is shown to be robust to the changes in the object’s pose/shape, and the object’s final shape is similar to that of training. It is also guaranteed not to overstretch the object during manipulation. A series of rope manipulation tests are performed to validate the effectiveness of the proposed algorithm.

From the driving strategy point of view, a major challenge for autonomous vehicles in urban environment is to behave defensively to potential dangers, yet to not overreact to threats with low probability. As it is overwhelming to program the action rules case-by-case, a unified planning framework under uncertainty is proposed in this paper, which achieves a non-conservatively defensive strategy (NCDS) in various kinds of scenarios for urban autonomous driving. First, uncertainties in urban scenarios are simplified to two probabilistic cases, namely passing and yielding. Two-way-stop intersection is used as an exemplar scenario to illustrate the derivation of probabilities for different intentions of others via a logistic regression model. Then a deterministic planner is designed as the baseline. Also, a safe set is defined, which considers both current and preview safety. The planning framework under uncertainty is then proposed, in which safety is guaranteed and overcautious behavior is prevented. Finally, the proposed planning framework is tested by simulation in the exemplar scenario, which demonstrates that an NCDS can be realistically achieved by employing the proposed framework.

To address the safety issues in human robot interactions (HRI), a safe set algorithm (SSA) was developed previously. However, during HRI, the uncertainty levels are changing in different phases of the interaction, which is not captured by SSA. A safe exploration algorithm (SEA) is proposed in this paper to address the uncertainty levels in the robot control. To estimate the uncertainty levels online, a learning method in the belief space is developed. A comparative study between SSA and SEA is conducted. The simulation results confirm that SEA can capture the uncertainty reduction behavior which is observed in human-human interactions.

Human workers and robots are two major workforces in modern factories. For safety reasons, they are separated, which limits the productive potentials of both parties. It is promising if we can combine human’s flexibility and robot’s productivity in manufacturing. This paper investigates the modeling and controller design method of workspace sharing human-robot assembly teams and adopts a two-layer interaction model between the human and the robot. In theoretical analysis, enforcing invariance in a safe set guarantees safety. In imple- mentation, an integrated method concerning online learning of closed loop human behavior and receding horizon control in the safe set is proposed. Simulation results in a 2D setup confirm the safety and efficiency of the algorithm.

Human-robot interactions (HRI) happen in a wide range of situations. Safety is one of the biggest concerns in HRI. This paper proposes a safe set method for designing the robot controller and offers theoretical guarantees of safety. The interactions are modeled in a multi-agent system framework. To deal with humans in the loop, we design a parameter adaptation algorithm (PAA) to learn the closed loop behavior of humans online. Then a safe set (a subset of the state space) is constructed and the optimal control law is mapped to the set of control which can make the safe set invariant. This algorithm is applied with different safety constraints to both mobile robots and robot arms. The simulation results confirm the effectiveness of the algorithm.