Talk@UWaterloo - Steering Towards Safe Human-AI Interactions

Changliu gave a talk at University of Waterloo, titled Steering Towards Safe Human-AI Interactions.

Abstract: As generative AI and robotics increasingly integrate into daily life, ensuring their safe interaction with humans remains a critical challenge. Safety concerns extend beyond physical interactions—such as preventing collisions—to conversational safety, where AI must avoid exchanging harmful or dangerous information. In this talk, I will discuss how we frame these challenges as constraint satisfaction problems and address them using forward invariance principles from control theory. The first line of work I will discuss focuses on Physical Safety. I will introduce SPARK, a comprehensive toolbox and benchmark designed to ensure safety in humanoid autonomy and teleoperation. The second line of work addresses Conversational Safety. Large language models (LLMs) are highly vulnerable to multi-turn jailbreaking attacks, where contextual drift gradually leads them away from safe behavior. To mitigate this, we propose a safety steering framework grounded in control theory to maintain invariant safety in multi-turn dialogues. Lastly, as many safety certificates are learned via neural networks, a critical question arises: how can we certify Neural Safety Certificates? I will discuss formal verification methods designed to provide guarantees on their reliability. By applying control-theoretic safety principles across diverse domains—from physical robot safety to conversational AI—we aim to build AI systems that interact with humans in both safe and trustworthy ways.

This talk highlights the following work from ICL:

[U] SPARK: A Modular Benchmark for Humanoid Robot Safety
Yifan Sun, Rui Chen, Kai S Yun, Yikuan Fang, Sebin Jung, Feihan Li, Bowei Li, Weiye Zhao and Changliu Liu
arXiv preprint arXiv:2502.03132, 2025
Citation Formats:
```
    
```
Abstract:

This paper introduces the Safe Protective and Assistive Robot Kit (SPARK), a comprehensive benchmark designed to ensure safety in humanoid autonomy and teleoperation. Humanoid robots pose significant safety risks due to their physical capabilities of interacting with complex environments. The physical structures of humanoid robots further add complexity to the design of general safety solutions. To facilitate the safe deployment of complex robot systems, SPARK can be used as a toolbox that comes with state-of-the-art safe control algorithms in a modular and composable robot control framework. Users can easily configure safety criteria and sensitivity levels to optimize the balance between safety and performance. To accelerate humanoid safety research and development, SPARK provides a simulation benchmark that compares safety approaches in a variety of environments, tasks, and robot models. Furthermore, SPARK allows quick deployment of synthesized safe controllers on real robots. For hardware deployment, SPARK supports Apple Vision Pro (AVP) or a Motion Capture System as external sensors, while also offering interfaces for seamless integration with alternative hardware setups. This paper demonstrates SPARK’s capability with both simulation experiments and case studies with a Unitree G1 humanoid robot. Leveraging these advantages of SPARK, users and researchers can significantly improve the safety of their humanoid systems as well as accelerate relevant research. The open-source code is available at (https://github.com/intelligent-control-lab/spark)

Video:

[U] Dexterous Safe Control for Humanoids in Cluttered Environments via Projected Safe Set Algorithm
Rui Chen, Yifan Sun and Changliu Liu
arXiv preprint arXiv:2502.02858, 2025
Citation Formats:
```
    
```
Abstract:

It is critical to ensure safety for humanoid robots in real-world applications without compromising performance. In this paper, we consider the problem of dexterous safety, featuring limb-level geometry constraints for avoiding both external and self-collisions in cluttered environments. Compared to safety with simplified bounding geometries in sprase environments, dexterous safety produces numerous constraints which often lead to infeasible constraint sets when solving for safe robot control. To address this issue, we propose Projected Safe Set Algorithm (p-SSA), an extension of classical safe control algorithms to multi-constraint cases. p-SSA relaxes conflicting constraints in a principled manner, minimizing safety violations to guarantee feasible robot control. We verify our approach in simulation and on a real Unitree G1 humanoid robot performing complex collision avoidance tasks. Results show that p-SSA enables the humanoid to operate robustly in challenging situations with minimal safety violations and directly generalizes to various tasks with zero parameter tuning.

[U] Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks
Hanjiang Hu, Alexander Robey and Changliu Liu
arXiv preprint arXiv:2503.00187, 2025
Citation Formats:
```
    
```
Abstract:

Large language models (LLMs) are highly vulnerable to jailbreaking attacks, wherein adversarial prompts are designed to elicit harmful responses. While existing defenses effectively mitigate single-turn attacks by detecting and filtering unsafe inputs, they fail against multi-turn jailbreaks that exploit contextual drift over multiple interactions, gradually leading LLMs away from safe behavior. To address this challenge, we propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues. Our approach models the dialogue with LLMs using state-space representations and introduces a novel neural barrier function (NBF) to detect and filter harmful queries emerging from evolving contexts proactively. Our method achieves invariant safety at each turn of dialogue by learning a safety predictor that accounts for adversarial queries, preventing potential context drift toward jailbreaks. Extensive experiments under multiple LLMs show that our NBF-based safety steering outperforms safety alignment baselines, offering stronger defenses against multi-turn jailbreaks while maintaining a better trade-off between safety and helpfulness under different multi-turn jailbreak methods.

[C81] Verification of Neural Control Barrier Functions with Symbolic Derivative Bounds Propagation
Hanjiang Hu, Yujie Yang, Tianhao Wei and Changliu Liu
Conference on Robot Learning, 2024
Citation Formats:
```
    
```
Abstract:

Control barrier functions (CBFs) are important in safety-critical systems and robot control applications. Neural networks have been used to parameterize and synthesize CBFs with bounded control input for complex systems. However, it is still challenging to verify pre-trained neural networks CBFs (neural CBFs) in an efficient symbolic manner. To this end, we propose a new efficient verification framework for ReLU-based neural CBFs through symbolic derivative bound propagation by combining the linearly bounded nonlinear dynamic system and the gradient bounds of neural CBFs. Specifically, with Heaviside step function form for derivatives of activation functions, we show that the symbolic bounds can be propagated through the inner product of neural CBF Jacobian and nonlinear system dynamics. Through extensive experiments on different robot dynamics, our results outperform the interval arithmetic-based baselines in verified rate and verification time along the CBF boundary, validating the effectiveness and efficiency of the proposed method with different model complexity. The code can be found at https://github.com/intelligent-control-lab/verify-neural-CBF.