2025

1. [J30] Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods
   Xusheng Luo, Tianhao Wei, Simin Liu, Ziwei Wang, Luis Mattei-Mendez, Taylor Loper, Joshua Neighbor, Casidhe Hutchison and Changliu Liu
   ACM Transactions on Cyber-Physical Systems, 2025
   

   Preprint Page Abstract Cite

   Citation Formats: BibTeX Plain Text

   Abstract:
   

   This work addresses the certification of the local robustness of vision-based two-stage 6D object pose estimation. The two-stage method for object pose estimation achieves superior accuracy by first employing deep neural network-driven keypoint regression and then applying a Perspective-n-Point (PnP) technique. Despite advancements, the certification of these methods’ robustness remains scarce. This research aims to fill this gap with a focus on their local robustness on the system level–the capacity to maintain robust estimations amidst semantic input perturbations. The core idea is to transform the certification of local robustness into neural network verification for classification tasks. The challenge is to develop model, input, and output specifications that align with off-the-shelf verification tools. To facilitate verification, we modify the keypoint detection model by substituting nonlinear operations with those more amenable to the verification processes. Instead of injecting random noise into images, as is common, we employ a convex hull representation of images as input specifications to more accurately depict semantic perturbations. Furthermore, by conducting a sensitivity analysis, we propagate the robustness criteria from pose to keypoint accuracy, and then formulating an optimal error threshold allocation problem that allows for the setting of a maximally permissible keypoint deviation thresholds. Viewing each pixel as an individual class, these thresholds result in linear, classification-akin output specifications. Under certain conditions, we demonstrate that the main components of our certification framework are both sound and complete, and validate its effects through extensive evaluations on realistic perturbations. To our knowledge, this is the first study to certify the robustness of large-scale, keypoint-based pose estimation given images in real-world scenarios.

1. [T5] Reachable Sets for Control and Planning: from Reactive Safety to Contact-Rich Manipulation
   Simin Liu
   PhD Thesis, 2025
   

   Link Abstract Cite

   Citation Formats: BibTeX Plain Text

   Abstract:
   

   This thesis investigates two fundamental challenges in robotics: ensuring reactive safety for agile systems and achieving scalable planning for contact-rich manipulation. Despite arising in distinct contexts, both problems hinge on reasoning about system dynamics and constraints. We argue that reachable sets—long studied in control theory—provide a unifying computational primitive for tackling both. In the first part, we study reactive safety for uncertain and high-dimensional systems. We develop new methods for synthesizing control barrier functions (CBFs) using both formal optimization and learning-based techniques. Specifically, we introduce a robust-adaptive synthesis framework based on sum-of-squares programming, which guarantees safety under parametric uncertainty while reducing unnecessary conservatism. To address scalability, we propose an adversarial training framework that learns neural CBFs capable of handling higher-dimensional dynamics. Across cartpole, quadrotor, and other benchmarks, our controllers achieve 100% safety while interfering substantially less with nominal performance objectives than existing baselines. In the second part, we extend reachable-set methods to long-horizon planning for contact-rich manipulation. We present a hierarchical planner for bimanual SE(2) reorientation tasks that uses mutual reachable sets as discrete motion primitives. By covering object space with a compact set of such primitives, we reduce the combinatorial complexity of hybrid contact planning to shortest-path search on a small graph. This yields efficient and expressive global plans that outperform sampling-based methods, with successful demonstrations on real robotic hardware. Together, these contributions demonstrate how reachable sets can serve as a versatile foundation across the robotics stack: from low-level safety filters for agile systems to high-level planning abstractions for dexterous manipulation. This unifying perspective suggests broader opportunities for integrating reachability-based methods into robust and scalable robotic autonomy.