Robotic Surface Finishing

Active · 2019–Now Advanced Robotics for Manufacturing Institute
Robotic Surface Finishing

Scan and Plan

Description of the image

Polishing and grinding of metallic parts is an important manufacturing operation in many industrial applications. It remains challenging to polish large complex surfaces. Polishing is predominantly done manually which is very time consuming, expensive and more importantly, can be a safety hazard for human operators using hand-held devices. We propose the assembly of a robot capable of polishing complex and relatively large surface areas of complex free form fabricated parts. By developing this robot for a complex and potentially un-safe operation, this application will lower the technical, operational, and economic barriers for companies to adopt robotic technologies.

Our planning methods support contact-rich motion in confined environments, including hybrid task constraints for robot manipulators (Sun et al., 2024) and trajectory generation using iterative convex optimization (Zhao et al., 2020).

Papers

  1. [C69] Hybrid Task Constrained Incremental Planner for Robot Manipulators in Confined Environments
    Yifan Sun, Weiye Zhao and Changliu Liu
    American Control Conference, 2024
  1. [C30] Contact-Rich Trajectory Generation in Confined Environments Using Iterative Convex Optimization
    Wei-Ye Zhao, Suqin He, Chengtao Wen and Changliu Liu
    Dynamic Systems and Control Conference, 2020

Watch the documentary below.

Spin-off: Instinct Robotics

This research is being commercialized through our spin-off, Instinct Robotics, which develops robotic surface-finishing solutions for manufacturing.

RoboFin3D

  1. [W] RoboFin3D: A Sim-to-Real Platform for Robotic Surface Finishing
    Haowei Wen, Shangtao Li, Vaibhav Sanjay, Philip Huang, Jiaoyang Li and Changliu Liu
    IROS 2026 Workshop on Sim2Real and Classical Control: From Rigorous Theory to Data-Driven Robotics, 2026

Each grinding or sanding trial permanently changes the workpiece, so real experiments are costly and hard to reproduce. RoboFin3D simulates both processes in Isaac Sim with the Newton physics engine: grinding removes material and changes the geometry, while sanding gradually changes the surface appearance. A simulated workpiece can be restored to any earlier state, so a trial can be rerun at no material cost.

Simulated and real robotic grinding and sanding
Simulated (a, c) and real (b, d) grinding with a rigid grinding wheel (top) and sanding with a spinning pad (bottom).

WeldGen samples weld curves of controllable difficulty on 8,918 workpiece meshes from the ABC dataset and adds bead geometry. The welded workpieces serve as simulation assets and as training data for perception. Real grinding and sanding results calibrate the simulation, which can be used to evaluate a control policy before real deployment.

RoboFin3D workflow linking WeldGen, the simulator, the perception model, and real experiments
RoboFin3D workflow.

A signed distance field (SDF) stores the geometry of the workpiece, and a separate surface field stores sanding progress. Grinding removes material wherever the contact pressure exceeds a threshold, which updates the SDF. Sanding removes only a thin coating, so the geometry stays fixed and the surface field records the progress. Contact computation, field updates, and rendering all operate on these fields directly on the GPU, without intermediate mesh extraction.

Simulation architecture with the SDF and the surface field in a shared GPU buffer
Simulation architecture. (a) The SDF and the surface field reside in a GPU buffer shared by CUDA and Vulkan. Newton computes hydroelastic contacts, Warp updates the fields, and custom shaders in the Isaac Sim RTX renderer query them. (b) SDF of the workpiece geometry. (c) Surface field of the workpiece appearance.

For each process, a real workpiece is ground or sanded in two passes at a 15 N force setpoint and scanned before and after each pass. The simulation repeats both passes from the same initial scan, with the same tool path and force setpoint. The first pass calibrates the contact and removal parameters, and the second is held out for evaluation. In both simulation and reality, the first grinding pass turns the rounded crest of the weld bead into a plateau, and the second pass lowers it. For sanding, the regions where the coating is fully removed in simulation and in the scans have an intersection over union (IoU) of 0.81 after the first pass and 0.89 after the second.

Optical scans and measured and simulated height profiles after two grinding passes
Measured and simulated height profiles after two grinding passes. Left: optical scans before grinding and after each pass, with the compared sections marked. Right: measured (Real) and simulated (Sim) height profiles at those sections.

The simulator labels its rendered images automatically, so a perception model can be trained on them without manual annotation. We fine-tune SAM2 to segment the unsanded regions of real workpieces. On real test images, fine-tuning on simulated images alone raises the IoU from 77.15% to 84.47%, and adding labeled real images raises it to 97.41%.

Unsanded regions predicted by pretrained and fine-tuned SAM2 on a real workpiece
Segmentation of unsanded regions on a real workpiece: (a) input image; (b) pretrained SAM2; (c) fine-tuned on simulated images; (d) fine-tuned on simulated and real images. Teal marks predicted unsanded regions.

The video below shows real and simulated grinding and sanding side by side, then fixtureless bimanual sanding in reality and in simulation.