PerSEVE website is an interactive web interface for browsing large-scale 3D assets, corresponding 2D renderings, and IoU-based evaluation results. It unifies model exploration, visual inspection, and evaluation into a single viewer within the ICL website.

Overview

Fixtureless manufacturing depends on reliable computer vision to estimate the poses of parts, objects, and subassemblies, enabling flexible and automated production workflows. Although modern computer-vision methods can achieve strong performance, their deployment remains constrained by the need for large quantities of product-specific labeled data, which are costly and time-consuming to produce manually. To overcome this limitation, PerSEVE proposes a machine-scale data-generation framework that aggregates manufacturing-specific 3D models and provides a turnkey pipeline for automatically generating labeled 2D training data from proprietary 3D assets. This capability will support the development of a manufacturing-focused image foundation model and substantially reduce the time and cost required to create accurate, task-specific computer-vision models for deployment in real-world manufacturing environments.

Authentic Data Collection, Data Curation

3D model dataset

We leverage the open-source ABC dataset, which contains approximately one million 3D models. From this collection, we analyze 20,000 models and assess their geometric quality using metrics such as facet count, vertex count, surface area, volume, volume-to-bounding-box ratio, watertightness, and connectivity. Models with poor geometric quality or limited mechanical relevance are removed through a systematic filtering process. The resulting curated dataset can be explored and accessed through an interactive data browser.

Interactive viewer Launch the PerSEVE model browser Browse curated 3D model assets and renderings. Open browser

Image description

2D synthetic data generation

We use NVIDIA Isaac Sim to automatically generate labeled training images. Varying numbers of 3D models are imported into simulated scenes and assigned randomized positions and orientations. We then apply diverse materials, lighting conditions, and camera viewpoints to produce visually varied renderings. During rendering, the pipeline automatically generates ground-truth annotations, including pixel-accurate segmentation masks, 2D bounding boxes, depth maps, and surface normals.

Image description

SAM4MFG Foundation Model Training

The Segment Anything Model (SAM) is a powerful foundation model for object segmentation that supports multiple prompt types, including points, bounding boxes, text, and visual exemplars. However, industrial imagery differs substantially from the natural-image datasets used to train SAM, resulting in a significant domain gap. To address this limitation, we fine-tune SAM 2.1 using our synthetic industrial dataset. The resulting models substantially outperform the off-the-shelf SAM 2 model in terms of intersection-over-union (IoU) accuracy.

Evaluation gallery Open the IOU evaluation gallery Compare benchmark results across SAM model variants. Open gallery

IOU comparison

Adversarial synthetic data generation

A closed-loop pipeline for continually improving a fine-tuned segmentation model after conventional fine-tuning has reached a performance plateau. The model is first evaluated to identify failure cases, which are then represented in a shared object–scene feature space and clustered into distinct failure modes. New synthetic scenes are generated by sampling from these clusters, and the model is retrained on the expanded data mixture. Because each iteration starts from the checkpoint produced in the previous round, performance gains accumulate over successive cycles.

IOU comparison

CAD-Prompted SAM3

Language-prompted segmentation struggles with uncommon or instance-specific industrial objects, while image exemplars often rely on appearance cues that vary across materials, finishes, and colors. We propose a CAD-prompted segmentation framework built on SAM3 that uses canonical multi-view CAD renderings to provide geometry-based conditioning. Trained on synthetic scenes with randomized appearances, the model enables single-stage mask prediction for objects that cannot be reliably identified through language or appearance alone.

IOU comparison

Contributors: Xusheng Luo, Kevin Tang, Rohan Nagabhirava, Jianlin Dou, Bowei Li, Han Zhou, Xiangyu Huang, Yuyang Pang, Yi Li

Publications:

[U] CAD-Prompted SAM3: Geometry-Conditioned Instance Segmentation for Industrial Objects
Zhenran Tang, Rohan Nagabhirava and Changliu Liu
arXiv:2602.20551, 2026
Citation Formats:
```
    
```