BrickGPT won the Best Paper Award (Marr Prize) at ICCV 2025

🏆 We are delighted and honored to share that BrickGPT has been awarded the Best Paper Award (Marr Prize) at the 2025 International Conference on Computer Vision (ICCV) in Hawaiʻi! Image description

🙌 Huge congratulations to the team! Image description

🌟 BrickGPT marks our first step into the realm of Physical AI, and we’re just getting started. The future is not only about models that imagine, but about systems that understand, create, and act in the physical world.

[C106] Generating Physically Stable and Buildable Brick Structures from Text
Ava Pun, Kangle Deng, Ruixuan Liu, Deva Ramanan, Changliu Liu and Jun-Yan Zhu
International Conference on Computer Vision, 2025
Best Paper Award (Marr Prize)
Citation Formats:
```
    
```
Abstract:

We introduce LegoGPT, the first approach for generating physically stable LEGO brick models from text prompts. To achieve this, we construct a large-scale, physically stable dataset of LEGO designs, along with their associated captions, and train an autoregressive large language model to predict the next brick to add via next-token prediction. To improve the stability of the resulting designs, we employ an efficient validity check and physics-aware rollback during autoregressive inference, which prunes infeasible token predictions using physics laws and assembly constraints. Our experiments show that LegoGPT produces stable, diverse, and aesthetically pleasing LEGO designs that align closely with the input text prompts. We also develop a text-based LEGO texturing method to generate colored and textured designs. We show that our designs can be assembled manually by humans and automatically by robotic arms. We also release our new dataset, StableText2Lego, containing over 47,000 LEGO structures of over 28,000 unique 3D objects accompanied by detailed captions, along with our code and models at the project website: this URL.

Teaser: