LegoGPT is an AI model that creates physically stable Lego structures from text prompts. Not only does it design Lego models that match text-supplied descriptions, it also ensures they can be built brick by brick in the real world, either by hand or with robotic assistance.

LegoGPT, which comes from Carnegie Mellon University, is outlined in a paper titled "Generating Physically Stable and Buildable Lego Designs from Text". In it the authors, led by Ava Pun explain that while many existing 3D generation models focus on making diverse objects with detailed geometry, such designs often can't be physically made commenting:

"Without proper support, parts of the design can collapse, float, or remain disconnected" 

By contrast their AI-generated LEGO structures are physically stable and are buildable.

To build LegoGPT, the Carnegie Mellon team repurposed the technology behind large language models (LLMs), similar to the kind that run ChatGPT, for "next-brick prediction" instead of next-word prediction. To do so, the team fine-tuned LLaMA-3.2-1B-Instruct, an instruction-following language model from Meta.

The team then augmented the brick-predicting model with a separate software tool that can verify physical stability using mathematical models that simulate gravity and structural forces.

To train the model, the team assembled a new dataset,  "StableText2Lego," containing over 47,000 stable Lego structures paired with descriptive captions generated by a separate AI model, OpenAI's GPT-4o. Each structure underwent physics analysis to ensure it could be built in the real world. 

The paper reports that LegoGPT produces stable, diverse, and aesthetically pleasing LEGO designs that align closely with the input text prompts and that our designs can be assembled manually by humans and automatically by robotic arms:

The project is open source on GitHub and there's a Demo on Hugging Face if you want to try it out.

More Information 

Generating Physically Stable and Buildable LEGO Designs from Text by Ava Pun, Kangle Deng, Ruixuan Liu, Deva Ramanan, Changliu Liu, Jun-Yan Zhu

LegoGPT on GitHub

Related Articles

AI Builds Lego From The Manual

DIY AI To Sort 2 Metric Tons Of Lego

A Worm's Mind In A Lego Body

A Turing Machine In LEGO

Lego Difference Engine

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

 

Early 2025 Java Conferences Galore Part 3 23/05/2025 We continue the lowdown on Java conferences. Having looked initally at sessions from three Voxxed events, last week we explored two Devoxx events and JavaOne. This week it's the turn of JCha [ ... ] + Full Story

Z3 Completed This Day In 1941 12/05/2025 On May 12, 1941 Konrad Zuse completed his Z3 computer, the first program-controlled electromechanical digital computer. It followed in the footsteps of the Z1 - the world’s first binary digital [ ... ] + Full Story

More News ACM Grace Hopper Award Recognizes Breakthrough Techniques in Algorithm Design FSF Hackathon To Improve Free Software Apache Ozone 2 Adds HSync Support The AI Scam Early 2025 Java Conferences Galore Part 2 Microsoft Launches Human Centered AI Tool Three Tools To Run MCP On Your Github Repositories AI Highlights From Google I/O 2025 Turing Papers At Auction NVIDIA Announces Robot Motion Simulation Generators NVIDIA CUDA Dive Using Python LiteCLI SQLite Client Is Now Powered By LLM Apache Gravitino 0.9 Released

 

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info