Roblox Gives Users AI Tools to Generate 3D Models From Text for Games

Roblox is a popular online gaming platform that quietly became the closest to an actual metaverse. Now, it has set its sights on generative AI, with the team behind it looking to revolutionize games creations.

Roblox is giving users a conversational AI assistant to generate 3D models. Dubbed Roblox Assistant, users can use natural language prompts to create assets for games or apply interactive behaviors to objects.

Roblox as a platform is all about making games – users can create their own games and even monetize them. The new Roblox Assistant is designed to empower users to build games, providing help with generating atmospheres in a game's biome.

Users can also add to existing models, like creating armor for bears, as shown in the example below.

Credit: Roblox

Alongside building assets for games, the Roblox Assistant can be used to help with code generation. The tool can improve existing code, or help debug and suggest fixes for code that isn’t working properly.

Daniel Sturman, Roblox’s chief technology officer, wrote in a blog post that the new Assistant tool “enables creators of all skill levels to spend significantly less time on the mundane, repetitive tasks involved in creating and more time on high-value activities, like narrative, game-play, and experience design.”

Related:Roblox Gets Generative AI: Users Can Build Virtual Worlds from Text

The Assistant tool was teased back in March. Roblox used publicly available 3D models to train it.

User data usage and 3D avatar generation

Roblox Assistant was announced at the annual Roblox Developer Conference (RDC). Also announced at the event was that developers can contribute their anonymized Luau script data for the company to use to improve its AI tools. Luau is the scripting language creators use in Roblox Studio to build games and assets.

Developers would have to opt in to provide the data, which would be used to improve tools like Assistant. Those that do would also allow for their script data to be made available to third parties to train their AI tools to be better at suggesting Luau code.

The Roblox team also announced it is working on training AI models to generate Roblox avatars.

In theory, it would allow users to generate 3D avatar models. The team is still figuring it out with plans to release an AI avatar creation tool in 2024.

Sturman said one possibility would be using pre-trained text-to-image diffusion models to create 3D models by means of a generative adversarial network (GAN)-based. A ControlNet would then be added to layer in predefined poses to guide the resulting multi-view images of the avatars.

Related:AI 3D Generation Models, Applications and Tools: The Big List