Googles AI helped me make bad Nintendo knockoffs

This week, a new generative AI tool from Google let me create bad knockoffs of 3D Nintendo worlds.

Check out my version of something like Super Mario 64:

I didn’t like Metroid Prime 4: Beyond, but it’s better than my version of a Metroid Prime experience:

Or how about my take on The Legend of Zelda: Breath of the Wild, complete with a paraglider (and, briefly, a second Link):

It was all possible thanks to Project Genie, an experimental research prototype that Google gave me access to this week, though I don’t think I’m using it in exactly the way Google intended.

Google DeepMind has been putting a lot of effort into building its AI “world” models that can generate virtual interactive spaces with text or images as prompts. The company announced its impressive-looking Genie 3 model last year, but it was only available as “a limited research preview” at the time. Project Genie, which will be rolling out to Google AI Ultra subscribers in the US starting today, will be the first opportunity for more people to actually try out what Genie 3 is capable of.

Google is releasing Project Genie now partly because it wants to see how people use it. “It’s really for us to actually learn about new use cases that we hadn’t thought about,” Diego Rivas, a product manager at Google DeepMind, tells The Verge. The company is already excited about how Genie could help to visualize scenes for filmmaking or for interactive educational media. You could, if you wanted, take a photo of your kids’ favorite toy and use it to prompt a Genie-generated world. Genie could potentially help robots navigate the real world, too. But Project Genie isn’t yet an “end-to-end product that we expect people to just use every day,” stressed Shlomi Fruchter, a Google DeepMind research director.

With Project Genie, you pick from a bunch of worlds designed by Google or define prompts for the environments and characters you want to create in your own world. After a brief wait, Genie first generates a thumbnail, then you can have it generate the world. You can explore each generated world for 60 seconds, and each has a resolution of about 720p and a frame rate of about 24fps. While you’re in one, you can (typically) move your character with the WASD keys, jump or go higher with a tap of your space bar, and turn the camera with arrow keys.

One of Google’s worlds, called “Rollerball,” features a blue orb in a white, snowy world, and as you roll around, the orb leaves a trail of paint behind it. As a “game,” Project Genie wasn’t great. There was nothing to do but roll around; there weren’t any objectives or goals. There was no sound. There was frustrating input lag that was even worse than what I sometimes experience with cloud gaming. (Some of this could be due to the generally poor Wi-Fi I get in my office.)

Over the course of the 60-second experience, Genie sometimes forgot to show a paint streak where I had previously rolled. Occasionally, the ball would randomly stop laying down paint at all. So I started to distrust Genie’s ability to recall what I had already seen with my own eyes.

Another Google-designed world, “Backyard Racetrack,” was a little more fun because there was an actual track to follow. My racing lines were awful — the input lag didn’t help — but I enjoyed trying to make the turns and stay on the road. Near the end of the experience, though, part of the track unexpectedly turned into grass, which ruined the immersion. And the wheel rims looked really janky.

I had a lot more fun pushing the limits of Project Genie to try and make 3D, AI-generated games featuring recognizable characters, like with my Super Mario, Metroid Prime, and The Legend of Zelda-themed worlds. While they made me laugh, the worlds don’t have scores or anything to strive for, so there’s nothing to do but walk or jump around. Even if there were specific things to do, the input lag made the worlds basically unplayable. (Again, this may be a Wi-Fi issue, but even when I was closer to my router, I still experienced lag.)

I wasn’t able to make everything I wanted. Project Genie wouldn’t generate a world that I prompted with the scenario of Kingdom Hearts — here was my prompt, if you’re curious:

Environment

It’s a world filled with Disney characters with a steampunk vibe. Donald and Goofy are your sidekicks. Jack Skellington is present, as is Cloud Strife.

Character

You are a spunky, anime teenager with spiky brown hair wielding a blade that is like a key.

When I removed the specific names of characters and wrote descriptions of them instead, Project Genie generated a thumbnail preview of the world featuring characters that were dead ringers for Sora (the series’ protagonist), Donald, Goofy, Jack Skellington, and Cloud. But when I tried to generate the actual experience, Project Genie blocked me.

I asked about why I was able to generate worlds with Nintendo characters. “Project Genie is an experimental research prototype designed to follow prompts a user provides,” Rivas says. “As with all experiments, we are monitoring closely and listening to user feedback.” Rivas also notes that the Genie 3 model was “trained primarily on publicly available data from the web.” (This probably partially explains why Link deployed his paraglider in my test, which surprised me. At a high level, the Genie model is constantly trying to predict the next frame, and I’m sure there are many videos of people jumping in Breath of the Wild and then gliding forward, which the model probably learned from.) Shortly before publishing this article, Project Genie stopped letting me generate worlds based on Super Mario 64 due to “interests of third-party content providers.”

Assuming Google clamps down on the ability to generate interactive worlds based on known gaming franchises — I can’t imagine Nintendo will be happy with what I was able to generate! — Project Genie otherwise isn’t that great at the moment. The input lag and 60-second limit make them pretty poor interactive experiences. Occasionally, I couldn’t control my character at all, only the camera. After the weirdness with the paint stripes and the road turning into grass, I had a general feeling that I couldn’t trust the worlds to stay consistent from moment to moment.

Project Genie is better than some AI-generated worlds I tried last year, but it’s still much worse than an actual handcrafted video game or interactive experience. Fruchter described a potential future where the line blurs between different kinds of media thanks to technology like Genie, but I think it has a long way to go to get there.

Perhaps my standards are too high. Project Genie is an experimental research prototype, after all. And maybe I’ll feel differently after the technology improves down the line. But I can’t imagine that people will want to spend an extended period of time jumping into these types of AI-generated worlds anytime soon. With world models, I don’t think we have to worry about the genie being out of the bottle just yet.