Enter the Oasis: An interactive, explorable world model

A few days ago, I came across this post on Reddit about an AI model trained on game footage of Minecraft. The model, Oasis, generates video on the fly in response to user input. But unlike other Gen AI video models like Sora, Oasis creates video that users can interact with on a frame-by-frame basis.

The blog post announcing Oasis described it as:

The first playable AI model that generates open-world games. Unlike many AI video models, which generate video from text, Oasis generates video frame-by-frame from keyboard and mouse inputs. Oasis is the first model in a research partnership between Etched and Decart, a new AI lab focused on building new generative experiences.

I hadn’t heard of Etched or Decart before this, but that’s kind of the norm these days. AI labs and AI-focused startups continue to pop up daily, not to mention the countless open-source and small-scale projects pushing the frontier of what Generative AI can do.

Oasis also serves as something of a technical demo-cum-buzz builder for Etched’s forthcoming transformer ASIC (application-specific integrated circuit). You can read the details on their site, but Etched is betting big on interactive video as “the future of the Internet.” They claim their chip will, “run video models in high-definition, at playable frame-rates, and to many simultaneous users.”

Whether or not Etched delivers a game-changing chip, I’m pretty sure they’re on the money when it comes to interactive video catching on with the masses.

We’ve already seen demos of Gen AI transforming the role of non-playable characters (NPCs) in games, giving them the ability to break out of hard-coded loops and interact with user characters in more spontaneous and creative ways.

Imagine, then, an entire game world generated in real-time based on your player actions. No two game sessions would be the same. Open-world games could go on infinitely, with ever-changing AI generated landscapes, characters, and play dynamics.

And the possibilities aren’t limited to gameplay. As Etched wrote:

As video models start scaling, they are learning to represent entire physical worlds and games, enabling entirely new product categories. Whether gaming, generative content, or education, we believe that large, low-latency, interactive video models will be central to the next wave of AI products.

For now, Oasis is essentially a technical demo. A very cool, very thought provoking demo, but also a demo rife with quirks, hallucinations, and a feature or two labelled with, “This is in beta: expect it to break” disclaimers.

My own spin through one of the Oasis maps began as a familiar facsimile of Minecraft. But it soon gave way to a disjointed experience largely centered around a blue sky with several random-looking objects floating through it (Landforms? Clouds? Not sure :thinking: ). A quick scroll of Reddit comments confirmed my experience as par for the course in these early days of the model, if not a bit tame when it came to the unexpected.

Not that the random, unplayable moments have deterred any early adopters’ enthusiasm:

Try Oasis yourself here.

@noah this is pretty wild… did they name it Oasis after Ready Player One? Going to check this out!

I’d have to think so, right? Unless someone involved is a big Wonderwall fan :wink:

The whole thing is wild, and the implications of training interactive world models is kind of mind boggling when you start really thinking about it.

But I do not want a world where we’re all living in the Stacks and reality is purely digital!