GameNGen Reproduction (Mini)

Key Insight

GameNGen stunned people by running a playable copy of DOOM with no game engine at all — every frame was generated by an action-conditioned diffusion model reacting to the player's controls. This project reproduces the idea at toy scale: train the same kind of model on a much simpler game — an Atari title or a GridWorld — then actually play it, feeding your live keypresses in as the action and watching the model paint the next frame. Going small matters because real-time play demands the model render a frame in tens of milliseconds, so a tiny game keeps both the training cost and the per-frame latency within reach of a single GPU. You will feel firsthand where a learned world model holds together and where it drifts into nonsense.