Implement DiT for Video

Key Insight

The DiT (Diffusion Transformer) replaced the U-Net with a pure transformer for image diffusion, and turning it into a video model is mostly a matter of changing what a "patch" is. Instead of cutting a single frame into flat 2D squares, you cut the 3D VAE latent into spatiotemporal patches — little boxes that also span a few frames — and add 3D RoPE so each token carries its row, column, and frame index. The rest of the recipe is unchanged from image DiT: project each patch to a token, mix them with attention, and condition on the denoising step through AdaLN-Zero. Building this once makes the whole "Sora-class" family demystified — it is the same transformer you already know, fed a sequence of motion-aware tokens instead of word tokens.