Real NVP Toy

Key Insight

A normalizing flow learns to generate data by warping simple random noise through a series of reversible steps, and because every step is reversible it can also report the exact probability of any point — something most generative models cannot do. This project trains a small Real NVP flow on easy 2D shapes like two moons or a swiss roll, then visualizes both the density it learned and the samples it draws. Working in 2D lets you actually see the learned distribution as a heatmap, which makes the otherwise abstract idea of "modeling a probability density" concrete. Why insist on reversibility at all? Because the flow's whole superpower — reading off the exact probability of any point — depends on it. To score a point, the flow runs it backwards through every step until it lands in the simple noise it must have come from, tracking how much each step squished or stretched space along the way; if even one step could not be undone, that backward trip would be impossible and the probability uncomputable. It is like a currency exchange that promises a fixed rate both ways: because you can always convert dollars → euros → dollars and get your exact amount back, you can always work out what any pile of euros is worth. A one-way paper shredder gives you no way back — and no way to value what went in. The catch you will notice: keeping every step reversible heavily constrains the architecture, which is why flows lost ground to diffusion on real images.