Noisy-TV Experiment

Key Insight

The noisy-TV problem is the Achilles' heel of simple prediction-error exploration methods, and this project reproduces it on purpose. You drop a "television" into the environment that displays fresh random static every step; because the static is truly unpredictable, a naive pixel-level curiosity agent — or one using RND (which maps raw states through a fixed random network) — earns a large intrinsic reward every single time it looks at the screen, so it sits and stares forever instead of exploring. Why it matters: it draws the sharp line between novelty (something genuinely new to learn) and mere stochasticity (randomness you can never learn), and it explains why methods like the ICM (Intrinsic Curiosity Module) measure surprise in a learned, controllable feature space (ignoring the uncontrollable static) rather than over raw pixels.