Cart-Pole LQR

Key Insight

The cart-pole is nonlinear, but near the upright balance point it behaves almost like a simple linear system, and the Linear-Quadratic Regulator (LQR) is the optimal feedback controller for exactly that case — it picks the control that minimizes a weighted sum of how far the state strays and how much force you spend. The trick is linearization: you replace the true dynamics with their state-space approximation ẋ = Ax + Bu taken right at the upright setpoint, solve once for a constant gain matrix K, and apply u = -Kx forever after. Because that approximation only holds nearby, the controller has a basin of attraction — the region of starting tilts and speeds it can actually recover from — and pushing past its edge is where you watch the linear model's promises break and the pole topple.