This work characterizes the effect of depth on the optimization landscape of linear regression, showing that, despite their nonconvexity, deeper models have more desirable optimization landscapes. We consider a robust and over-parameterized setting, where a subset of measurements are grossly corrupted with noise, and the true linear model is captured via an N-layer linear neural network. On the negative side, we show that this problem does not have a benign landscape: given any N≥ 1, with consta…