Deep neural networks often fail catastrophically because they rely on spurious features. Most prior work assumes a clear dichotomy into spurious and non-spurious features; however, we argue that this dichotomy is often unrealistic. For example, most of the time we do not want an autonomous car to use the speed of the car in the next lane to determine its own speed—we don't want our car to run a red light if a neighboring car does so. However, we cannot simply call next lane speed a spurious feat…