Owing to the susceptibility of deep learning systems to adversarial attacks, there has been a great deal of work in developing (both empirically and certifiably) robust classifiers. While most work has defended against a single type of attack, recent work has looked at defending against multiple threat models using simple aggregations of multiple attacks. However, these methods can be difficult to tune, and can easily result in imbalanced degrees of robustness to individual threat models, resulting in a sub-optimal worst-case loss over the combined threat model. In this work, we develop a natural generalization of the standard PGD-based procedure to incorporate multiple threat models into a single attack, by taking the worst-case over all steepest descent directions. This approach has the advantage of directly converging upon a trade-off between different threat models which minimizes the worst-case performance over the union. With this approach, we are able to train standard architectures which are simultaneously robust against l_∞, l_2, and l_1 attacks, outperforming past approaches on the MNIST and CIFAR10 datasets and achieving adversarial accuracy of 46.1