Deep neural networks (DNNs) are susceptible to adversarial examples – small and imperceptible changes in the natural inputs yet incorrectly classified by models. The most effective defense mechanism against these examples is adversarial training which constructs adversarial examples during training by iterative maximization of loss. The model is then trained to minimize the loss on these constructed examples. This min-max optimization requires more data, larger capacity models, more compute power, and degrades the standard generalization performance of a model. Can we have the robustness more efficiently? In this work, we explore this question from the perspective of knowledge transfer. We first theoretically show the robustness transfer from a robust teacher model to a student model with the help of mixup augmentation. We then propose a novel robustness transfer method called Mixup-Based Activated Channel Maps (MixACM) Transfer. MixACM transfer robustness from a robust teacher to a student by matching activated channel maps generated without expensive adversarial perturbations. Extensive experiments on multiple benchmark datasets show our method can transfer robustness while also improving generalization on natural images.