Augmenting data to improve robustness - a blessing or a curse?