Consider a collection of datasets generated by unknown interventions on an unknown structural causal model G. Recently, Bengio et al. (2020) support the hypothesis that among all candidate models, G is the \emph{fastest to adapt} from one dataset to another. Intuitively G has less mechanisms to adapt, and their experiments were promising. However their justification was incomplete. Our contribution is a more thorough analysis of this hypothesis. We investigate the adaptation speed of simple two-variable SCMs. Using convergence rates from stochastic optimization we justify that a relevant proxy for adaptation speed is distance in parameter space after intervention. Applying this proxy to categorical variables, we show two results. When the intervention is on the cause variable, the SCM with the correct causal direction is advantaged by a large factor. When the intervention is on the effect variable, we characterize the relative adaptation speed. In particular, we highlight situations where the anticausal model is advantaged, falsifying the initial hypothesis.