Unlabeled linear regression, or “linear regression with an unknown permutation”, has attracted increasing attentions due to its applications in linkage record and de-anonymization. However, its computation proves to be cumbersome and all existing algorithms require considerable time in the high dimensional regime. This paper proposes a one-step estimator which are optimal from both the computational and statistical sense. From the computational perspective, our estimator exhibits the same order of computational time as that of the oracle case, where the covariates are known in advance and only the permutation needs recovery. From the statistical perspective, when comparing with the necessary conditions for permutation recovery, our requirement on signal-to-noise ratio () agrees up to Ologlog n difference in certain regimes. Numerical experiments have also been provided to corroborate the above claims.