We present Multi-Scale Sparse Voxel Convolution (MS-SVConv), a fast multi-scale deep neural network that outputs the descriptors from point clouds for 3D registration between two scenes. We compute descriptors using a 3D sparse voxel convolutional network on a point cloud at different scales and then fuse the descriptors through fully-connected layers. With supervised learning, we show significant improvements compared to state-of-the-art methods on the competitive and well-known 3DMatch benchmark. We also achieve better generalization through different source and target datasets, with very fast computation. Finally, we present a strategy for transferring MS-SVConv on unknown datasets in a unsupervised way called UDGE, which leads to state-of-the-art results on the ETH and TUM datasets. [The code will be publicly available before publication].