Recent work suggests that overparameterized autoencoders can be trained to implement associative memory via iterative maps. This phenomenon happens when converged input-output Jacobian of the network has all eigenvalue norms strictly below one. In this work, we theoretically analyze this behavior for sigmoid networks by leveraging recent developments in deep learning theories, especially the Neural Tangent Kernel (NTK) theory. We find that overparameterized sigmoid autoencoders can have attracto…