Neural networks are widely used for processing time series data, yet such models often ignore the underlying physical structures in the input measurements. Recently Koopman-based models have been suggested, as a promising alternative to recurrent neural networks, for forecasting complex high-dimensional dynamical systems. We propose a novel Consistent Koopman Autoencoder that exploits the forward and backward dynamics to achieve long time predictions. Key to our approach is a new analysis where we unravel the interplay between invertible dynamics and their associated Koopman operators. Our architecture and loss function are interpretable from a physical viewpoint, and the computational requirements are comparable to other baselines. We evaluate the proposed algorithm on a wide range of high-dimensional problems, from simple canonical systems such as linear and nonlinear oscillators, to complex ocean dynamics and fluid flows on a curved domain. Overall, our results show that our model yields accurate estimates for significant prediction horizons, while being robust to noise in the input data.