Learning to Build High-fidelity and Robust Environment Models