The research community is gradually coming to a realization that policies trained arcade-like video games are very limited. They overfit badly and are not going to take us far along the way to some sort of general intelligence. This should perhaps not be surprising, given that such games generally have tightly defined tasks, fixed perspectives, and generally static worlds. More and more attention is therefore given to games that are in some sense open-ended or feature open worlds. Could such games be the solution to our problems, allowing the development of more general artificial intelligence? Perhaps, but basing competitions or benchmarks on open-ended games is not going to be easy, as the very features which make for a good benchmark are the same that lead to brittle policies. Shoe-horning open-world games into a standard RL framework is unlikely to be the best option for going forward. Many of the most interesting opportunities for developing intelligent behavior is likely to come from agents constructing their own challenges and environments. The boundary between playing a game and constructing a world is not well-defined: I will give examples from where the same RL setup was used to play SimCity and to develop game levels. I will also briefly introduce the Generative Design in Minecraft Competition, which focuses on building believable settlements.