In existing machine learning (ML) applications, once a model is built it is deployed to perform its intended task. During the application, the model is fixed due to the closed-world assumption of the classic ML paradigm – everything seen in testing/application must have been seen in training. However, many real-life environments - such as those for chatbots and self-driving cars - are full of unknown, which are called the open environments/worlds. We humans can deal with such environments comfortably - detecting unknowns and learning them continuously in the interaction with other humans and the environment to adapt to the new environment and to become more and more knowledgeable. In fact, we humans never stop learning. After formal education, we continue to learn on the job or while working. AI systems should have the same on-the-job learning capability. It is impossible for them to rely solely on manually labeled data and offline training to deal with the dynamic open world. This talk discusses this problem and presents some initial work in the context of natural language processing.