At CEAi, we don’t like repetitive or tedious work. On the other hand, we like to build cool stuff that simplifies our lives. That's why we decided to get rid of some pain points that are pervasive in machine learning. “I trained a model 6 months ago … which data did I use again? What parameters did I try?” Don’t tell my boss but let’s be honest: usually, I can’t remember. Hence my motivation to talk about our effort to build a system that facilitates reproducibility, automates validation, testing, auditing and continuously searches for the next best model. In addition to this, the system will use active learning strategies to shorten the total time to model deployment and reduce load on annotator pools.