For a while the ML engineering focus was around the Data Scientist needs, libs like scikit-learn, TenserFlow and more, serves that need. The industry started to change the focus and put it more on the post-DS phases, we can see big companies like Uber developing internal platforms like Michelangelo to address these needs, but not enough mature open sources are out there yet. That gave us @Fundbox the motivation to develop a ML Monitoring platform. In this talk I would like to walk you thru the product architecture, decisions we took on the way and the end solution. We will talk about the differences between stability and performance tests, what is a "concept drifting" and what kind of widgets we developed to reflect our models quality in production.