Whether you are a data scientist or a machine learning developer, you have probably encountered the following scenario: you have a new bright and shiny machine-learning model that you validated with a test-set and it produces great results. Yay! After the successful deployment, users start complaining about the performance of this new model and you wonder - what went wrong? In this talk, we will map 3 major reasons to explain why machine learning models perform so much better on the test set compared to production and propose ways to avoid this situation.