Taboola’s R&D had expanded from ~50 developers to 250 in the past two years. With a fast and effective CICD pipeline we are rolling out releases on a daily basis. Because multiple developers commit new features to the same code base, we had to have a smart error detection tool to prevent a bad merge from happening. This led us to develop an in house Error Detector Mechanism, which is sensitive to the nature of our data.
A development error can be DC specific, and peaks of errors can appear in certain rush hours of the day, certain days of the week, etc. Taking into account these factors, our constantly updating class-specific baselines can detect errors fast, with zero effect on performance, saving money and post mortem analysis effort.
We invite you to see how it is done at Taboola.