While the concept of using an open source in your business is old news, integrating several of them for the purpose of moving huge data sets in real time from hundreds of databases - might sound crazy. It was a real roller coaster ride - integrating and relaying on 5 open sources (Debezium, docker, kafka connect, kafka, Kubernetes) as the major part of the application code, especially when the main source, Debezium, didn’t even celebrate his first birthday. However, it works like a charm.

In this talk, I will show you how to combine few reliable open source technologies, get some help from their community, and after a month of no sleep, get the project running on massive scale successfully. By “massive” I mean 600 different data sources and millions of events per second.