Full session (30 minutes)
Engineering
Airflow
GCP

Processing data is the bread-and-butter of most if not all technology companies. Plenty of tools, conventions, and terminologies were developed around this topic in the last 10 years but looks like a new winner, a tool that is here to stay surfaced: Apache Airflow. In this short lecture, I'll be talking about what it is but will mostly present what I believe is the ultimate setup and approach to running Airflow in production - utilizing the power of containers and Kubernetes. The lecture will contain practical examples and additional goodies that we at MyndYou learned from using it in production.

Arik Liber