Mastering Apache Airflow! Deploy to Kubernetes in AWS
- 4.6
-
![]()
![]()
![]()
![]()
![]()
Brief Introduction
Learn to programmatically author, schedule and monitor workflows with Apache Airflow. Deploy to Kubernetes in AWS.Description
Apache Airflow is an open-source platform to programmatically author, schedule and monitor workflows. In this course we are going to start with covering some basic concepts related to Apache Airflow - from the main components - web server and scheduler, to the internal components like DAG, Plugin, Operator, Sensor, Hook, Xcom, Variable and Connection.
Later in the course I will teach you some more advanced topics like branching, metrics, performance and log monitoring, and Airflow's REST API. Additionally I will help you to build your development environment with just one click using Docker and Docker Compose.
Why stop here? After all this, we will create a Kubernetes cluster in Amazon and we will deploy our application there!
Finally, I will share with you some useful advanced tips which will be helpful to enhance your simple Airflow project to a production ready system.
Requirements
- Requirements
- Internet connection
- Computer with either MacOS or Linux
- Basic Python knowledge
- A desire to learn
Knowledge
- Advanced tips for production
- Create your first pipeline
- Create ETL pipeline using Pandas
- Build Docker image for Apache Airflow
- Create helm chart for Apache Airflow
- Deploy Airflow to Kubernetes in AWS
- Basic Airflow components - DAG, Plugin, Operator, Sensor, Hook, Xcom, Variable and Connection
- Advance in branching, metrics, performance and log monitoring
- Run development environment with one command through Docker Compose
- Run development environment with one command through Helm and Kubernetes
- The difference between Sequential, Local, Celery and Kubernetes Executors
- Understand Apache Airflow's configuration properties
- Investigate Apache Airflow's REST Api
- Explore Apache Airflow's web interface
Mihail Petkov
4.6 Raiting
Apache Airflow: The Hands-On Guide
- 4.6
-
![]()
![]()
![]()
![]()
![]()
Master Apache Airflow from A to Z. Hands-on videos on Airflow with AWS, Kubernetes, Docker and more It is scalable, dynamic, extensible and modulable....
Mastering Apache Airflow! Deploy to Kubernetes in AWS
- 4.6
-
![]()
![]()
![]()
![]()
![]()
Learn to programmatically author, schedule and monitor workflows with Apache Airflow. Deploy to Kubernetes in AWS. Apache Airflow is an open-source platform to programmatically author, schedule and monitor workflows....
