Aerofoil is an Apache Airflow plugin. It provides some very useful extensions, that are missing in the Airflow. The plugin is available with MIT license, the intention is to keep it simple and friendly.

A convenient way to run Backfill DAG from UI. In most of the enterprise setup, DAG developers do not have access to Airflow servers to be able to run backfill jobs. This makes running backfill an expensive operation. With the Backfilll UI, Airflow users can submit backfill command from UI. The backfill itself runs as a Airflow DAG and is compatible with most production Airflow configurations. This is also the most used feature in Aerofoil.

Large Airflow installations can have hundreds of DAGs. This feature allows logged in user to view to see the DAG's authored by her/him. The DAG Author must be set correctly in the DAG file and it must match with the Airflow user name.

Sometime complete DAG history needs to be reset. There is currently no way of clearing a DAGs history in Airflow. This functionality allows clearing/resetting DAGs history via a convenient UI.

A shortcut to mark your DAG/Task runs successful. The Fake success is a powerful featurs, and to avoid misuse and Audit Trail is maintained. Adminstratos can always find out, who is trying to fake it.

Context aware BashOperator, that can take a context and pass it to callback e.g. on_success_callback, on_failure_callback etc.

• Working Airflow installation.
• works with any executor other than Sequential Executor or LocalExecutor (e.g. KubernetasExecutor, CeleryExecutor)
• Requires a non-sqlite database in the backend.

• pip install apache-airflow-providers-aerofoil
• Add rquired tables to Airflow Metadata DB by executing src/aerofoil/models.sql
• Deploy DAG file in the src/aerofoil/aerofoil_backfill.py DAGs to your Airflow (usually by putting the DAG in Airflow's DAGs folder.
• Make sure the aerofoil_backfill DAG is enabled.

When the user submits a backfill, it creates an entry in the aerofoil_backfill backfill table. aerofoil_backfill DAG is scheduled to run every minute (which can be changed, by modifying the DAG). The Sesor in the DAG, picks up the entries in the DB and create dynamic task for each entry. The Backfill job itself runs as a Bash command in the executor. This provides a flexible and executor agnostic design.

If you find something missing and would like to contrinute, feel free to raise a pull request. If you find a bug and would like me to fix it, please feel free to raise an issue.