[jira] [Created] (AIRFLOW-3775) Support for custom task scheduler in airflow, apart of time based [default] one.

["Abhishek Sharma (JIRA)" <ji...@apache.org>]

Abhishek Sharma created AIRFLOW-3775:
----------------------------------------

             Summary: Support for custom task scheduler in airflow, apart of time based [default] one.
                 Key: AIRFLOW-3775
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3775
             Project: Apache Airflow
          Issue Type: Wish
          Components: scheduler
            Reporter: Abhishek Sharma


Currently, airflow scheduler picks DAGs for execution based on cron string mentioned in the schedule. 

I have a requirement in which I want to trigger the DAGs based on the event. Currently, I am managing such requirement by running an application which triggers DAGs when an event occurs.  DAGs schedule with such requirement is NONE and concurrency is 1. 

Airflow considers all runs as manual triggers, and Airflow UI is messed up for that. Also, multiple applications required for triggering DAGs based on different condition.

I propose to have to multiple schedulers support in Airflow, and like queue feature at a task level, we should have the scheduler option to at the task level, which by default would be time-based however on providing specific value will be picked by that instance of the scheduler.  

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)