You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Ash Berlin-Taylor (JIRA)" <ji...@apache.org> on 2019/01/29 11:15:00 UTC
[jira] [Commented] (AIRFLOW-3775) Support for custom task scheduler
in airflow, apart of time based [default] one.
[ https://issues.apache.org/jira/browse/AIRFLOW-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754886#comment-16754886 ]
Ash Berlin-Taylor commented on AIRFLOW-3775:
--------------------------------------------
How does the UI mess up with manual triggers?
> Support for custom task scheduler in airflow, apart of time based [default] one.
> --------------------------------------------------------------------------------
>
> Key: AIRFLOW-3775
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3775
> Project: Apache Airflow
> Issue Type: Wish
> Components: scheduler
> Reporter: Abhishek Sharma
> Priority: Major
>
> Currently, airflow scheduler picks DAGs for execution based on cron string mentioned in the schedule.
> I have a requirement in which I want to trigger the DAGs based on the event. Currently, I am managing such requirement by running an application which triggers DAGs when an event occurs. DAGs schedule with such requirement is NONE and concurrency is 1.
> Airflow considers all runs as manual triggers, and Airflow UI is messed up for that. Also, multiple applications required for triggering DAGs based on different condition.
> I propose to have to multiple schedulers support in Airflow, and like queue feature at a task level, we should have the scheduler option to at the task level, which by default would be time-based however on providing specific value will be picked by that instance of the scheduler.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)