[jira] [Comment Edited] (AIRFLOW-3775) Support for custom task scheduler in airflow, apart of time based [default] one.

["Abhishek Sharma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755017#comment-16755017 ] 

Abhishek Sharma edited comment on AIRFLOW-3775 at 1/29/19 1:36 PM:
-------------------------------------------------------------------

[~ashb] attached are the two screenshots.
 * First one shows missing 'Last Run' information in the DAGs listing page, which I think is a relevant piece of information to determine when was your DAG run last time.
 * The second one shows a real mess where you have to browse all the way to the right to get the logs and information of your last run. DAGs with NONE schedule and triggered externally is considered as  'manual_trigger'  and it not gets coalesce into 25 or 50 limits. 

However, my intent with this Jira is for asking the support of custom scheduler in Airflow. The logic of the custom scheduler is user based, and tasks will get scheduled by the perspective scheduler.


was (Author: abhioncbr):
[~ashb] attached are the two screenshots.
 * First one shows missing 'Last Run' information in the DAGs listing page, which I think is a relevant piece of information to determine when was you DAG run last time.
 * The second one shows a real mess where you have to browse all the way to the right to get the logs and information of your last run. DAGs with NONE schedule and triggered externally consider as  'manual_trigger'  not gets coalesce into 25 or 50 limits. 

However, my intent with this Jira is for asking the support of custom scheduler in Airflow. The logic of the custom scheduler is user based, and tasks will get scheduled by the perspective scheduler.

> Support for custom task scheduler in airflow, apart of time based [default] one.
> --------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-3775
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3775
>             Project: Apache Airflow
>          Issue Type: Wish
>          Components: scheduler
>            Reporter: Abhishek Sharma
>            Priority: Major
>         Attachments: Screenshot1.png, screenshot2.png
>
>
> Currently, airflow scheduler picks DAGs for execution based on cron string mentioned in the schedule. 
> I have a requirement in which I want to trigger the DAGs based on the event. Currently, I am managing such requirement by running an application which triggers DAGs when an event occurs.  DAGs schedule with such requirement is NONE and concurrency is 1. 
> Airflow considers all runs as manual triggers, and Airflow UI is messed up for that. Also, multiple applications required for triggering DAGs based on different condition.
> I propose to have to multiple schedulers support in Airflow, and like queue feature at a task level, we should have the scheduler option to at the task level, which by default would be time-based however on providing specific value will be picked by that instance of the scheduler.  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)