[jira] [Commented] (AIRFLOW-2027) Only trigger sleep in scheduler after all files have parsed

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430253#comment-16430253 ] 

ASF subversion and git services commented on AIRFLOW-2027:
----------------------------------------------------------

Commit 3c4f1fd9e668a71531ec07d1e7bd60523cb31d32 in incubator-airflow's branch refs/heads/master from [~aoen]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=3c4f1fd ]

[AIRFLOW-2027] Only trigger sleep in scheduler after all files have parsed

Closes #2986 from aoen/ddavydov--open_source_disab
le_unecessary_sleep_in_scheduler_loop


> Only trigger sleep in scheduler after all files have parsed
> -----------------------------------------------------------
>
>                 Key: AIRFLOW-2027
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2027
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: scheduler
>            Reporter: Dan Davydov
>            Assignee: Dan Davydov
>            Priority: Major
>             Fix For: 1.10.0
>
>
> The scheduler loop sleeps for 1 second every loop unnecessarily. Remove this sleep to slightly speed up scheduling, and instead do it once all files have been parsed. It can add up since it runs to every scheduler loop which runs # of dags to parse/scheduler parallelism times.
> Also remove the unnecessary increased file processing interval in tests which slows them down.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)