You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2019/09/18 15:47:00 UTC
[jira] [Commented] (AIRFLOW-5210) Resolving Template Files for
large DAGs hurts performance
[ https://issues.apache.org/jira/browse/AIRFLOW-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932578#comment-16932578 ]
ASF subversion and git services commented on AIRFLOW-5210:
----------------------------------------------------------
Commit 85ac7f748f13cb71ba989acb6800dbcea383541e in airflow's branch refs/heads/v1-10-test from Daniel Frank
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=85ac7f7 ]
[AIRFLOW-5210] Make finding template files more efficient (#5815)
For large DAGs, iterating over template fields to find template files can be time intensive.
Save this time for tasks that do not specify a template file extension.
(cherry picked from commit eeac82318a6440b2d65f9a35b3437b91813945f4)
> Resolving Template Files for large DAGs hurts performance
> ----------------------------------------------------------
>
> Key: AIRFLOW-5210
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5210
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.4
> Reporter: Daniel Frank
> Priority: Major
> Fix For: 1.10.6
>
>
> During task execution, "resolve_template_files" runs for all tasks in a given DAG. For large DAGs this takes a long time and is not necessary for tasks that do not use the template_ext field
--
This message was sent by Atlassian Jira
(v8.3.4#803005)