You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/08/14 02:27:00 UTC
[jira] [Commented] (AIRFLOW-5210) Resolving Template Files for
large DAGs hurts performance
[ https://issues.apache.org/jira/browse/AIRFLOW-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906809#comment-16906809 ]
ASF GitHub Bot commented on AIRFLOW-5210:
-----------------------------------------
danfrankj commented on pull request #5815: [AIRFLOW-5210] Make finding template files more efficient
URL: https://github.com/apache/airflow/pull/5815
Make sure you have checked _all_ steps below.
### Jira
https://issues.apache.org/jira/browse/AIRFLOW-5210
### Description
For large DAGs (1000s of tasks) iterating over template_fields adds significant time to task execution and is not necessary for tasks that do not specify template_ext
### Tests
I can add tests if you think necessary, but this is a _very_ simple change
### Code Quality
- [ ] Passes `flake8`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Resolving Template Files for large DAGs hurts performance
> ----------------------------------------------------------
>
> Key: AIRFLOW-5210
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5210
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.4
> Reporter: Daniel Frank
> Priority: Major
>
> During task execution, "resolve_template_files" runs for all tasks in a given DAG. For large DAGs this takes a long time and is not necessary for tasks that do not use the template_ext field
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)