You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/15 17:47:11 UTC
[GitHub] [airflow] pingzh commented on a change in pull request #21501: Support different timeout value for dag file parsing
pingzh commented on a change in pull request #21501:
URL: https://github.com/apache/airflow/pull/21501#discussion_r807133702
##########
File path: docs/apache-airflow/faq.rst
##########
@@ -119,6 +119,30 @@ How do I trigger tasks based on another task's failure?
You can achieve this with :ref:`concepts:trigger-rules`.
+How to control DAG file parsing timeout for different DAG files?
+---------------------------------------------------------------------------------
+(only valid for Airflow >= 2.3.0)
+
+You can add a function ``get_dagbag_import_timeout`` in the ``airflow_local_settings.py``. This function gets
+called right before a DAG file is parsed. You can return different timeout value based on the DAG file.
+When the return value is less than or equal to 0, it means no timeout during the DAG parsing.
+
+.. code-block:: python
+ :caption: airflow_local_settings.py
+ :name: airflow_local_settings.py
+
+ def get_dagbag_import_timeout(dag_file_path: str) -> Union[int, float]:
+ """
+ This setting allows to dynamically control the DAG file parsing timeout.
+
+ It is useful when there are a few DAG files requiring longer parsing times, while others do not.
+ You can control them separately instead of having one value for all DAG files.
+
+ If the return value is less than or equal to 0, it means no timeout during the DAG parsing.
+ """
Review comment:
good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org