You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ph...@apache.org on 2024/08/01 08:21:03 UTC

(airflow) branch main updated: docs(dataset): add more detail on when dataset alias is resolved (#41152)

This is an automated email from the ASF dual-hosted git repository.

phanikumv pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 19744420c4 docs(dataset): add more detail on when dataset alias is resolved (#41152)
19744420c4 is described below

commit 19744420c43fb207e70dd34aaf2cdb589847a428
Author: Wei Lee <we...@gmail.com>
AuthorDate: Thu Aug 1 16:20:56 2024 +0800

    docs(dataset): add more detail on when dataset alias is resolved (#41152)
---
 docs/apache-airflow/authoring-and-scheduling/datasets.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/apache-airflow/authoring-and-scheduling/datasets.rst b/docs/apache-airflow/authoring-and-scheduling/datasets.rst
index 53b4ad38cd..b424324c8f 100644
--- a/docs/apache-airflow/authoring-and-scheduling/datasets.rst
+++ b/docs/apache-airflow/authoring-and-scheduling/datasets.rst
@@ -472,6 +472,8 @@ Scheduling based on dataset aliases
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Since dataset events added to an alias are just simple dataset events, a downstream depending on the actual dataset can read dataset events of it normally, without considering the associated aliases. A downstream can also depend on a dataset alias. The authoring syntax is referencing the ``DatasetAlias`` by name, and the associated dataset events are picked up for scheduling. Note that a DAG can be triggered by a task with ``outlets=DatasetAlias("xxx")`` if and only if the alias is resol [...]
 
+The dataset alias is resolved to the datasets during DAG parsing. Thus, if the "min_file_process_interval" configuration is set to a high value, there is a possibility that the dataset alias may not be resolved. To resolve this issue, you can trigger DAG parsing.
+
 .. code-block:: python
 
     with DAG(dag_id="dataset-producer"):