[airflow] 10/23: Add docs about ``.airflowignore`` (#20311)

[je...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

jedcunningham pushed a commit to branch v2-2-test
in repository https://gitbox.apache.org/repos/asf/airflow.git

commit 1a7f94389274a689590c718d1e4ae800b50bbfa9
Author: Matt Rixman <58...@users.noreply.github.com>
AuthorDate: Wed Dec 15 08:13:19 2021 -0700

    Add docs about ``.airflowignore`` (#20311)
    
    This was deleted  in an [earlier refactor](https://github.com/apache/airflow/pull/15444/files) (see `concepts.rst`).  This PR brings it back.  I added it under the "DAGs" section because even though it's file-based and not dag-based, excluding files that define dags is the most likely use case for this feature (I think).
    
    (cherry picked from commit 6eac2e0807a8be5f39178f079db28ebcd2f83621)
---
 docs/apache-airflow/concepts/dags.rst | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/docs/apache-airflow/concepts/dags.rst b/docs/apache-airflow/concepts/dags.rst
index b871895..83d1cbd 100644
--- a/docs/apache-airflow/concepts/dags.rst
+++ b/docs/apache-airflow/concepts/dags.rst
@@ -616,6 +616,35 @@ Note that packaged DAGs come with some caveats:
 
 In general, if you have a complex set of compiled dependencies and modules, you are likely better off using the Python ``virtualenv`` system and installing the necessary packages on your target systems with ``pip``.
 
+``.airflowignore``
+------------------
+
+A ``.airflowignore`` file specifies the directories or files in ``DAG_FOLDER``
+or ``PLUGINS_FOLDER`` that Airflow should intentionally ignore.
+Each line in ``.airflowignore`` specifies a regular expression pattern,
+and directories or files whose names (not DAG id) match any of the patterns
+would be ignored (under the hood, ``Pattern.search()`` is used to match the pattern).
+Overall it works like a ``.gitignore`` file.
+Use the ``#`` character to indicate a comment; all characters
+on a line following a ``#`` will be ignored.
+
+``.airflowignore`` file should be put in your ``DAG_FOLDER``.
+For example, you can prepare a ``.airflowignore`` file with content
+
+.. code-block::
+
+    project_a
+    tenant_[\d]
+
+Then files like ``project_a_dag_1.py``, ``TESTING_project_a.py``, ``tenant_1.py``,
+``project_a/dag_1.py``, and ``tenant_1/dag_1.py`` in your ``DAG_FOLDER`` would be ignored
+(If a directory's name matches any of the patterns, this directory and all its subfolders
+would not be scanned by Airflow at all. This improves efficiency of DAG finding).
+
+The scope of a ``.airflowignore`` file is the directory it is in plus all its subfolders.
+You can also prepare ``.airflowignore`` file for a subfolder in ``DAG_FOLDER`` and it
+would only be applicable for that subfolder.
+
 DAG Dependencies
 ================