You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2023/07/06 22:27:39 UTC
[airflow] 26/42: Update python.rst with multi-Scheduler LocalExecutor info (#32310)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-6-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 288fb12e836785f87f7bb4e6bc6ea24945aa3b37
Author: Sai Pragna Etikyala <pr...@gmail.com>
AuthorDate: Wed Jul 5 02:22:30 2023 -0700
Update python.rst with multi-Scheduler LocalExecutor info (#32310)
Co-authored-by: Pankaj Koti <pa...@gmail.com>
(cherry picked from commit e53320d62030a53c6ffe896434bcf0fc85803f31)
---
docs/apache-airflow/core-concepts/executor/local.rst | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/docs/apache-airflow/core-concepts/executor/local.rst b/docs/apache-airflow/core-concepts/executor/local.rst
index 256c612553..94a3f51211 100644
--- a/docs/apache-airflow/core-concepts/executor/local.rst
+++ b/docs/apache-airflow/core-concepts/executor/local.rst
@@ -47,3 +47,11 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou
parallelism of just 1 worker, i.e. ``self.parallelism = 1``.
This option could lead to the unification of the executor implementations, running
locally, into just one :class:`~airflow.executors.local_executor.LocalExecutor` with multiple modes.
+
+.. note::
+
+ When multiple Schedulers are configured with ``executor = LocalExecutor`` in the ``[core]`` section of your ``airflow.cfg``, each Scheduler will run a LocalExecutor. This means tasks would be processed in a distributed fashion across the machines running the Schedulers.
+
+ One consideration should be taken into account:
+
+ - Restarting a Scheduler: If a Scheduler is restarted, it may take some time for other Schedulers to recognize the orphaned tasks and restart or fail them.