You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "pankajkoti (via GitHub)" <gi...@apache.org> on 2023/07/02 09:48:42 UTC
[GitHub] [airflow] pankajkoti commented on a diff in pull request #32310: Update python.rst with multi-Scheduler LocalExecutor info
pankajkoti commented on code in PR #32310:
URL: https://github.com/apache/airflow/pull/32310#discussion_r1249395981
##########
docs/apache-airflow/core-concepts/executor/local.rst:
##########
@@ -47,3 +47,14 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou
parallelism of just 1 worker, i.e. ``self.parallelism = 1``.
This option could lead to the unification of the executor implementations, running
locally, into just one :class:`~airflow.executors.local_executor.LocalExecutor` with multiple modes.
+
+.. note::
+
+ When using the ``@task`` decorator with multiple Schedulers configured with ``executor = LocalExecutor`` in the [core] section of airflow.cfg, each scheduler will run a LocalExecutor. This means tasks would be processed in a distributed fashion across the machines running the Schedulers.
Review Comment:
Does this apply only when using the @task decorator? Or it is a general behaviour. If general, can we remove the leading part of the sentence which mentions about the case for @task decorator?
##########
docs/apache-airflow/core-concepts/executor/local.rst:
##########
@@ -47,3 +47,14 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou
parallelism of just 1 worker, i.e. ``self.parallelism = 1``.
This option could lead to the unification of the executor implementations, running
locally, into just one :class:`~airflow.executors.local_executor.LocalExecutor` with multiple modes.
+
+.. note::
+
+ When using the ``@task`` decorator with multiple Schedulers configured with ``executor = LocalExecutor`` in the [core] section of airflow.cfg, each scheduler will run a LocalExecutor. This means tasks would be processed in a distributed fashion across the machines running the Schedulers.
Review Comment:
```suggestion
When using the ``@task`` decorator with multiple Schedulers configured with ``executor = LocalExecutor`` in the ``[core]`` section of your ``airflow.cfg``, each scheduler will run a LocalExecutor. This means tasks would be processed in a distributed fashion across the machines running the Schedulers.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org