You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2025/03/29 08:23:06 UTC
[PR] Nuking SequentialExecutor from the in-tree executors [airflow]
amoghrajesh opened a new pull request, #48528:
URL: https://github.com/apache/airflow/pull/48528
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
closes: #48522
Let me do the honors!
Tried generating the image using AI, the spellings is what is hallucinates with! Anyway, worthwhile to add i'd say!
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2768759353
Thanks for the reviews folks, merging this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2766763852
> Besides the leftovers if you search for "SequentialExecutor" (and also there are some hits when you search for "Sequential Executor" with space!) and some CI fails I think this is looking pretty good. If you lag access to a PC, let me know if I can help. Would be cool to remove before RC0.
I have access now, thanks @jscheffl. Just looking at the CI and missing occurences now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2766795678
> Besides the leftovers if you search for "SequentialExecutor" (and also there are some hits when you search for "Sequential Executor" with space!) and some CI fails I think this is looking pretty good. If you lag access to a PC, let me know if I can help. Would be cool to remove before RC0.
@jscheffl it has been handled now 🗡️
Thanks for finding that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2022127278
##########
airflow-core/docs/core-concepts/executor/index.rst:
##########
@@ -46,14 +46,17 @@ If you want to check which executor is currently set, you can use the ``airflow
.. code-block:: bash
$ airflow config get-value core executor
- SequentialExecutor
+ LocalExecutor
Executor Types
--------------
-There are two types of executors - those that run tasks *locally* (inside the ``scheduler`` process), and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``SequentialExecutor`` by default, which is a local executor, and the simplest option for execution. However, the ``SequentialExecutor`` is not suitable for production since it does not allow for parallel task running and due to that, some Airflow features (e.g. running sensors) will not work properly. You should instead use the ``LocalExecutor`` for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
+There is only one type of executor that runs tasks *locally* (inside the ``scheduler`` process),
+and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``LocalExecutor`` by default, which is a local executor, and the simplest option for execution.
Review Comment:
Just updated it to be more explicit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021365816
##########
airflow-core/docs/core-concepts/executor/index.rst:
##########
@@ -46,14 +46,17 @@ If you want to check which executor is currently set, you can use the ``airflow
.. code-block:: bash
$ airflow config get-value core executor
- SequentialExecutor
+ LocalExecutor
Executor Types
--------------
-There are two types of executors - those that run tasks *locally* (inside the ``scheduler`` process), and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``SequentialExecutor`` by default, which is a local executor, and the simplest option for execution. However, the ``SequentialExecutor`` is not suitable for production since it does not allow for parallel task running and due to that, some Airflow features (e.g. running sensors) will not work properly. You should instead use the ``LocalExecutor`` for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
+There is only one type of executor that runs tasks *locally* (inside the ``scheduler`` process),
+and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``LocalExecutor`` by default, which is a local executor, and the simplest option for execution.
+However, the ``LocalExecutor`` is not advised for production use since it runs processes in the scheduler process and that can have an impact on the performance of the scheduler. You can use the ``LocalExecutor``
+for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
Review Comment:
Yeah more accurate!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2763341190
I know the failures here, but unfortunately wont have access to computer till tomorrow evening. I will fix those then..
But its ready for review id say 😀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021604703
##########
airflow-core/docs/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -172,7 +172,7 @@ Serving logs from workers and triggerer
Most task handlers send logs upon completion of a task. In order to view logs in real time, Airflow starts an HTTP server to serve the logs in the following cases:
-- If ``SequentialExecutor`` or ``LocalExecutor`` is used, then when ``airflow scheduler`` is running.
+- If ``LocalExecutor`` is used, then when ``airflow scheduler`` is running.
- If ``CeleryExecutor`` is used, then when ``airflow worker`` is running.
Review Comment:
not related to this PR but what does the sentence even mean:
maybe improve it in a follow-up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2019835840
##########
airflow-core/docs/core-concepts/executor/index.rst:
##########
@@ -46,14 +46,17 @@ If you want to check which executor is currently set, you can use the ``airflow
.. code-block:: bash
$ airflow config get-value core executor
- SequentialExecutor
+ LocalExecutor
Executor Types
--------------
-There are two types of executors - those that run tasks *locally* (inside the ``scheduler`` process), and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``SequentialExecutor`` by default, which is a local executor, and the simplest option for execution. However, the ``SequentialExecutor`` is not suitable for production since it does not allow for parallel task running and due to that, some Airflow features (e.g. running sensors) will not work properly. You should instead use the ``LocalExecutor`` for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
+There is only one type of executor that runs tasks *locally* (inside the ``scheduler`` process),
+and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``LocalExecutor`` by default, which is a local executor, and the simplest option for execution.
+However, the ``LocalExecutor`` is not advised for production use since it runs processes in the scheduler process and that can have an impact on the performance of the scheduler. You can use the ``LocalExecutor``
+for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
Review Comment:
That is not correct, LocalExecutor _can_ be used and I even saw multiple schedulers each with a LocalExecutor in the wild.
```suggestion
However, as the ``LocalExecutor`` runs processes in the scheduler process that can have an impact on the performance of the scheduler. You can use the ``LocalExecutor``
for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2764300081
Was https://github.com/apache/airflow/discussions/44243 resolved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2022123609
##########
airflow-core/docs/administration-and-deployment/production-deployment.rst:
##########
@@ -56,11 +56,8 @@ Once that is done, you can run -
Multi-Node Cluster
==================
-Airflow uses :class:`~airflow.executors.sequential_executor.SequentialExecutor` by default. However, by its
-nature, the user is limited to executing at most one task at a time. ``Sequential Executor`` also pauses
-the scheduler when it runs a task, hence it is not recommended in a production setup. You should use the
-:class:`~airflow.executors.local_executor.LocalExecutor` for a single machine.
-For a multi-node setup, you should use the :doc:`Kubernetes executor <apache-airflow-providers-cncf-kubernetes:kubernetes_executor>` or
+Airflow uses :class:`~airflow.executors.local_executor.LocalExecutor` by default. For a multi-node setup,
+you should use the :doc:`Kubernetes executor <apache-airflow-providers-cncf-kubernetes:kubernetes_executor>` or
the :doc:`Celery executor <apache-airflow-providers-celery:celery_executor>`.
Review Comment:
Yeah maybe! Just created this so we do not forget it: https://github.com/apache/airflow/issues/48605. Feel free to grab it @o-nikolas
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2766772800
> Was #44243 resolved?
Looks like https://github.com/apache/airflow/discussions/44243#discussioncomment-12139153 is a solution that works. Although i'd want someone to test it again if possible, I know its important but we've had more successes than not in that migration. LocalExec has been tested plenty :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2763658546
Besides the leftovers if you search for "SequentialExecutor" (and also there are some hits when you search for "Sequential Executor" with space!) and some CI fails I think this is looking pretty good. If you lag access to a PC, let me know if I can help. Would be cool to remove before RC0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021554407
##########
airflow-core/docs/howto/set-up-database.rst:
##########
@@ -72,8 +72,8 @@ Setting up a SQLite Database
----------------------------
SQLite database can be used to run Airflow for development purpose as it does not require any database server
-(the database is stored in a local file). There are many limitations of using the SQLite database (for example
Review Comment:
Not needed in my view. Can be concurrent as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #48528:
URL: https://github.com/apache/airflow/pull/48528#issuecomment-2763232334
Guess this is what's left as of now:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh merged PR #48528:
URL: https://github.com/apache/airflow/pull/48528
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021448704
##########
airflow-core/docs/administration-and-deployment/production-deployment.rst:
##########
@@ -56,11 +56,8 @@ Once that is done, you can run -
Multi-Node Cluster
==================
-Airflow uses :class:`~airflow.executors.sequential_executor.SequentialExecutor` by default. However, by its
-nature, the user is limited to executing at most one task at a time. ``Sequential Executor`` also pauses
-the scheduler when it runs a task, hence it is not recommended in a production setup. You should use the
-:class:`~airflow.executors.local_executor.LocalExecutor` for a single machine.
-For a multi-node setup, you should use the :doc:`Kubernetes executor <apache-airflow-providers-cncf-kubernetes:kubernetes_executor>` or
+Airflow uses :class:`~airflow.executors.local_executor.LocalExecutor` by default. For a multi-node setup,
+you should use the :doc:`Kubernetes executor <apache-airflow-providers-cncf-kubernetes:kubernetes_executor>` or
the :doc:`Celery executor <apache-airflow-providers-celery:celery_executor>`.
Review Comment:
We have many more executors since this was written, I wonder if we should update that. Out of scope for this PR though, just noting it as I thought of it!
##########
airflow-core/docs/core-concepts/executor/index.rst:
##########
@@ -46,14 +46,17 @@ If you want to check which executor is currently set, you can use the ``airflow
.. code-block:: bash
$ airflow config get-value core executor
- SequentialExecutor
+ LocalExecutor
Executor Types
--------------
-There are two types of executors - those that run tasks *locally* (inside the ``scheduler`` process), and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``SequentialExecutor`` by default, which is a local executor, and the simplest option for execution. However, the ``SequentialExecutor`` is not suitable for production since it does not allow for parallel task running and due to that, some Airflow features (e.g. running sensors) will not work properly. You should instead use the ``LocalExecutor`` for small, single-machine production installations, or one of the remote executors for a multi-machine/cloud installation.
+There is only one type of executor that runs tasks *locally* (inside the ``scheduler`` process),
+and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``LocalExecutor`` by default, which is a local executor, and the simplest option for execution.
Review Comment:
There is only one "core" executor that does. But anyone can write an executor using the public interface and many of those could be local. I don't see the reason to specifically call out only one here, I think it's more misleading than it is helpful?
```suggestion
There are two types of executors - those that run tasks *locally* (inside the ``scheduler`` process),
and those that run their tasks *remotely* (usually via a pool of *workers*). Airflow comes configured with the ``LocalExecutor`` by default, which is a local executor, and the simplest option for execution.
```
##########
airflow-core/src/airflow/cli/commands/standalone_command.py:
##########
@@ -174,12 +174,8 @@ def calculate_env(self):
# Make sure we're using a local executor flavour
executor_class, _ = ExecutorLoader.import_default_executor_cls()
if not executor_class.is_local:
- if "sqlite" in conf.get("database", "sql_alchemy_conn"):
- self.print_output("standalone", "Forcing executor to SequentialExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.SEQUENTIAL_EXECUTOR
- else:
- self.print_output("standalone", "Forcing executor to LocalExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
+ self.print_output("standalone", "Forcing executor to LocalExecutor")
+ env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
Review Comment:
Don't we want to still check for sqlite and disable parallelism if so?
##########
airflow-core/docs/howto/set-up-database.rst:
##########
@@ -72,8 +72,8 @@ Setting up a SQLite Database
----------------------------
SQLite database can be used to run Airflow for development purpose as it does not require any database server
-(the database is stored in a local file). There are many limitations of using the SQLite database (for example
Review Comment:
We should provide guidance here for LocalExecutor, parallelism must be set to 1 right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021553920
##########
airflow-core/src/airflow/cli/commands/standalone_command.py:
##########
@@ -174,12 +174,8 @@ def calculate_env(self):
# Make sure we're using a local executor flavour
executor_class, _ = ExecutorLoader.import_default_executor_cls()
if not executor_class.is_local:
- if "sqlite" in conf.get("database", "sql_alchemy_conn"):
- self.print_output("standalone", "Forcing executor to SequentialExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.SEQUENTIAL_EXECUTOR
- else:
- self.print_output("standalone", "Forcing executor to LocalExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
+ self.print_output("standalone", "Forcing executor to LocalExecutor")
+ env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
Review Comment:
No, with current setup and TaskSDK there is no problem with parallelism anymore. Tested locally (LocalExecutor with SQLite) and was running very well.
Especially also because nowadays SQLite allows concurrent access via a lock file which was not possible in the past.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2021579938
##########
airflow-core/src/airflow/cli/commands/standalone_command.py:
##########
@@ -174,12 +174,8 @@ def calculate_env(self):
# Make sure we're using a local executor flavour
executor_class, _ = ExecutorLoader.import_default_executor_cls()
if not executor_class.is_local:
- if "sqlite" in conf.get("database", "sql_alchemy_conn"):
- self.print_output("standalone", "Forcing executor to SequentialExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.SEQUENTIAL_EXECUTOR
- else:
- self.print_output("standalone", "Forcing executor to LocalExecutor")
- env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
+ self.print_output("standalone", "Forcing executor to LocalExecutor")
+ env["AIRFLOW__CORE__EXECUTOR"] = executor_constants.LOCAL_EXECUTOR
Review Comment:
Ah, nice, that's a great side-effect of task sdk :smiley:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Nuking SequentialExecutor from the in-tree executors [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #48528:
URL: https://github.com/apache/airflow/pull/48528#discussion_r2022128019
##########
airflow-core/docs/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -172,7 +172,7 @@ Serving logs from workers and triggerer
Most task handlers send logs upon completion of a task. In order to view logs in real time, Airflow starts an HTTP server to serve the logs in the following cases:
-- If ``SequentialExecutor`` or ``LocalExecutor`` is used, then when ``airflow scheduler`` is running.
+- If ``LocalExecutor`` is used, then when ``airflow scheduler`` is running.
- If ``CeleryExecutor`` is used, then when ``airflow worker`` is running.
Review Comment:
Oh lol, let me update in a follow-up :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org