You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2025/01/29 08:53:03 UTC
[PR] Moving EmptyOperator to standard provider [airflow]
amoghrajesh opened a new pull request, #46231:
URL: https://github.com/apache/airflow/pull/46231
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1939031839
##########
providers/tests/system/microsoft/azure/example_adf_run_pipeline.py:
##########
@@ -22,12 +22,12 @@
from airflow.models import DAG
from airflow.models.xcom_arg import XComArg
+from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
+from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
# Ignore missing args provided by default_args
# mypy: disable-error-code="call-arg"
-from airflow.operators.empty import EmptyOperator
-from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
-from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
I dont think we need to move it out, seems like its correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630111566
> May I know what is [test_zip.zip](https://github.com/apache/airflow/pull/46231/files#diff-e7119e5543b7e06772411ff4addb8796acc9dfa5d671bc937fce641ca6c01924) and why do we need it?
You can deploy dag in zip files.
I assume that the example dag in that zip contains EmptyOperator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh closed pull request #46231: Moving EmptyOperator to standard provider
URL: https://github.com/apache/airflow/pull/46231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1939049371
##########
tests/serialization/test_dag_serialization.py:
##########
@@ -64,9 +64,9 @@
from airflow.models.expandinput import EXPAND_INPUT_EMPTY
from airflow.models.mappedoperator import MappedOperator
from airflow.models.xcom import XCom
-from airflow.operators.empty import EmptyOperator
from airflow.providers.cncf.kubernetes.pod_generator import PodGenerator
from airflow.providers.standard.operators.bash import BashOperator
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Right yeah, let me move it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630066297
> @vincbeck we will be adding the migration rules for this as requested. @sunank200 / @Lee-W will be assisting with that.
It would still be nice if you could list down the migration rules so that I can focus on the implementation. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630118325
> > > May I know what is [test_zip.zip](https://github.com/apache/airflow/pull/46231/files#diff-e7119e5543b7e06772411ff4addb8796acc9dfa5d671bc937fce641ca6c01924) and why do we need it?
> >
> >
> > You can deploy dag in zip files. I assume that the example dag in that zip contains EmptyOperator
>
> @amoghrajesh Is it possible for us to zip it through script instead of just uploading a zip file? It's hard to know what's actually inside the zip
Hmm, its been done this way historically too, I guess we can but I can do it in a follow up if thats ok? I will list the zip changes here:
```
(airflow) ➜ dags git:(AIP72-move-empty-operator) ✗ ls -lR test_zip
total 16
-rw-r--r-- 1 amoghdesai staff 566 Jun 16 2018 file_no_airflow_dag.py
-rw-r--r-- 1 amoghdesai staff 1171 Jan 30 12:50 test_zip.py
drwxr-xr-x 4 amoghdesai staff 128 Apr 13 2016 test_zip_module
test_zip/test_zip_module:
total 8
-rw-r--r-- 1 amoghdesai staff 0 Apr 13 2016 __init__.py
-rw-r--r-- 1 amoghdesai staff 642 Apr 13 2016 test.py
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1936784338
##########
newsfragments/46231.significant.rst:
##########
@@ -0,0 +1,19 @@
+.. Moving EmptyOperator from Airflow core to the ``standard`` provider.
+
+.. EmptyOperator has been moved from Airflow core (``airflow.operators directory``) to the ``standard`` provider.
+For new and existing DAGs, users must import ``EmptyOperator`` from ``airflow.providers.standard.operators.empty``.
+
+* Types of change
+
+ * [x] Dag changes
+ * [ ] Config changes
+ * [ ] API changes
+ * [ ] CLI changes
+ * [ ] Behaviour changes
+ * [ ] Plugin changes
+ * [ ] Dependency changes
+ * [ ] Code interface changes
+
+.. List the migration rules needed for this change (see https://github.com/apache/airflow/issues/41641)
+
+* Migration rules needed
Review Comment:
Yep, checking with the team on this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630063419
@vincbeck we will be adding the migration rules for this as requested. @sunank200 / @Lee-W will be assisting with that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630114530
> > May I know what is [test_zip.zip](https://github.com/apache/airflow/pull/46231/files#diff-e7119e5543b7e06772411ff4addb8796acc9dfa5d671bc937fce641ca6c01924) and why do we need it?
>
> You can deploy dag in zip files. I assume that the example dag in that zip contains EmptyOperator
It does yeah, it contains a reference to the older EmptyOperator path, i updated that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2632698980
@amoghrajesh Sorry for the CI error. The fix is created https://github.com/apache/airflow/pull/46404
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh merged PR #46231:
URL: https://github.com/apache/airflow/pull/46231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2627326095
@eladkal @ashb got a green CI finally. Can you take a look at this when you have some time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1938885394
##########
providers/tests/system/microsoft/azure/example_adf_run_pipeline.py:
##########
@@ -22,12 +22,12 @@
from airflow.models import DAG
from airflow.models.xcom_arg import XComArg
+from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
+from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
# Ignore missing args provided by default_args
# mypy: disable-error-code="call-arg"
-from airflow.operators.empty import EmptyOperator
-from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
-from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
yep, but you moved other provider things out of this block so just want to confirm whether we could do that for empty operator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1938881716
##########
providers/tests/system/microsoft/azure/example_adf_run_pipeline.py:
##########
@@ -22,12 +22,12 @@
from airflow.models import DAG
from airflow.models.xcom_arg import XComArg
+from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
+from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
# Ignore missing args provided by default_args
# mypy: disable-error-code="call-arg"
-from airflow.operators.empty import EmptyOperator
-from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
-from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Hm just followed the general pattern and renamed occurences. Was added as part of #22974
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1938866912
##########
tests/serialization/test_dag_serialization.py:
##########
@@ -64,9 +64,9 @@
from airflow.models.expandinput import EXPAND_INPUT_EMPTY
from airflow.models.mappedoperator import MappedOperator
from airflow.models.xcom import XCom
-from airflow.operators.empty import EmptyOperator
from airflow.providers.cncf.kubernetes.pod_generator import PodGenerator
from airflow.providers.standard.operators.bash import BashOperator
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
if it's for mypy, shouldn't we use `if TYPE_CHECKING`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2632786252
> @amoghrajesh Sorry for the CI error. The fix is created #46404
No problem, thanks! I was wondering what the issue was!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2621734868
@eladkal aren't the other ones for example: bash operator breaking in that case too? https://github.com/apache/airflow/pull/42252
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2626484813
@potiuk it'd be nice if you can take a look on this too, since you were involved in the past standard operator moves
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "sunank200 (via GitHub)" <gi...@apache.org>.
sunank200 commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1936814566
##########
tests_common/pytest_plugin.py:
##########
@@ -40,7 +40,7 @@
from airflow.models.dag import DAG, ScheduleArg
from airflow.models.dagrun import DagRun, DagRunType
from airflow.models.taskinstance import TaskInstance
- from airflow.operators.empty import EmptyOperator
+ from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
why import it here if it is imported lazily in the file inside functions?
##########
tests/serialization/test_dag_serialization.py:
##########
@@ -64,9 +64,9 @@
from airflow.models.expandinput import EXPAND_INPUT_EMPTY
from airflow.models.mappedoperator import MappedOperator
from airflow.models.xcom import XCom
-from airflow.operators.empty import EmptyOperator
from airflow.providers.cncf.kubernetes.pod_generator import PodGenerator
from airflow.providers.standard.operators.bash import BashOperator
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
why import it here if it is imported lazily in the file inside functions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1936929742
##########
tests_common/pytest_plugin.py:
##########
@@ -40,7 +40,7 @@
from airflow.models.dag import DAG, ScheduleArg
from airflow.models.dagrun import DagRun, DagRunType
from airflow.models.taskinstance import TaskInstance
- from airflow.operators.empty import EmptyOperator
+ from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Thats because it is in a type checking block used for mypy etc. But we import it normally in functions that need it for sake of usage
##########
tests/serialization/test_dag_serialization.py:
##########
@@ -64,9 +64,9 @@
from airflow.models.expandinput import EXPAND_INPUT_EMPTY
from airflow.models.mappedoperator import MappedOperator
from airflow.models.xcom import XCom
-from airflow.operators.empty import EmptyOperator
from airflow.providers.cncf.kubernetes.pod_generator import PodGenerator
from airflow.providers.standard.operators.bash import BashOperator
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Thats because it is in a type checking block used for mypy etc. But we import it normally in functions that need it for sake of usage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1938855097
##########
providers/tests/system/microsoft/azure/example_adf_run_pipeline.py:
##########
@@ -22,12 +22,12 @@
from airflow.models import DAG
from airflow.models.xcom_arg import XComArg
+from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
+from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
# Ignore missing args provided by default_args
# mypy: disable-error-code="call-arg"
-from airflow.operators.empty import EmptyOperator
-from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
-from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Do we need to make it part of mypy disable?
##########
newsfragments/41368.significant.rst:
##########
@@ -30,10 +30,10 @@ For example, instead of ``from airflow.sensors import TimeDeltaSensor``, use ``f
* [x] ``airflow.operators.bash_operator.BashOperator`` → ``airflow.operators.bash.BashOperator``
* [x] ``airflow.operators.branch_operator.BaseBranchOperator`` → ``airflow.operators.branch.BaseBranchOperator``
- * [x] ``airflow.operators....EmptyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators....DummyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators....EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators....DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
Review Comment:
```suggestion
* [ ] ``airflow.operators....EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
* [ ] ``airflow.operators....DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
* [ ] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
* [ ] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
```
As we'll need to change the rules, let's unmark them so that I won't miss them
##########
newsfragments/46231.significant.rst:
##########
@@ -0,0 +1,19 @@
+.. Moving EmptyOperator from Airflow core to the ``standard`` provider.
+
+.. EmptyOperator has been moved from Airflow core (``airflow.operators directory``) to the ``standard`` provider.
+For new and existing DAGs, users must import ``EmptyOperator`` from ``airflow.providers.standard.operators.empty``.
+
+* Types of change
+
+ * [x] Dag changes
+ * [ ] Config changes
+ * [ ] API changes
+ * [ ] CLI changes
+ * [ ] Behaviour changes
+ * [ ] Plugin changes
+ * [ ] Dependency changes
+ * [ ] Code interface changes
+
+.. List the migration rules needed for this change (see https://github.com/apache/airflow/issues/41641)
+
+* Migration rules needed
Review Comment:
Yep, we'll need migration rules for this one. I think this is `AIR302`. You can take a look at how other newsfragments are arranged. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1936926316
##########
tests_common/pytest_plugin.py:
##########
@@ -40,7 +40,7 @@
from airflow.models.dag import DAG, ScheduleArg
from airflow.models.dagrun import DagRun, DagRunType
from airflow.models.taskinstance import TaskInstance
- from airflow.operators.empty import EmptyOperator
+ from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Not particularly done on purpose. It was a migration effort, so the old occurences => new occurences were made.
Anyway, unified it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2632962656
Feel free to resolve my comments. I think we're close to merge 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2629671007
> @eladkal aren't the other ones for example: bash operator breaking in that case too? #42252
Yep, we'll need to create a newsfragment for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630114344
> > May I know what is [test_zip.zip](https://github.com/apache/airflow/pull/46231/files#diff-e7119e5543b7e06772411ff4addb8796acc9dfa5d671bc937fce641ca6c01924) and why do we need it?
>
> You can deploy dag in zip files. I assume that the example dag in that zip contains EmptyOperator
@amoghrajesh Is it possible for us to zip it through script instead of just uploading a zip file? It's hard to know what's actually inside the zip
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1938880173
##########
newsfragments/41368.significant.rst:
##########
@@ -30,10 +30,10 @@ For example, instead of ``from airflow.sensors import TimeDeltaSensor``, use ``f
* [x] ``airflow.operators.bash_operator.BashOperator`` → ``airflow.operators.bash.BashOperator``
* [x] ``airflow.operators.branch_operator.BaseBranchOperator`` → ``airflow.operators.branch.BaseBranchOperator``
- * [x] ``airflow.operators....EmptyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators....DummyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators....EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators....DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
Review Comment:
Its done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630131008
Yeah, will pick it up @Lee-W.
It isn't related to this change, so better to follow up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1935835556
##########
newsfragments/46231.significant.rst:
##########
@@ -0,0 +1,19 @@
+.. Moving EmptyOperator from Airflow core to the ``standard`` provider.
+
+.. EmptyOperator has been moved from Airflow core (``airflow.operators directory``) to the ``standard`` provider.
+For new and existing DAGs, users must import ``EmptyOperator`` from ``airflow.providers.standard.operators.empty``.
+
+* Types of change
+
+ * [x] Dag changes
+ * [ ] Config changes
+ * [ ] API changes
+ * [ ] CLI changes
+ * [ ] Behaviour changes
+ * [ ] Plugin changes
+ * [ ] Dependency changes
+ * [ ] Code interface changes
+
+.. List the migration rules needed for this change (see https://github.com/apache/airflow/issues/41641)
+
+* Migration rules needed
Review Comment:
To make migration easier for users we should have a migration rule for it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1936926316
##########
tests_common/pytest_plugin.py:
##########
@@ -40,7 +40,7 @@
from airflow.models.dag import DAG, ScheduleArg
from airflow.models.dagrun import DagRun, DagRunType
from airflow.models.taskinstance import TaskInstance
- from airflow.operators.empty import EmptyOperator
+ from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
Not particularly done on purpose. It was a migration effort, so the old occurences => new occurences were made.
Anyway, unified it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630069503
Oops bad rebase. FIxing!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2630128600
> > > > May I know what is [test_zip.zip](https://github.com/apache/airflow/pull/46231/files#diff-e7119e5543b7e06772411ff4addb8796acc9dfa5d671bc937fce641ca6c01924) and why do we need it?
> > >
> > >
> > > You can deploy dag in zip files. I assume that the example dag in that zip contains EmptyOperator
> >
> >
> > @amoghrajesh Is it possible for us to zip it through script instead of just uploading a zip file? It's hard to know what's actually inside the zip
>
> Hmm, its been done this way historically too, I guess we can but I can do it in a follow up if thats ok? I will list the zip changes here:
>
> ```
> (airflow) ➜ dags git:(AIP72-move-empty-operator) ✗ ls -lR test_zip
> total 16
> -rw-r--r-- 1 amoghdesai staff 566 Jun 16 2018 file_no_airflow_dag.py
> -rw-r--r-- 1 amoghdesai staff 1171 Jan 30 12:50 test_zip.py
> drwxr-xr-x 4 amoghdesai staff 128 Apr 13 2016 test_zip_module
>
> test_zip/test_zip_module:
> total 8
> -rw-r--r-- 1 amoghdesai staff 0 Apr 13 2016 __init__.py
> -rw-r--r-- 1 amoghdesai staff 642 Apr 13 2016 test.py
> ```
Yep, I know it's done this way previously. If this is blocking something, we can create an issue for tracking this and do it in follow up PRs. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1938877070
##########
newsfragments/41368.significant.rst:
##########
@@ -30,10 +30,10 @@ For example, instead of ``from airflow.sensors import TimeDeltaSensor``, use ``f
* [x] ``airflow.operators.bash_operator.BashOperator`` → ``airflow.operators.bash.BashOperator``
* [x] ``airflow.operators.branch_operator.BaseBranchOperator`` → ``airflow.operators.branch.BaseBranchOperator``
- * [x] ``airflow.operators....EmptyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators....DummyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.operators.empty.EmptyOperator``
- * [x] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators....EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators....DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators.dummy_operator.EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
+ * [x] ``airflow.operators.dummy_operator.DummyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``
Review Comment:
Cool
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #46231:
URL: https://github.com/apache/airflow/pull/46231#discussion_r1939039460
##########
providers/tests/system/microsoft/azure/example_adf_run_pipeline.py:
##########
@@ -22,12 +22,12 @@
from airflow.models import DAG
from airflow.models.xcom_arg import XComArg
+from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
+from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
# Ignore missing args provided by default_args
# mypy: disable-error-code="call-arg"
-from airflow.operators.empty import EmptyOperator
-from airflow.providers.microsoft.azure.operators.data_factory import AzureDataFactoryRunPipelineOperator
-from airflow.providers.microsoft.azure.sensors.data_factory import AzureDataFactoryPipelineRunStatusSensor
+from airflow.providers.standard.operators.empty import EmptyOperator
Review Comment:
sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Moving EmptyOperator to standard provider [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #46231:
URL: https://github.com/apache/airflow/pull/46231#issuecomment-2621065787
It needs newsfragment as breaking change for Airflow 3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org