You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Lee-W (via GitHub)" <gi...@apache.org> on 2025/06/11 03:51:45 UTC
[PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Lee-W opened a new pull request, #51603:
URL: https://github.com/apache/airflow/pull/51603
## Why
[Ruff 0.11.13](https://github.com/astral-sh/ruff/releases/tag/0.11.13) comes with the latest autofixes which we should suggest users to use for easing dag code migration
## What
Add instructions for how to use `ruff>=0.11.13` with `AIR` rules
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139278721
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
+
+In AIR rules, unsafe fixes involve changing import paths while keeping the name of the imported member the same. For instance, changing the import from ``from airflow.sensors.base_sensor_operator import BaseSensorOperator`` to ``from airflow.sdk.bases.sensor import BaseSensorOperator`` requires ruff to remove the original import before adding the new one. In contrast, safe fixes include changes to both the member name and the import path, such as changing ``from airflow.datasets import Dataset`` to `from airflow.sdk import Asset``. These adjustments do not require ruff to remove the old import. To remove unused legacy imports, it is necessary to enable the `unused-import` rule (F401) <https://docs.astral.sh/ruff/rules/unused-import/#unused-import-f401>.
Review Comment:
put it to `.. note::` sounds like a good idea. Will update it
According to the Ruff doc, I believe any runtime change should be marked as unsafe. `airflow.sensors.base_sensor_operator.BaseSensorOperator` is not entirely the same as `airflow.sdk.bases.sensor.BaseSensorOperator`, especially if users are checking the import path in some circumstances.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #51603:
URL: https://github.com/apache/airflow/pull/51603
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139262447
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
Review Comment:
Hmmm -- not so sure about this one!
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
Review Comment:
```suggestion
The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check
for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
```
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
Review Comment:
```suggestion
To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules.
The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
```
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
Review Comment:
Can we move this line above the command?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139280987
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
Review Comment:
sure 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139190955
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
+
+In AIR rules, unsafe fixes involve changing import paths while keeping the name of the imported member the same. For instance, changing the import from ``from airflow.sensors.base_sensor_operator import BaseSensorOperator`` to ``from airflow.sdk.bases.sensor import BaseSensorOperator`` requires ruff to remove the original import before adding the new one. In contrast, safe fixes include changes to both the member name and the import path, such as changing ``from airflow.datasets import Dataset`` to `from airflow.sdk import Asset``. These adjustments do not require ruff to remove the old import. To remove unused legacy imports, it is necessary to enable the `unused-import` rule (F401) <https://docs.astral.sh/ruff/rules/unused-import/#unused-import-f401>.
Review Comment:
May be a good idea to put this in an info box (`.. note::` or `.. tip::`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139312894
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
Review Comment:
Ah alright, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139182324
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
Review Comment:
```suggestion
Unsafe fixes usually do not break dag code. They're marked as unsafe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
```
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
+
+In AIR rules, unsafe fixes involve changing import paths while keeping the name of the imported member the same. For instance, changing the import from ``from airflow.sensors.base_sensor_operator import BaseSensorOperator`` to ``from airflow.sdk.bases.sensor import BaseSensorOperator`` requires ruff to remove the original import before adding the new one. In contrast, safe fixes include changes to both the member name and the import path, such as changing ``from airflow.datasets import Dataset`` to `from airflow.sdk import Asset``. These adjustments do not require ruff to remove the old import. To remove unused legacy imports, it is necessary to enable the `unused-import` rule (F401) <https://docs.astral.sh/ruff/rules/unused-import/#unused-import-f401>.
Review Comment:
Hmm, just because the class moved, and thus ruff has to delete the original import, is enough to make it "unsafe"?
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
+
+In AIR rules, unsafe fixes involve changing import paths while keeping the name of the imported member the same. For instance, changing the import from ``from airflow.sensors.base_sensor_operator import BaseSensorOperator`` to ``from airflow.sdk.bases.sensor import BaseSensorOperator`` requires ruff to remove the original import before adding the new one. In contrast, safe fixes include changes to both the member name and the import path, such as changing ``from airflow.datasets import Dataset`` to `from airflow.sdk import Asset``. These adjustments do not require ruff to remove the old import. To remove unused legacy imports, it is necessary to enable the `unused-import` rule (F401) <https://docs.astral.sh/ruff/rules/unused-import/#unused-import-f401>.
Review Comment:
This level of detail is probably not necessary, but I'm 50/50 on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139138776
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
----------------------------------------------------------------
-To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_.
+To minimize friction for users upgrading from prior versions of Airflow, we have created a dag upgrade check utility using `Ruff <https://docs.astral.sh/ruff/>`_ combined with `AIR <https://docs.astral.sh/ruff/rules/#airflow-air>`_ rules. The rules AIR301 and AIR302 indicate breaking changes in Airflow 3, while AIR311 and AIR312 highlight changes that are not currently breaking but are strongly recommended for updates.
-The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.6``. The below example demonstrates how to check
-for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
+The latest available ``ruff`` version will have the most up-to-date rules, but be sure to use at least version ``0.11.13``. The below example demonstrates how to check for dag incompatibilities that will need to be fixed before they will work as expected on Airflow 3.
.. code-block:: bash
- ruff check dag/ --select AIR301 --preview
+ ruff check dags/ --select AIR301 --preview
To preview the recommended fixes, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --show-fixes --preview
+ ruff check dags/ --select AIR301 --show-fixes --preview
Some changes can be automatically fixed. To do so, run the following command:
.. code-block:: bash
- ruff check dag/ --select AIR301 --fix --preview
+ ruff check dags/ --select AIR301 --fix --preview
+Some of the fixes are marked as unsafe. To trigger these fixes, run the following command:
+
+.. code-block:: bash
+
+ ruff check dags/ --select AIR301 --fix --unsafe-fixes --preview
+
+Unsafe fixes usually do not break dag code. They're marked as safe as they may change some runtime behavior. For more information, see `Fix Safety <https://docs.astral.sh/ruff/linter/#fix-safety>`_.
+
+In AIR rules, unsafe fixes involve changing import paths while keeping the name of the imported member the same. For instance, changing the import from ``from airflow.sensors.base_sensor_operator import BaseSensorOperator`` to ``from airflow.sdk.bases.sensor import BaseSensorOperator`` requires ruff to remove the original import before adding the new one. In contrast, safe fixes include changes to both the member name and the import path, such as changing ``from airflow.datasets import Dataset`` to `from airflow.sdk import Asset``. These adjustments do not require ruff to remove the old import. To remove unused legacy imports, it is necessary to enable the `unused-import` rule (F401) <https://docs.astral.sh/ruff/rules/unused-import/#unused-import-f401>.
Review Comment:
Not sure whether this section contains too much information but this explains why some rules're unsafe fixes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139275844
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
Review Comment:
[This is the direction we've been moving for a while](https://lists.apache.org/thread/lktrzqkzrpvc1cyctxz7zxfmc0fwtq2j). It does initially feel weird though after years of the other direction :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] docs(upgrading_to_airflow3): add instruction to use ruff >= 0.11.13 for more auto fixes [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #51603:
URL: https://github.com/apache/airflow/pull/51603#discussion_r2139280182
##########
airflow-core/docs/installation/upgrading_to_airflow3.rst:
##########
@@ -46,31 +46,40 @@ Step 2: Clean and back up your existing Airflow Instance
ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed
(and all errors gone) before you proceed with upgrade.
-Step 3: DAG Authors - Check your Airflow DAGs for compatibility
+Step 3: Dag Authors - Check your Airflow dags for compatibility
Review Comment:
@jedcunningham beats me haha, I was trying to search for this thread. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org