You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "bugraoz93 (via GitHub)" <gi...@apache.org> on 2026/02/08 11:44:21 UTC
[PR] Add min version to prek upgrade [airflow]
bugraoz93 opened a new pull request, #61634:
URL: https://github.com/apache/airflow/pull/61634
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779253221
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ # We should not add minimum_prek_version into automation unless this installation part automated with prek
Review Comment:
Or should we include something above the actual `minimum_prek_version` part in the YAML definition too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779201206
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
Would this then force everyone to keep up with prek version to continue developing?
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
Would this then force everyone to keep up with prek version to continue developing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779199213
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
I overlooked the part of user and developer perspective no user here to be clear :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779205736
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
> Would this then force everyone to keep up with prek version to continue developing?
Yes. It would. Again - we can do it once in a while (this will make everyone having to upgrade - they will get the error that they need to upgrade prek, but we should likely just remove the automated upgrade of it.
So we can merge that one today - but without automation of min_version upgrade (and possibly with a comment explaining all that so that others are not tempted to add it again).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779252305
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ # We should not add minimum_prek_version into automation unless this installation part automated with prek
Review Comment:
@potiuk do you mean something like this as comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade prek patch release and min_prek_version to align [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2784366218
##########
.pre-commit-config.yaml:
##########
@@ -16,7 +16,7 @@
# under the License.
---
default_stages: [pre-commit, pre-push]
-minimum_prek_version: '0.2.22'
+minimum_prek_version: '0.3.2'
Review Comment:
Thanks for the comment!
I tried to explain the reasoning here https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1770561908477689
Let me summarise here too, as I saw that the description of the PR is weak,
Even small changes have an impact on the entire CI, so making changes step by step is also easier to review and inform the community well. Since static checks do not fail with the update, this is the most atomic part of the change.
We regularly update packages/dependencies. There are certain packages/dependencies that we upgrade in intervals through automated detection, and with a semi-automated approach, we are upgrading those packages/dependencies. For these CI team and contributors, trying to keep everything green and adopting breaking changes or following those up. For example, `hooks` are also included in this process if there is no `# frozen: ...` comment on it. The contributors checking CI channel regularly and ensuring those are updated.
There are some packages/dependencies we manually upgrade or min version like this that impacts entire community. There are not so many of them, but for those who generally find it hard to keep up with the updates daily or at more intervals, or we haven't updated them because new features are not needed.
This PR is one of the changes that impact the community rather than keep things green with rebases. Hope it will enable us to use new features and try to provide one of the maintenance work at some point we need to do.
**Why are we doing that?**
For some, even minor upgrades cause a break in the CI or some part of the code, for example, due to an annotation change. Those are from the `uv` package manager to `prek` and even `fastapi` updates.
**Why did I keep this PR atomic?**
Normally, the upgrade PRs are generally merged within hours, and this one will have an impact on each developer in the community. This should be well communicated and people checking the PR shouldn't see lots of updates noises the actual change in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779207840
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
Yes, https://prek.j178.dev/configuration/?h=minimum_prek_version#minimum_prek_version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade prek patch release and min_prek_version to align [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2784366218
##########
.pre-commit-config.yaml:
##########
@@ -16,7 +16,7 @@
# under the License.
---
default_stages: [pre-commit, pre-push]
-minimum_prek_version: '0.2.22'
+minimum_prek_version: '0.3.2'
Review Comment:
Thanks for the comment!
I tried to explain the reasoning here https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1770561908477689
Let me summarise here too, asI saw that the description of the PR is weak,
Even small changes have an impact on the entire CI, so making changes step by step is also easier to review and inform the community well. Since static checks do not fail with the update, this is the most atomic part of the change.
We regularly update packages/dependencies. There are certain packages/dependencies that we upgrade in intervals through automated detection, and with a semi-automated approach, we are upgrading those packages/dependencies. For these CI team and contributors, trying to keep everything green and adopting breaking changes or following those up. For example, `hooks` are also included in this process if there is no `# frozen: ...` comment on it. The contributors checking CI channel regularly and ensuring those are updated.
There are some packages/dependencies we manually upgrade or min version like this that impacts entire community. There are not so many of them, but for those who generally find it hard to keep up with the updates daily or at more intervals, or we haven't updated them because new features are not needed.
This PR is one of the changes that impact the community rather than keep things green with rebases. Hope it will enable us to use new features and try to provide one of the maintenance work at some point we need to do.
**Why are we doing that?**
For some, even minor upgrades cause a break in the CI or some part of the code, for example, due to an annotation change. Those are from the `uv` package manager to `prek` and even `fastapi` updates.
**Why did I keep this PR atomic?**
Normally, the upgrade PRs are generally merged within hours, and this one will have an impact on each developer in the community. This should be well communicated and people checking the PR shouldn't see lots of updates noises the actual change in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade minimum prek version [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 merged PR #61634:
URL: https://github.com/apache/airflow/pull/61634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779270689
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ # We should not add minimum_prek_version into automation unless this installation part automated with prek
Review Comment:
Here is fine :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade minimum prek version [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61634:
URL: https://github.com/apache/airflow/pull/61634#issuecomment-3880443506
### Backport successfully created: v3-1-test
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-1-test</td>
<td><a href="https://github.com/apache/airflow/pull/61744"><img src="https://img.shields.io/badge/PR-61744-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779193820
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
As discussed in slack:
I think we should really reserve min-version upgrade in prek when we need some new feature (like we did with workspace and inline script metadata).
Generally it is not too important for everyone to have the same prek - because prek is merely executing all our scripts with prek hooks. We can also do it "from time to time" - i.e. every few months we bump the min version manually, just to make sure people do not use some ancient version, and we should still upgrade it in our CI - so that we know things are still working with the latest version, but I do not think min-version for prek should be "always" updated when we bump it in CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779195313
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
I am all up for closing this or just reverting that part, this could indeed then force too much people to do this upgrade as well while we aren't using those features
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade prek patch release and min_prek_version to align [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #61634:
URL: https://github.com/apache/airflow/pull/61634#issuecomment-3867333948
Sent a ping to #contributors channel to raise awareness beforethe merge :) I will soon, just allowing some time to read from a couple to see :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade prek patch release and min_prek_version to align [airflow]
Posted by "ulgens (via GitHub)" <gi...@apache.org>.
ulgens commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2783170649
##########
.pre-commit-config.yaml:
##########
@@ -16,7 +16,7 @@
# under the License.
---
default_stages: [pre-commit, pre-push]
-minimum_prek_version: '0.2.22'
+minimum_prek_version: '0.3.2'
Review Comment:
Is this a common practice? I couldn't catch why the min version is updated while the hooks are the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add min version to prek upgrade [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2779201272
##########
scripts/ci/prek/upgrade_important_versions.py:
##########
@@ -373,6 +373,7 @@ class Quoting(Enum):
),
Quoting.UNQUOTED,
),
+ (re.compile(r"(minimum_prek_version: )('[0-9.abrc]+')", re.MULTILINE), Quoting.SINGLE_QUOTED),
Review Comment:
Would this then force everyone to keep up with prek version to continue developing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade prek patch release and min_prek_version to align [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #61634:
URL: https://github.com/apache/airflow/pull/61634#discussion_r2789723026
##########
.pre-commit-config.yaml:
##########
@@ -16,7 +16,7 @@
# under the License.
---
default_stages: [pre-commit, pre-push]
-minimum_prek_version: '0.2.22'
+minimum_prek_version: '0.3.2'
Review Comment:
Hope this is a good one for you as well, Ulgen! I will resolve the comment to merge it soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Upgrade prek patch release and min_prek_version to align [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #61634:
URL: https://github.com/apache/airflow/pull/61634#issuecomment-3880063487
Will resolve the conflict and fix this. Then will ping contributors once more from the same thread to follow up with the suggested approaches
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org