You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2024/06/24 04:59:14 UTC
[PR] Ability to update provider version dynamically during a release [airflow]
amoghrajesh opened a new pull request, #40395:
URL: https://github.com/apache/airflow/pull/40395
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
If you run `breeze release-management prepare-provider-documentation` and generate the docs, like for [this PR](https://github.com/apache/airflow/pull/40273) and if the PR isn't merged because of whatever reason, you need to rebase and rerun the command to update the docs.
The problem is the command doesn't notify you or let you change versions on the fly. It generated docs for version 8.24.1, but after rebasing, you need to update it to 8.25.0 due to [this addition](https://github.com/apache/airflow/pull/40287). The command only confirms the current version, forcing you to manually edit provider.yaml and rerun it.
This PR adds changes that offer to change the version (bug, feature, major etc)
Example output:
```
➜ airflow git:(breezeImprovement2) ✗ breeze release-management prepare-provider-documentation amazon
Good version of Docker: 25.0.3.
Good version of docker-compose: 2.24.6
Executable permissions on entrypoints are OK
Fetching full history and tags from remote.
This might override your local tags!
From https://github.com/apache/airflow
* [new branch] 31851 -> apache-https-for-providers/31851
* [new branch] add-missing-input-in-special-tests -> apache-https-for-providers/add-missing-input-in-special-tests
* [new branch] add-zlib1g-dev-package-to-image -> apache-https-for-providers/add-zlib1g-dev-package-to-image
* [new branch] another-remove-backcompat-gcs -> apache-https-for-providers/another-remove-backcompat-gcs
* [new branch] better-cleanup-for-ci -> apache-https-for-providers/better-cleanup-for-ci
* [new branch] cleanup-runs-on-in-workflows -> apache-https-for-providers/cleanup-runs-on-in-workflows
* [new branch] cncf-kubernetes-4.4.1-rc -> apache-https-for-providers/cncf-kubernetes-4.4.1-rc
* [new branch] consistent-approach-for-runs-on-usage -> apache-https-for-providers/consistent-approach-for-runs-on-usage
* [new branch] constraints-1-10 -> apache-https-for-providers/constraints-1-10
* [new branch] constraints-2-0 -> apache-https-for-providers/constraints-2-0
* [new branch] constraints-2-1 -> apache-https-for-providers/constraints-2-1
* [new branch] constraints-2-1-fixed -> apache-https-for-providers/constraints-2-1-fixed
* [new branch] constraints-2-2 -> apache-https-for-providers/constraints-2-2
* [new branch] constraints-2-2-2-fixed -> apache-https-for-providers/constraints-2-2-2-fixed
* [new branch] constraints-2-2-3-fixed -> apache-https-for-providers/constraints-2-2-3-fixed
* [new branch] constraints-2-3 -> apache-https-for-providers/constraints-2-3
* [new branch] constraints-2-4 -> apache-https-for-providers/constraints-2-4
* [new branch] constraints-2-5 -> apache-https-for-providers/constraints-2-5
* [new branch] constraints-2-5-2-fixed -> apache-https-for-providers/constraints-2-5-2-fixed
* [new branch] constraints-2-6 -> apache-https-for-providers/constraints-2-6
* [new branch] constraints-2-7 -> apache-https-for-providers/constraints-2-7
* [new branch] constraints-2-8 -> apache-https-for-providers/constraints-2-8
* [new branch] constraints-2-9 -> apache-https-for-providers/constraints-2-9
* [new branch] constraints-2.0.0-fix -> apache-https-for-providers/constraints-2.0.0-fix
* [new branch] constraints-2.0.1-fix -> apache-https-for-providers/constraints-2.0.1-fix
* [new branch] constraints-2.0.2-fix -> apache-https-for-providers/constraints-2.0.2-fix
* [new branch] constraints-2.1.0-fix -> apache-https-for-providers/constraints-2.1.0-fix
* [new branch] constraints-2.1.1-fix -> apache-https-for-providers/constraints-2.1.1-fix
* [new branch] constraints-2.1.2-fix -> apache-https-for-providers/constraints-2.1.2-fix
* [new branch] constraints-2.1.3-fix -> apache-https-for-providers/constraints-2.1.3-fix
* [new branch] constraints-2.1.4-fix -> apache-https-for-providers/constraints-2.1.4-fix
* [new branch] constraints-2.2.0-fix -> apache-https-for-providers/constraints-2.2.0-fix
* [new branch] constraints-2.2.1-fix -> apache-https-for-providers/constraints-2.2.1-fix
* [new branch] constraints-2.2.2-fix -> apache-https-for-providers/constraints-2.2.2-fix
* [new branch] constraints-2.2.3-fix -> apache-https-for-providers/constraints-2.2.3-fix
* [new branch] constraints-2.2.4-fix -> apache-https-for-providers/constraints-2.2.4-fix
* [new branch] constraints-2.2.5-fix -> apache-https-for-providers/constraints-2.2.5-fix
* [new branch] constraints-2.3.0-fix -> apache-https-for-providers/constraints-2.3.0-fix
* [new branch] constraints-2.3.0-fixed -> apache-https-for-providers/constraints-2.3.0-fixed
* [new branch] constraints-2.3.1-fix -> apache-https-for-providers/constraints-2.3.1-fix
* [new branch] constraints-2.3.1-fixed -> apache-https-for-providers/constraints-2.3.1-fixed
* [new branch] constraints-2.3.2-fix -> apache-https-for-providers/constraints-2.3.2-fix
* [new branch] constraints-2.3.2-fixed -> apache-https-for-providers/constraints-2.3.2-fixed
* [new branch] constraints-2.3.3-fix -> apache-https-for-providers/constraints-2.3.3-fix
* [new branch] constraints-2.3.4-fix -> apache-https-for-providers/constraints-2.3.4-fix
* [new branch] constraints-2.4.0-fix -> apache-https-for-providers/constraints-2.4.0-fix
* [new branch] constraints-2.4.1-fix -> apache-https-for-providers/constraints-2.4.1-fix
* [new branch] constraints-2.4.2-fix -> apache-https-for-providers/constraints-2.4.2-fix
* [new branch] constraints-2.4.3-fix -> apache-https-for-providers/constraints-2.4.3-fix
* [new branch] constraints-2.5.0-fix -> apache-https-for-providers/constraints-2.5.0-fix
* [new branch] constraints-2.5.1-fix -> apache-https-for-providers/constraints-2.5.1-fix
* [new branch] constraints-2.5.2-fix -> apache-https-for-providers/constraints-2.5.2-fix
* [new branch] constraints-2.5.3-fix -> apache-https-for-providers/constraints-2.5.3-fix
* [new branch] constraints-2.6.0-fix -> apache-https-for-providers/constraints-2.6.0-fix
* [new branch] constraints-2.6.1-fix -> apache-https-for-providers/constraints-2.6.1-fix
* [new branch] constraints-2.6.2-fix -> apache-https-for-providers/constraints-2.6.2-fix
* [new branch] constraints-2.6.3-fix -> apache-https-for-providers/constraints-2.6.3-fix
* [new branch] constraints-2.7.1-fix -> apache-https-for-providers/constraints-2.7.1-fix
* [new branch] constraints-2.7.2-fix -> apache-https-for-providers/constraints-2.7.2-fix
* [new branch] constraints-2.7.3-fix -> apache-https-for-providers/constraints-2.7.3-fix
* [new branch] constraints-2.8.1-fix -> apache-https-for-providers/constraints-2.8.1-fix
* [new branch] constraints-2.8.2-fix -> apache-https-for-providers/constraints-2.8.2-fix
* [new branch] constraints-mai -> apache-https-for-providers/constraints-mai
* [new branch] constraints-main -> apache-https-for-providers/constraints-main
* [new branch] correct-casing-of-python-in-ci.yml -> apache-https-for-providers/correct-casing-of-python-in-ci.yml
* [new branch] disable-pushing-cache-temporarily -> apache-https-for-providers/disable-pushing-cache-temporarily
* [new branch] dont-get-dag-we-already-have-it -> apache-https-for-providers/dont-get-dag-we-already-have-it
* [new branch] extract-basic-tests-to-separate-workflow -> apache-https-for-providers/extract-basic-tests-to-separate-workflow
* [new branch] extract-docs-to-separate-workflow -> apache-https-for-providers/extract-docs-to-separate-workflow
* [new branch] extract-early-image-checks-workflow -> apache-https-for-providers/extract-early-image-checks-workflow
* [new branch] extract-finalization-of-tests-to-separate-workflow -> apache-https-for-providers/extract-finalization-of-tests-to-separate-workflow
* [new branch] feat/max_active_tis_per_dagrun -> apache-https-for-providers/feat/max_active_tis_per_dagrun
* [new branch] fix-broken-compatibility-check -> apache-https-for-providers/fix-broken-compatibility-check
* [new branch] fix-canary-run-check -> apache-https-for-providers/fix-canary-run-check
* [new branch] fix-docker-cache-input-name -> apache-https-for-providers/fix-docker-cache-input-name
* [new branch] fix-image-cache -> apache-https-for-providers/fix-image-cache
* [new branch] fix-image-cache-2 -> apache-https-for-providers/fix-image-cache-2
* [new branch] fix-missing-condition-on-image-push -> apache-https-for-providers/fix-missing-condition-on-image-push
* [new branch] fix-openlineage-tests -> apache-https-for-providers/fix-openlineage-tests
* [new branch] fix-prod-image-ci-preparation -> apache-https-for-providers/fix-prod-image-ci-preparation
* [new branch] fix_prev_dagrun_dep_for_dynamic_tasks -> apache-https-for-providers/fix_prev_dagrun_dep_for_dynamic_tasks
* [new branch] hooks-lineage -> apache-https-for-providers/hooks-lineage
* [new branch] improve-test-finalization -> apache-https-for-providers/improve-test-finalization
* [new branch] listener-task-timeout -> apache-https-for-providers/listener-task-timeout
* [new branch] main -> apache-https-for-providers/main
* [new branch] mlnsharma/main -> apache-https-for-providers/mlnsharma/main
* [new branch] move-cleanup-docker-to-bash-script -> apache-https-for-providers/move-cleanup-docker-to-bash-script
* [new branch] move-ol-config-to-local-file -> apache-https-for-providers/move-ol-config-to-local-file
* [new branch] notify-exception-caught -> apache-https-for-providers/notify-exception-caught
* [new branch] ol-improve-sqlparser-logging -> apache-https-for-providers/ol-improve-sqlparser-logging
* [new branch] openlineage-bigquery-operation -> apache-https-for-providers/openlineage-bigquery-operation
* [new branch] openlineage-do-not-submit-busyjobs -> apache-https-for-providers/openlineage-do-not-submit-busyjobs
* [new branch] openlineage-gcs-operator -> apache-https-for-providers/openlineage-gcs-operator
* [new branch] openlineage-process-execution -> apache-https-for-providers/openlineage-process-execution
* [new branch] openlineage-processexecution -> apache-https-for-providers/openlineage-processexecution
* [new branch] openlineage-sagemaker-operators -> apache-https-for-providers/openlineage-sagemaker-operators
* [new branch] openlineage-sftp-operator -> apache-https-for-providers/openlineage-sftp-operator
* [new branch] openlineage-system-tests -> apache-https-for-providers/openlineage-system-tests
* [new branch] optimize-cache-building-workflow -> apache-https-for-providers/optimize-cache-building-workflow
* [new branch] optimize-caching-workflows -> apache-https-for-providers/optimize-caching-workflows
* [new branch] optimize-package-preparation-for-prod-builds -> apache-https-for-providers/optimize-package-preparation-for-prod-builds
* [new branch] optimize-tests-when-ci-scripts-change -> apache-https-for-providers/optimize-tests-when-ci-scripts-change
* [new branch] pankajkoti-patch-1 -> apache-https-for-providers/pankajkoti-patch-1
* [new branch] provider-cncf-kubernetes/v4-4 -> apache-https-for-providers/provider-cncf-kubernetes/v4-4
* [new branch] pytest-import-mode-importlib -> apache-https-for-providers/pytest-import-mode-importlib
* [new branch] remove-canary-typo -> apache-https-for-providers/remove-canary-typo
* [new branch] remove-fab-from-chicken-egg-providers -> apache-https-for-providers/remove-fab-from-chicken-egg-providers
* [new branch] restore-test-airflow-release-commands -> apache-https-for-providers/restore-test-airflow-release-commands
* [new branch] revert-38054-onikolas/aip-61/db_migration -> apache-https-for-providers/revert-38054-onikolas/aip-61/db_migration
* [new branch] separate-cache-workflow -> apache-https-for-providers/separate-cache-workflow
* [new branch] simplify-provider-state -> apache-https-for-providers/simplify-provider-state
* [new branch] switch-cache-building-to-public-macos-runners -> apache-https-for-providers/switch-cache-building-to-public-macos-runners
* [new branch] test-cache-refreshing -> apache-https-for-providers/test-cache-refreshing
* [new branch] test-image-cache -> apache-https-for-providers/test-image-cache
* [new branch] turn-optional-dependencies-in-dynamic-metadata -> apache-https-for-providers/turn-optional-dependencies-in-dynamic-metadata
* [new branch] use-common-image-workflows-in-pull-request-target -> apache-https-for-providers/use-common-image-workflows-in-pull-request-target
* [new branch] utkarsharma2-patch-1 -> apache-https-for-providers/utkarsharma2-patch-1
* [new branch] v1-10-stable -> apache-https-for-providers/v1-10-stable
* [new branch] v1-10-test -> apache-https-for-providers/v1-10-test
* [new branch] v1-8-stable -> apache-https-for-providers/v1-8-stable
* [new branch] v1-8-test -> apache-https-for-providers/v1-8-test
* [new branch] v1-9-stable -> apache-https-for-providers/v1-9-stable
* [new branch] v1-9-test -> apache-https-for-providers/v1-9-test
* [new branch] v2-0-stable -> apache-https-for-providers/v2-0-stable
* [new branch] v2-1-stable -> apache-https-for-providers/v2-1-stable
* [new branch] v2-1-test -> apache-https-for-providers/v2-1-test
* [new branch] v2-2-stable -> apache-https-for-providers/v2-2-stable
* [new branch] v2-2-test -> apache-https-for-providers/v2-2-test
* [new branch] v2-3-stable -> apache-https-for-providers/v2-3-stable
* [new branch] v2-3-test -> apache-https-for-providers/v2-3-test
* [new branch] v2-4-stable -> apache-https-for-providers/v2-4-stable
* [new branch] v2-5-stable -> apache-https-for-providers/v2-5-stable
* [new branch] v2-5-test -> apache-https-for-providers/v2-5-test
* [new branch] v2-6-stable -> apache-https-for-providers/v2-6-stable
* [new branch] v2-6-test -> apache-https-for-providers/v2-6-test
* [new branch] v2-7-stable -> apache-https-for-providers/v2-7-stable
* [new branch] v2-8-stable -> apache-https-for-providers/v2-8-stable
* [new branch] v2-8-test -> apache-https-for-providers/v2-8-test
* [new branch] v2-9-stable -> apache-https-for-providers/v2-9-stable
* [new branch] v2-9-test -> apache-https-for-providers/v2-9-test
* [new branch] xdist-tests-distribution -> apache-https-for-providers/xdist-tests-distribution
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Update release notes for package 'amazon'
Updating documentation for the latest release version.
New version of the 'amazon' package is ready to be released!
Provider amazon with version: 8.25.0 marked for release. Proceed?
Press y/N/q: y
Provider amazon is to be released with version: 8.25.0. Do you want to check and update versions?
Press y/N/q: y
Type of change (d)ocumentation, (b)ugfix, (f)eature, (x)breaking change, (s)kip, (q)uit ? x
Provider amazon has been classified as:
Breaking changes - bump in MAJOR version needed
Bumped version to 9.0.0
Updated source-date-epoch to 1719204731
New version of the 'amazon' package is ready to be released!
```
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Ability to update provider version dynamically during a release [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #40395:
URL: https://github.com/apache/airflow/pull/40395#issuecomment-2185903006
yes, I use
`breeze release-management prepare-provider-documentation --include-removed-providers --answer yes`
which is why I encountered this issue as I first used it to check which providers had changes then needed to manually confirm and modify the version only to run this command again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Ability to update provider version dynamically during a release [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #40395:
URL: https://github.com/apache/airflow/pull/40395#issuecomment-2185631580
Love it!
```
Provider amazon with version: 8.25.0 marked for Provider amazon with version: 8.25.0 marked for release. Proceed?
Press y/N/q: y
Provider amazon is to be released with version: 8.25.0. Do you want to check and update versions?
Press y/N/q: y
Type of change (d)ocumentation, (b)ugfix, (f)eature, (x)breaking change, (s)kip, (q)uit ? x
Provider amazon has been classified as:
```
Can we merge this to 1 question?
```
Provider amazon with version: 8.25.0 marked for release. Proceed?
Press y/N/q/d/s/f/b/x: y
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Ability to update provider version dynamically during a release [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #40395:
URL: https://github.com/apache/airflow/pull/40395#issuecomment-2185608005
@eladkal as you requested. I tested it once, but best to test it during a provider release to get real time feedback. It won't be perfect in one shot (expected) and we should stabilise it over time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Ability to update provider version dynamically during a release [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #40395:
URL: https://github.com/apache/airflow/pull/40395#issuecomment-2185892559
> Can we merge this to 1 question?
Yes that would be preffered and there is one more, important point.
One of the interesting "properties" of the current setup is that it allows you to very quickly check if there is no new changes in main.
If you look at https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#generate-release-notes
You will see this step:
> When you want to regenerate the changes before the release and make sure all changelogs are updated, run it in non-interactive mode:
```bash
breeze release-management prepare-provider-documentation --include-removed-providers --answer yes
```
This basically means that when release manager regenerates the documentation with `--answer yes` (i.e. all questions should be asnwered yes) and there is no change or the changes in the docs, this one should end up with `no-changes` and without unnecessary interaction of the release manager - just quickly answer "yes" to all the questions and see if there are no new changes generated.
The current proposal will break this pattern. because if you answer "yes" to both question, the release manager will ba asked how to modify the release.
So generally speaking only "no" answer should lead to re-classifying the provider. So the question should rather be:
"Do you want to leave the version x.y.z as is?" - answering "Yes" should continue without changes, answering "no" should lead to the question "how do you want to modify it".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org