You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/17 23:14:16 UTC
[GitHub] [airflow] potiuk opened a new pull request #8898: Correct generation and package generation scripts
potiuk opened a new pull request #8898:
URL: https://github.com/apache/airflow/pull/8898
---
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Target Github ISSUE in description if exists
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427242623
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
Ah I see. Ther is no 'airflow version" command in those packages. And we can easily do it for airflow in the future - instead of reading from version.py we could generate an embedded version.txt file generated on the fly during package building and read the version from there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#issuecomment-630693893
HEy @ashb @kaxil @mik-laj -> all good. Just quarntined tests failed. I think we can merge it and I can cut rc2 release of the backports right after !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427241523
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
@kaxil - It will still show rc1 it for the PyPIi packages (but not for the SVN ones). It works the same way as airflow packages but without having to modify the source code - just dynamically appending it with a flag.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427186700
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
We need to change the version.py file before we upload it to PyPI to include rcX.
Do we already do that somewhere ?
##########
File path: backport_packages/setup_backport_packages.py
##########
@@ -1074,19 +1103,32 @@ def update_release_notes_for_packages(package_ids: List[str], release_version: s
import_all_classes(package_ids)
make_sure_remote_apache_exists()
if len(package_ids) == 0:
- package_ids = list(PROVIDERS_REQUIREMENTS.keys())
+ package_ids = get_all_backportable_providers()
for package in package_ids:
update_release_notes_for_package(package, release_version)
+def get_all_backportable_providers() -> List[str]:
+ """
+ Returns all providers that should be taken into account when preparing backports.
+ For now remove cncf.kubernetes as it has no chances to work with current core of Airflow 2.0
+ :return: list of providers that are considered for backport packages
+ """
+ # TODO: Maybe we should fix it and release cnncf.kubernetes separately
Review comment:
```suggestion
# TODO: Maybe we should fix it and release cncf.kubernetes separately
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #8898:
URL: https://github.com/apache/airflow/pull/8898
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427242623
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
Ah I see. Ther is no 'airflow version" command in those packages. And we can easily do it for airflow in the future - instead of reading from version.py we could generate an embedded version.txt file generated on the flight during package building and read the version from there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427240664
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
> I think historically this was done so that even if someone installed the rc packages when they do `airflow version` it shows `rc` string in it. However maybe for backport packages it might not matter, thoughts @ashb ?
Yeah, since Airflow has `airflow.__version__` we want that to be right for RC releases, but as you say, the backport providers don't have this attribute anywhere (and it's probably not needed to add one) so this is a good approach for providers.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#issuecomment-630095741
> @ashb @kaxil -> I figured out that we have really two modes of release - one for "SVN" where we only rename the files with rc but the version inside is without the suffix, and the PyPI ones where we set version to "rc" and the package name also has "rc" as part of the name.
Yes, as documented (albeit not clearly) https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow#ReleasingAirflow-PyPISnapshots(Optional).1
>
> I solved it with two switches
>
> * `--artifact-name-suffix rc1`-> only changes name of the .tar.gz/.whl packages - used for SVN package generation. Those are signed and cheksummed.
>
> * `--version-suffix rc1` -> adds the rc1 suffix to both - version in metadata and binary package names. This is used to prepare packages for PyPI.
>
>
> I updated the docs to mention that.
I would probably suggest we make this less error-prone, by doing `--rc-for-pypi` and have it do the right things
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427242623
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
Ah I see. There is no 'airflow version" command in those packages. And we can easily do it for airflow in the future - instead of reading from version.py we could generate an embedded version.txt file generated on the fly during package building and read the version from there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#issuecomment-630352556
Hey @ashb @kaxil - I think I corrected all the issues that were mentioned in rc1 thread :).
Please take a look :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#issuecomment-630097988
> > @ashb @kaxil -> I figured out that we have really two modes of release - one for "SVN" where we only rename the files with rc but the version inside is without the suffix, and the PyPI ones where we set version to "rc" and the package name also has "rc" as part of the name.
>
> Yes, as documented (albeit not clearly) https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow#ReleasingAirflow-PyPISnapshots(Optional).1
>
> > I solved it with two switches
> > ```
> > * `--artifact-name-suffix rc1`-> only changes name of the .tar.gz/.whl packages - used for SVN package generation. Those are signed and cheksummed.
> >
> > * `--version-suffix rc1` -> adds the rc1 suffix to both - version in metadata and binary package names. This is used to prepare packages for PyPI.
> > ```
> >
> >
> > I updated the docs to mention that.
>
> I would probably suggest we make this less error-prone, by doing `--rc-for-pypi` and have it do the right things
Good Idea. I am adding few last touches to the docs and I will update it - including also the -py3 only tag and some updates in the readmes as suggested.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#issuecomment-630361086
* Wheels are now only -py3 python type
* We have now two options:
```
--version-suffix-for-svn flag to generate rc candidate packages to upload to SVN or
--version-suffix-for-pypi flag to generate rc candidates for PyPI packages.
```
* Names of the commands were unified:
```
prepare-backport-readme Prepares backport packages readme files
prepare-backport-packages Prepares backport packages
```
* Added explicit information about Python 3.6+ in the README
* The `cncf.kubernetes` package is excluded (I added generic mechanism to exclude packages)
* Additional info in Google package about excluding BigQuery classes (I added a generic
mechanism to add additional informatio to generated README.md - separate per provider).
* Package description does not have double-backports in one line:
`Back-ported {package_prefix}.* package for Airflow 1.10.*`
* Updated BACKPORT_PACKAGES.md with clearer instructions about SVN/PyPI packages, additional explanation about removing leading 0s from months/days
Please take a look - once merged, I can release rc2 :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427238122
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
I think historically this was done so that even if someone installed the rc packages when they do `airflow version` it shows `rc` string in it. However maybe for backport packages it might not matter, thoughts @ashb ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#issuecomment-629877057
@ashb @kaxil -> I figured out that we have really two modes of release - one for "SVN" where we only rename the files with rc but the version inside is without the suffix, and the PyPI ones where we set version to "rc" and the package name also has "rc" as part of the name.
I solved it with two switches
* `--artifact-name-suffix rc1`-> only changes name of the .tar.gz/.whl packages - used for SVN package generation. Those are signed and cheksummed.
* `--version-suffix rc1` -> adds the rc1 suffix to both - version in metadata and binary package names. This is used to prepare packages for PyPI.
I updated the docs to mention that.
Note that the .tar.gz packages in SVN have also -bin.tar.gz added where the version uploaded to PyPI still keep the original .tar.gz name without -bin added. I think it's better to keep the names as generated by setup methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427236160
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
We don't have to. Let me explain.
When we use the `--version-suffix-for-svn rc1` we generate packages without any suffix and we rename them to contain the rc1 in the name. This is so that when we release we can rename the packages back.
However when you use the `--version-suffix-for-pypi`, the suffix is added dynamically during package generation - it's appended to the version of package.
The version is not read from version.py, it is read from the most recent PROVIDERS_CHANGES_YYYY_MM_DD.md -> YYYY.[M]M.[D]D
Theoretically, it can be a different version for each package being prepared (depending which PROVIDERS_CHANGES_YYYY_MM_DD.md is most recent (by the date).
It works like this:
1)
```
./breeze prepare-backport-readme -- 2020.05.20 PACKAGE1 PACKAGE2
```
Prepares the README.md and PROVIDERS_CHANGES_2020.05.20.md for the packages specified.
You commit that and only tag the release after it has been merged into master.
2) Then from that tagged release. when you want to generate those packages you run:
```
./breeze prepare-backport-packages --version-suffix-for-pypi rc1 -- PACKAGE1 PACKAGE2
```
and the packages are generated with 2020.5.20rc1 version. Under the hood `setup_backport_packages.py` is run with --version-suffix rc1 flag and it will simply append the suffix to the package version. There is no need to modify any sources temporarily, the suffix is added dynamically via result o this method call:
```
def get_package_release_version(provider_package_id: str, version_suffix: str = "") -> str:
"""
Returns release version including optional suffix.
:param provider_package_id: package id
:param version_suffix: optional suffix (rc1, rc2 etc).
:return:
"""
return get_latest_release(
get_package_path(provider_package_id=provider_package_id)).release_version + version_suffix
```
3) Later when you want to release final packages for PyPI you run (from the same tag/commit) :
```
./breeze prepare-backport-packages -- PACKAGE1 PACKAGE2
```
And no version suffix is added in this case.
4) For SVN packages you simply rename the packages already in SVN. They do not contain the rc1 suffix internally - only in the name.
Currently, I am only using it to generate all packages so there is no risk of mistakes but later I want to add additional switch `prepare-backport-packages --package-version 2020.05.20`. This will verify if the packages being released have all the 'CHANGES' prepared (via `prepare-backport-readme`) and that they are the latest ''CHANGES' and that this version has not already been released (checking for tag version in the git log and possibly pypi). But I will only be able to test it after we release officially the first wave of packages so I wanted to implement it later. This will protect against the accidental generation of the same package version as already released.
Eventually, I want to get rid of the manual operations and have all the steps automated within breeze so that you do not have to copy&paste commands from BACKPORT_PACKAGES.md, but this will require at least few iterations of releasing packages to get it consistently right.
I want the release process to take a few minutes and have it fully repeatable by anyone without setting up anything else but the local key and pypirc configuration. I want to be able to release quite a bit more often in the future - especially if we go AIP-8 route and split Airflow 2.0 into separate "core" and "providers" packages from day one. I treat backport packages as a testing ground for that approach to see if it is workable, so that I can propose it.
We can likely adopt it later to release airflow in the same way - without having to modify and discard version.py and add breeze command to do the same - but I planned to do it later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #8898: Correct generation and package generation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #8898:
URL: https://github.com/apache/airflow/pull/8898#discussion_r427236160
##########
File path: breeze
##########
@@ -1629,9 +1647,13 @@ function flag_push_docker_images() {
# Prints flags that control version of generated packages
function flag_version_suffix() {
echo "
--S, --version-suffix
- Adds optional suffix to the generated backport package version. It can be used to generate
- rc1/rc2 ... versions of the packages.
+-S, --version-suffix-for-pypi
+ Adds optional suffix to the version in the generated backport package. It can be used
+ to generate rc1/rc2 ... versions of the packages to be uploaded to PyPI.
Review comment:
We don't have to. Let me explain.
When we use the `--version-suffix-for-svn rc1` we generate packages without any suffix and we rename them to contain the rc1 in the name. This is so that when we release we can rename the packages back.
However when you use the `--version-suffix-for-pypi`, the suffix is added dynamically during package generation - it's appended to the version of package.
The version is not read from version.py, it is read from the most recent PROVIDERS_CHANGES_YYYY_MM_DD.md -> YYYY.[M]M.[D]D
Theoretically, it can be a different version for each package being prepared (depending which PROVIDERS_CHANGES_YYYY_MM_DD.md is most recent (by the date).
It works like this:
1) Prepare release notes
```
./breeze prepare-backport-readme -- 2020.05.20 PACKAGE1 PACKAGE2
```
Prepares the README.md and PROVIDERS_CHANGES_2020.05.20.md for the packages specified.
You commit that and only tag the release after it has been merged into master.
2) Then from that tagged release. when you want to generate those packages you run:
```
./breeze prepare-backport-packages --version-suffix-for-pypi rc1 -- PACKAGE1 PACKAGE2
```
and the packages are generated with 2020.5.20rc1 version. Under the hood `setup_backport_packages.py` is run with --version-suffix rc1 flag and it will simply append the suffix to the package version. There is no need to modify any sources temporarily, the suffix is added dynamically via result o this method call:
```
def get_package_release_version(provider_package_id: str, version_suffix: str = "") -> str:
"""
Returns release version including optional suffix.
:param provider_package_id: package id
:param version_suffix: optional suffix (rc1, rc2 etc).
:return:
"""
return get_latest_release(
get_package_path(provider_package_id=provider_package_id)).release_version + version_suffix
```
3) Later when you want to release final packages for PyPI you run (from the same tag/commit) :
```
./breeze prepare-backport-packages -- PACKAGE1 PACKAGE2
```
And no version suffix is added in this case.
4) For SVN packages you simply rename the packages already in SVN. They do not contain the rc1 suffix internally - only in the name.
Currently, I am only using it to generate all packages so there is no risk of mistakes but later I want to add additional switch `prepare-backport-packages --package-version 2020.05.20`. This will verify if the packages being released have all the 'CHANGES' prepared (via `prepare-backport-readme`) and that they are the latest ''CHANGES' and that this version has not already been released (checking for tag version in the git log and possibly pypi). But I will only be able to test it after we release officially the first wave of packages so I wanted to implement it later. This will protect against the accidental generation of the same package version as already released.
Eventually, I want to get rid of the manual operations and have all the steps automated within breeze so that you do not have to copy&paste commands from BACKPORT_PACKAGES.md, but this will require at least few iterations of releasing packages to get it consistently right.
I want the release process to take a few minutes and have it fully repeatable by anyone without setting up anything else but the local key and pypirc configuration. I want to be able to release quite a bit more often in the future - especially if we go AIP-8 route and split Airflow 2.0 into separate "core" and "providers" packages from day one. I treat backport packages as a testing ground for that approach to see if it is workable, so that I can propose it.
We can likely adopt it later to release airflow in the same way - without having to modify and discard version.py and add breeze command to do the same - but I planned to do it later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org