You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/06/14 07:58:48 UTC
[GitHub] [airflow] potiuk opened a new pull request #16422: Add rules describing SemVer approach for various Airflow packages.
potiuk opened a new pull request #16422:
URL: https://github.com/apache/airflow/pull/16422
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #16422: Add rules describing SemVer approach for various Airflow packages.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #16422:
URL: https://github.com/apache/airflow/pull/16422#discussion_r650562256
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, airflow supports strict [SemVer](https://semver.org/) approach for all packages released.
Review comment:
```suggestion
As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Change in minimal dependency versions of Airflow is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are any requirements between providers and Airflow packages,
+ they are present in providers as `install_requires` limitations. We aim to keep backwards
+ compatibility of providers with all previously released Airflow 2 versions but
+ there will be sometimes breaking changes that might make some, or all
+ providers depend on minimal Airflow version. Change of the minimal MAJOR.MINOR version of Airflow
+ is a breaking change for provider, because installing the new provider might automatically
+ upgrade Airflow (which might be undesired).
Review comment:
```suggestion
upgrade Airflow (which might be undesired side effect of upgrading provider).
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Change in minimal dependency versions of Airflow is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are any requirements between providers and Airflow packages,
Review comment:
```suggestion
with `Airflow 2.1.1`. If there are limits of cross-dependencies between providers and Airflow packages,
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Change in minimal dependency versions of Airflow is not a breaking change on its own.
Review comment:
```suggestion
Changing limits for versions of Airflow dependencies is not a breaking change on its own.
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Change in minimal dependency versions of Airflow is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are any requirements between providers and Airflow packages,
+ they are present in providers as `install_requires` limitations. We aim to keep backwards
+ compatibility of providers with all previously released Airflow 2 versions but
+ there will be sometimes breaking changes that might make some, or all
+ providers depend on minimal Airflow version. Change of the minimal MAJOR.MINOR version of Airflow
+ is a breaking change for provider, because installing the new provider might automatically
+ upgrade Airflow (which might be undesired).
+* **Airflow Helm Chart**: SemVer rules apply to changes in the chart only. SemVer MAJOR and MINOR
+ versions for the chart are independent from apache-airflow versions. We aim to keep backwards
+ compatibility of the Helm Chart with all released Airflow 2 versions, but some new features might
+ only work starting from specific Airlfow releases. At some point in time we might limit the Helm
Review comment:
```suggestion
only work starting from specific Airlfow releases. We might however limit the Helm
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Change in minimal dependency versions of Airflow is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are any requirements between providers and Airflow packages,
+ they are present in providers as `install_requires` limitations. We aim to keep backwards
+ compatibility of providers with all previously released Airflow 2 versions but
+ there will be sometimes breaking changes that might make some, or all
+ providers depend on minimal Airflow version. Change of the minimal MAJOR.MINOR version of Airflow
+ is a breaking change for provider, because installing the new provider might automatically
+ upgrade Airflow (which might be undesired).
+* **Airflow Helm Chart**: SemVer rules apply to changes in the chart only. SemVer MAJOR and MINOR
+ versions for the chart are independent from apache-airflow versions. We aim to keep backwards
Review comment:
```suggestion
versions for the chart are independent from the Airflow version. We aim to keep backwards
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Change in minimal dependency versions of Airflow is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are any requirements between providers and Airflow packages,
+ they are present in providers as `install_requires` limitations. We aim to keep backwards
+ compatibility of providers with all previously released Airflow 2 versions but
+ there will be sometimes breaking changes that might make some, or all
+ providers depend on minimal Airflow version. Change of the minimal MAJOR.MINOR version of Airflow
Review comment:
```suggestion
providers, to have minimum Airflow version specified. Change of that minimum supported Airflow version
```
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Changing limits for versions of Airflow dependencies is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are limits of cross-dependencies between providers and Airflow packages,
+ they are present in providers as `install_requires` limitations. We aim to keep backwards
+ compatibility of providers with all previously released Airflow 2 versions but
+ there will be sometimes breaking changes that might make some, or all
+ providers, to have minimum Airflow version specified. Change of that minimum supported Airflow version
+ is a breaking change for provider, because installing the new provider might automatically
+ upgrade Airflow (which might be undesired side effect of upgrading provider).
+* **Airflow Helm Chart**: SemVer rules apply to changes in the chart only. SemVer MAJOR and MINOR
Review comment:
Correct. I have not written at all which versions of Airfllow are supported by Helm Chart (it's "current" situation and I want this description to only describes rules). Currently Helm Chart supports 1.10 and 2.0 but it might change in the future (and I guess even in the near future) when we stop supporting 1.10 for example. we might also stop supporting 2.0 (similarly as we did with providers).
Is there anything there that tells otherwise?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #16422: Add rules describing SemVer approach for various Airflow packages.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #16422:
URL: https://github.com/apache/airflow/pull/16422#issuecomment-860363289
The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #16422: Add rules describing SemVer approach for various Airflow packages.
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #16422:
URL: https://github.com/apache/airflow/pull/16422#discussion_r650563765
##########
File path: README.md
##########
@@ -76,6 +77,35 @@ Airflow is not a streaming solution, but it is often used to process real-time d
- **Elegant**: Airflow pipelines are lean and explicit. Parameterizing your scripts is built into the core of Airflow using the powerful **Jinja** templating engine.
- **Scalable**: Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers.
+## Semantic versioning
+
+As of Airflow 2.0.0, we support strict [SemVer](https://semver.org/) approach for all packages released.
+
+There are few specific rules that we agreed to, that define details of versioning of the different
+packages:
+
+* **Airflow**: SemVer rules apply to core airflow only (excludes any changes to providers).
+ Changing limits for versions of Airflow dependencies is not a breaking change on its own.
+* **Airflow Providers**: SemVer rules apply to changes in the particular provider's code only.
+ SemVer MAJOR and MINOR versions for the packages are independent from Airflow version.
+ For example `google 4.1.0` and `amazon 3.0.3` providers can happily be installed
+ with `Airflow 2.1.1`. If there are limits of cross-dependencies between providers and Airflow packages,
+ they are present in providers as `install_requires` limitations. We aim to keep backwards
+ compatibility of providers with all previously released Airflow 2 versions but
+ there will be sometimes breaking changes that might make some, or all
+ providers, to have minimum Airflow version specified. Change of that minimum supported Airflow version
+ is a breaking change for provider, because installing the new provider might automatically
+ upgrade Airflow (which might be undesired side effect of upgrading provider).
+* **Airflow Helm Chart**: SemVer rules apply to changes in the chart only. SemVer MAJOR and MINOR
Review comment:
Hełm Char supports Airflow 1.10+and 2.0+. See:
http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/helm-chart/latest/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #16422: Add rules describing SemVer approach for various Airflow packages.
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #16422:
URL: https://github.com/apache/airflow/pull/16422
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org