You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/12 16:07:20 UTC
[GitHub] [airflow] potiuk opened a new pull request #22209: Switched cgroupspy to vendored version
potiuk opened a new pull request #22209:
URL: https://github.com/apache/airflow/pull/22209
This is part of the effort needed to implement Python 3.10
compatibility: #22050
This is based on #22204, #22206, #22207 and #22208 so review only last
commit please.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826120030
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
Review comment:
```suggestion
The `airflow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826121162
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
Review comment:
```suggestion
the `_vendor` package.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #22209:
URL: https://github.com/apache/airflow/pull/22209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826121618
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
+
+All Vendored libraries must follow these rules:
+
+1. Vendored libraries must be pure Python--no compiling.
Review comment:
Can you elaborate this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826141331
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
+
+All Vendored libraries must follow these rules:
+
+1. Vendored libraries must be pure Python--no compiling.
Review comment:
This means that we should not vendor-in code that woudl requrire us to produce platform/architecture specific packaes (such as numpy etc.). Otherwise it will require from us to build multiple binary wheels for `airflow` core - where wa only need one pure-python only.
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
+
+All Vendored libraries must follow these rules:
+
+1. Vendored libraries must be pure Python--no compiling.
Review comment:
This means that we should not vendor-in code that woudl requrire us to produce platform/architecture specific packages (such as numpy etc.). Otherwise it will require from us to build multiple binary wheels for `airflow` core - where wa only need one pure-python only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#issuecomment-1067026131
BTW. The process description is taken from the https://github.com/apache/airflow/pull/22204 so it should be corrected there :).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826141331
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
+
+All Vendored libraries must follow these rules:
+
+1. Vendored libraries must be pure Python--no compiling.
Review comment:
This means that we should not vendor-in code that woudl requrire us to produce platform/architecture specific packages (such as numpy etc.). Otherwise it will require from us to build multiple binary wheels for `airflow` core - where we only need one pure-python only today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826142677
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
+
+All Vendored libraries must follow these rules:
+
+1. Vendored libraries must be pure Python--no compiling.
Review comment:
Can't find shorter way to describe it :). But seems too long to add it to the instructions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#issuecomment-1074403967
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#issuecomment-1073289792
This is the last-but-one to make Airflow Python 3.10 compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#issuecomment-1074361423
🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#discussion_r826122438
##########
File path: airflow/_vendor/README.md
##########
@@ -0,0 +1,37 @@
+# Vendor package
+
+## What vendored packages are for
+
+The `airlfow._vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
+because authors of the packages do not have time to modify them themselves. This is often temporary
+and once the packages implement fixes that we need, and then we remove the packages from
+the _vendor package.
+
+All Vendored libraries must follow these rules:
+
+1. Vendored libraries must be pure Python--no compiling.
+2. Source code for the libary is included in this directory.
+3. License must be included in this repo and in the [LICENSE](../../LICENSE) file and in the
+ [licenses](../../licenses) folder.
+4. Requirements of the library become requirements of airflow core.
+5. Version of the library should be included in the [vendor.md](vendor.md) file.
+6. No modifications to the library may be made in the initial commit.
+7. Apply the fixes necessary to use the vendored library as separate commits - each package separately,
+ so that they can be cherry-picked later if we upgrade the vendored package. Changes to airflow code to
+ use the vendored packages should be applied as separate commits/PRs.
+8. The `_vendor` packages should be excluded from any refactorings, static checks and automated fixes.
+
+## Adding and upgrading a vendored package
+
+Way to vendor a library or update a version:
+
+1. Update ``vendor.txt`` with the library, version, and SHA256 (`pypi` provides gashes as of recently)
Review comment:
```suggestion
1. Update ``vendor.txt`` with the library, version, and SHA256 (`pypi` provides hashes as of recently)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #22209: Switched cgroupspy to vendored version
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #22209:
URL: https://github.com/apache/airflow/pull/22209#issuecomment-1067027577
> BTW. The process description is taken from the #22204 so it should be corrected there :).
But I can cherry-pick those fixes no worres (just the approval needs to be given at #22204.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org