You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/17 21:04:44 UTC
[GitHub] [airflow] ashb opened a new pull request #12417: Move setup properties out of setup.py in to setup.cfg
ashb opened a new pull request #12417:
URL: https://github.com/apache/airflow/pull/12417
I've moved all the ones that are "static" -- any form of dynamic or interpolated values are left in setup.py
If a value is passed as n kwrg to setup and in setup.cfg, the kwarg wins out.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#discussion_r525859346
##########
File path: .github/workflows/ci.yml
##########
@@ -788,6 +789,7 @@ jobs:
with:
path: ".build/bin"
key: "bin-${{ env.cache-name }}-${{ github.job }}-${{ hashFiles('setup.py') }}\
+-${{ hashFiles('setup.cfg') }}\
Review comment:
Actually here we can remove both setup.py and setup.cfg. The ./buld/bin content only depends on the version of tools used (helm//kind/kubectl) and it does not depend on setup.py nor setup.cfg in any way. That was probably a copy&paste problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#discussion_r526030296
##########
File path: setup.cfg
##########
@@ -44,6 +46,111 @@ license_files =
licenses/LICENSE-python-nvd3.txt
licenses/LICENSE-python-slugify.txt
# End of licences generated automatically
+classifiers =
+ Development Status :: 5 - Production/Stable
+ Environment :: Console
+ Environment :: Web Environment
+ Intended Audience :: Developers
+ Intended Audience :: System Administrators
+ License :: OSI Approved :: Apache Software License
+ Programming Language :: Python :: 3.6
+ Programming Language :: Python :: 3.7
+ Programming Language :: Python :: 3.8
+ Topic :: System :: Monitoring
+project_urls =
+ Documentation=https://airflow.apache.org/docs/
+ Bug Tracker=https://github.com/apache/airflow/issues
+ Source Code=https://github.com/apache/airflow
+
+[options]
+zip_safe = False
+include_package_data = True
+python_requires = ~=3.6
+packages = find:
+setup_requires =
+ bowler
+ docutils
+ gitpython
+ setuptools
+ wheel
+#####################################################################################################
+# IMPORTANT NOTE!!!!!!!!!!!!!!!
+# IF you are removing dependencies from this list, please make sure that you also increase
+# DEPENDENCIES_EPOCH_NUMBER in the Dockerfile.ci
+#####################################################################################################
+install_requires =
+ alembic>=1.2, <2.0
+ argcomplete~=1.10
+ attrs>=20.0, <21.0
+ cached_property~=1.5
+ # cattrs >= 1.1.0 dropped support for Python 3.6
+ cattrs>=1.0, <1.1.0;python_version<="3.6"
+ cattrs>=1.0, <2.0;python_version>"3.6"
+ colorlog==4.0.2
+ connexion[swagger-ui,flask]>=2.6.0,<3
+ croniter>=0.3.17, <0.4
+ cryptography>=0.9.3
+ dill>=0.2.2, <0.4
+ flask>=1.1.0, <2.0
+ flask-appbuilder~=3.1.1
+ flask-caching>=1.5.0, <2.0.0
+ flask-login>=0.3, <0.5
+ flask-swagger==0.2.13
+ flask-wtf>=0.14.3, <0.15
+ funcsigs>=1.0.0, <2.0.0
+ graphviz>=0.12
+ gunicorn>=19.5.0, <20.0
+ iso8601>=0.1.12
+ jinja2>=2.10.1, <2.12.0
+ json-merge-patch==0.2
+ jsonschema~=3.0
+ lazy_object_proxy~=1.3
+ lockfile>=0.12.2
+ markdown>=2.5.2, <4.0
+ markupsafe>=1.1.1, <2.0
+ marshmallow-oneofschema>=2.0.1
+ pandas>=0.17.1, <2.0
+ pendulum~=2.0
+ pep562~=1.0;python_version<"3.7"
+ psutil>=4.2.0, <6.0.0
+ pygments>=2.0.1, <3.0
+ python-daemon>=2.1.1
+ python-dateutil>=2.3, <3
+ python-nvd3~=0.15.0
+ python-slugify>=3.0.0,<5.0
+ requests>=2.20.0, <3
+ rich==9.2.0
+ setproctitle>=1.1.8, <2
+ sqlalchemy>=1.3.18, <2
+ sqlalchemy_jsonfield~=0.9
+ tabulate>=0.7.5, <0.9
+ tenacity~=6.2.0
+ termcolor>=1.1.0
+ thrift>=0.9.2
+ typing;python_version<"3.6"
+ typing-extensions>=3.7.4;python_version<"3.8"
+ tzlocal>=1.4,<2.0.0
+ unicodecsv>=0.14.1
+ werkzeug~=1.0, >=1.0.1
+
+[options.packages.find]
+exclude =
+ airflow.providers
Review comment:
I just did a lift-and-shift of the `find_packages()` call from setup.py. You are right though.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#issuecomment-729489939
The PR needs to run all tests because it modifies core of Airflow! Please rebase it to latest master or ask committer to re-run it!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#issuecomment-729245379
Updated the sort checking in pre-commit. Setup tool provides a `read_configuration` function we can use.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#discussion_r525871825
##########
File path: .github/workflows/ci.yml
##########
@@ -788,6 +789,7 @@ jobs:
with:
path: ".build/bin"
key: "bin-${{ env.cache-name }}-${{ github.job }}-${{ hashFiles('setup.py') }}\
+-${{ hashFiles('setup.cfg') }}\
Review comment:
Will do. (I didn't check the logic here, just saw setup.py being fingerprinted and added setup.cfg)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #12417:
URL: https://github.com/apache/airflow/pull/12417
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#issuecomment-729232021
I had a quick look before going to bed - and there is nothing that I would not like in this change @ashb :). As long as all the cases we had (installing locally, installing via Github llnk, installing via PyPI) work - this is all fine.
There are a few (or maybe just one - INSTALL_REQUIREMENTS order - check_install_and_setup_requires in `pre_commit_check_order_setup.py` ) static check/pre-commit that you will find that need updating. I think that's all - but we should also carefully look at the artifacts generated by the PR to be sure of that (especially whether the docker images built contain all providers and whether the provider packages have the right dependencies etc.
Nothing controversial in this PR really :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#issuecomment-729216429
I'm just moving across install_reuirements too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#discussion_r525851493
##########
File path: docs/installation.rst
##########
@@ -27,7 +27,7 @@ Getting Airflow
Airflow is published as ``apache-airflow`` package in PyPI. Installing it however might be sometimes tricky
because Airflow is a bit of both a library and application. Libraries usually keep their dependencies open and
applications usually pin them, but we should do neither and both at the same time. We decided to keep
-our dependencies as open as possible (in ``setup.py``) so users can install different version of libraries
+our dependencies as open as possible (in ``setup.cfg``) so users can install different version of libraries
Review comment:
Should not that be (for now at least) both setup.py and setup.cfg?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#discussion_r525857365
##########
File path: setup.cfg
##########
@@ -44,6 +46,111 @@ license_files =
licenses/LICENSE-python-nvd3.txt
licenses/LICENSE-python-slugify.txt
# End of licences generated automatically
+classifiers =
+ Development Status :: 5 - Production/Stable
+ Environment :: Console
+ Environment :: Web Environment
+ Intended Audience :: Developers
+ Intended Audience :: System Administrators
+ License :: OSI Approved :: Apache Software License
+ Programming Language :: Python :: 3.6
+ Programming Language :: Python :: 3.7
+ Programming Language :: Python :: 3.8
+ Topic :: System :: Monitoring
+project_urls =
+ Documentation=https://airflow.apache.org/docs/
+ Bug Tracker=https://github.com/apache/airflow/issues
+ Source Code=https://github.com/apache/airflow
+
+[options]
+zip_safe = False
+include_package_data = True
+python_requires = ~=3.6
+packages = find:
+setup_requires =
+ bowler
+ docutils
+ gitpython
+ setuptools
+ wheel
+#####################################################################################################
+# IMPORTANT NOTE!!!!!!!!!!!!!!!
+# IF you are removing dependencies from this list, please make sure that you also increase
+# DEPENDENCIES_EPOCH_NUMBER in the Dockerfile.ci
+#####################################################################################################
+install_requires =
+ alembic>=1.2, <2.0
+ argcomplete~=1.10
+ attrs>=20.0, <21.0
+ cached_property~=1.5
+ # cattrs >= 1.1.0 dropped support for Python 3.6
+ cattrs>=1.0, <1.1.0;python_version<="3.6"
+ cattrs>=1.0, <2.0;python_version>"3.6"
+ colorlog==4.0.2
+ connexion[swagger-ui,flask]>=2.6.0,<3
+ croniter>=0.3.17, <0.4
+ cryptography>=0.9.3
+ dill>=0.2.2, <0.4
+ flask>=1.1.0, <2.0
+ flask-appbuilder~=3.1.1
+ flask-caching>=1.5.0, <2.0.0
+ flask-login>=0.3, <0.5
+ flask-swagger==0.2.13
+ flask-wtf>=0.14.3, <0.15
+ funcsigs>=1.0.0, <2.0.0
+ graphviz>=0.12
+ gunicorn>=19.5.0, <20.0
+ iso8601>=0.1.12
+ jinja2>=2.10.1, <2.12.0
+ json-merge-patch==0.2
+ jsonschema~=3.0
+ lazy_object_proxy~=1.3
+ lockfile>=0.12.2
+ markdown>=2.5.2, <4.0
+ markupsafe>=1.1.1, <2.0
+ marshmallow-oneofschema>=2.0.1
+ pandas>=0.17.1, <2.0
+ pendulum~=2.0
+ pep562~=1.0;python_version<"3.7"
+ psutil>=4.2.0, <6.0.0
+ pygments>=2.0.1, <3.0
+ python-daemon>=2.1.1
+ python-dateutil>=2.3, <3
+ python-nvd3~=0.15.0
+ python-slugify>=3.0.0,<5.0
+ requests>=2.20.0, <3
+ rich==9.2.0
+ setproctitle>=1.1.8, <2
+ sqlalchemy>=1.3.18, <2
+ sqlalchemy_jsonfield~=0.9
+ tabulate>=0.7.5, <0.9
+ tenacity~=6.2.0
+ termcolor>=1.1.0
+ thrift>=0.9.2
+ typing;python_version<"3.6"
+ typing-extensions>=3.7.4;python_version<"3.8"
+ tzlocal>=1.4,<2.0.0
+ unicodecsv>=0.14.1
+ werkzeug~=1.0, >=1.0.1
+
+[options.packages.find]
+exclude =
+ airflow.providers
Review comment:
I believe find: uses "find_packages" not the namespace ones (there is a separate find_namespace:)? So we probably do not need those "excludes" here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#discussion_r525851088
##########
File path: .github/workflows/ci.yml
##########
@@ -779,7 +779,8 @@ jobs:
cache-name: cache-kubernetes-tests-v7-master
with:
path: ".build/.kubernetes_venv*"
- key: "venv-${{ env.cache-name }}-${{ github.job }}-${{ hashFiles('setup.py') }}\
+ key: "venv-${{ env.cache-name }}-${{ github.job }}-${{ hashFiles('setup.py')\
+-${{ hashFiles('setup.cfg') }}\
Review comment:
Good point!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #12417: Move setup properties out of setup.py in to setup.cfg
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #12417:
URL: https://github.com/apache/airflow/pull/12417#issuecomment-729244698
Yeah the possible contriversal stuff (version detection, extra requires) needed more brain power to think of that I have tonight. So I'm doing sort of mechanical easy-to-do changes :D
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org