You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/18 11:36:00 UTC
[GitHub] [airflow] potiuk opened a new pull request #11630: The .tar.gz provider packages are installable now.
potiuk opened a new pull request #11630:
URL: https://github.com/apache/airflow/pull/11630
The packages lacked setup.py and they could not be installed.
This change automatically generates setup.py for the packages and
adds them to the packages.
Fixes: #11546
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510811913
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) > 0 and sys.argv[1] == "--version-suffix-for-pypi":
Review comment:
Ah yeah I did test it and corrected it . But in the generated setup.py not the template :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510177229
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
Yep the line is not needed.
My intention (as I explained when submitted this PR) IS to commit those files. This way (as I explained there) it will be easier to track changes (for example in the dependencies) for those packages.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510056509
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,101 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{RELEASE_NO_LEADING_ZEROS}}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
+ sys.exit(1)
+ if sys.argv[1] == "--version-suffix":
+ if len(sys.argv) < 3:
+ print("ERROR! --version-suffix needs parameter!", file=sys.stderr)
+ sys.exit(1)
+ suffix = sys.argv[2]
+ sys.argv = [sys.argv[0]] + sys.argv[3:]
+ do_setup(version_suffix=suffix)
Review comment:
-1 from me, sorry -- hte dists we release, be it pypi or for svn, should be plain old non-configurable dists.
When releasing apache-airflow, we produce two versions, one for SVN and voting, and one for pypi.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510042435
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,101 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{RELEASE_NO_LEADING_ZEROS}}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
+ sys.exit(1)
+ if sys.argv[1] == "--version-suffix":
+ if len(sys.argv) < 3:
+ print("ERROR! --version-suffix needs parameter!", file=sys.stderr)
+ sys.exit(1)
+ suffix = sys.argv[2]
+ sys.argv = [sys.argv[0]] + sys.argv[3:]
+ do_setup(version_suffix=suffix)
Review comment:
This should not be in the generated setup.py -- it means you can't take the dist and run setup.py without again providing this suffix. It should instead be "baked" in, so that the setup.py in the dist is fully static and operates like a plain module.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510285790
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if sys.argv[1] == "--version-suffix-for-pypi":
Review comment:
```suggestion
if sys.argv[1:1] == ["--version-suffix-for-pypi"]:
```
(Or some other way of avoiding list index out of range when no args at all
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-714771390
Looks like finally GREEN! @ashb!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510809350
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) > 0 and sys.argv[1] == "--version-suffix-for-pypi":
Review comment:
Argh, sorry, you meant
```suggestion
if len(sys.argv) > 1 and sys.argv[1] == "--version-suffix-for-pypi":
```
```
Traceback (most recent call last):
File "setup.py", line 101, in <module>
if len(sys.argv) > 0 and sys.argv[1] == "--version-suffix-for-pypi":
IndexError: list index out of range
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-714658624
[The Workflow run](https://github.com/apache/airflow/actions/runs/322581761) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks$,^Build docs$,^Spell check docs$,^Backport packages$,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-713826137
Hey @ashb - any unresolved issues here?
I think we need to finally release the cncf.kubernetes backport and the 2.0 Alpha also depends on this one. After this one is merged, I need to regenerate READMES/versions as separate change. Any blockers here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-714489648
[The Workflow run](https://github.com/apache/airflow/actions/runs/322103166) is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510192825
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
But If you have good reasons, why we should not commit those files, I am happy to discuss and hear the reasoning . I basically followed the same pattern when I always generate and commit README files when I run "Prepare" step - mainly because this is the time I can dump the release history. But for setup it is pretty much convenience only and I could potentially move it to the package generation time.
Let me know what you think!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-711154970
CC: @mjpieters
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r507834565
##########
File path: provider_packages/prepare_provider_packages.py
##########
@@ -253,91 +228,59 @@ def get_package_release_version(provider_package_id: str,
backport_packages=backport_packages).release_version + version_suffix
-def do_setup_package_providers(provider_package_id: str,
- version_suffix: str,
- package_dependencies: Iterable[str],
- extras: Dict[str, List[str]],
- backport_packages: bool) -> None:
+def get_install_requirements(
+ provider_package_id: str,
+ backport_packages: bool) -> List[str]:
"""
- The main setup method for package.
+ Returns install requirements for the package.
:param provider_package_id: id of the provider package
- :param version_suffix: version suffix to be added to the release version (for example rc1)
- :param package_dependencies: dependencies of the package
- :param extras: extras of the package
- :param backport_packages: whether to prepare regular (False) or backport (True) packages
+ :param backport_packages: whether we are preparing backport packages
+ :return: install requirements of the package
"""
- setup.write_version()
- provider_package_name = get_pip_package_name(provider_package_id, backport_packages=backport_packages)
- package_name = f'{provider_package_name}'
- package_prefix = f'airflow.providers.{provider_package_id}'
- found_packages = find_packages()
- found_packages = [package for package in found_packages if package.startswith(package_prefix)]
+ dependencies = PROVIDERS_REQUIREMENTS[provider_package_id]
if backport_packages:
airflow_dependency = 'apache-airflow~=1.10' if provider_package_id != 'cncf.kubernetes' \
else 'apache-airflow>=1.10.12, <2.0.0'
else:
airflow_dependency = 'apache-airflow>=2.0.0a0'
+ install_requires = [airflow_dependency]
+ install_requires.extend(dependencies)
+ return install_requires
- install_requires = [
- airflow_dependency
+
+def get_setup_requirements() -> List[str]:
+ """
+ Returns setup requirements (common for all package for now).
+ :return: setup requirements
+ """
+ return [
+ 'bowler',
+ 'docutils',
+ 'gitpython',
+ 'setuptools',
+ 'semver',
Review comment:
It's all done. And even removed docutils and gitpython for now, since we are not using them. We might add them later if we split the docs per-provider #11423
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510822498
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) > 0 and sys.argv[1] == "--version-suffix-for-pypi":
Review comment:
Perfect, thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510048893
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,101 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{RELEASE_NO_LEADING_ZEROS}}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
+ sys.exit(1)
+ if sys.argv[1] == "--version-suffix":
+ if len(sys.argv) < 3:
+ print("ERROR! --version-suffix needs parameter!", file=sys.stderr)
+ sys.exit(1)
+ suffix = sys.argv[2]
+ sys.argv = [sys.argv[0]] + sys.argv[3:]
+ do_setup(version_suffix=suffix)
Review comment:
Sure I can bake it in for alphas, no problem. But I will leave an option to modify it manually for RCs. I use the same setup approach for backports and I prefer to add rc suffixes when generating the RCs rather than manually modifying the sources during the builds as it happens currently. Effectively we are releasing tio PyPI packages that haave slightly modified sources vs. the ones we actually put in SVN. I will rename the switch as "--pypi-rc-suffix" though to be perfectly clear about it's usage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-711170090
OK. I reviewed the generated packages and the licence files/notices/setup.py etc. The version that I pushed now is much closer to what I wanted to get with the provider packages, and the process of generating them/preparing per-provider configuration is fully automated and tested on our CI.
Once the build completes, (hopefully) you will find the packages generated as artifacts in the build and you will be able to review/test them yourself. Looking forward to reviews :). It is now super-easy to modify those generated packages and adapt them in a way that will fit us - separately for Backports and separately for the Regular provider packages.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r507643475
##########
File path: provider_packages/prepare_provider_packages.py
##########
@@ -253,91 +228,59 @@ def get_package_release_version(provider_package_id: str,
backport_packages=backport_packages).release_version + version_suffix
-def do_setup_package_providers(provider_package_id: str,
- version_suffix: str,
- package_dependencies: Iterable[str],
- extras: Dict[str, List[str]],
- backport_packages: bool) -> None:
+def get_install_requirements(
+ provider_package_id: str,
+ backport_packages: bool) -> List[str]:
"""
- The main setup method for package.
+ Returns install requirements for the package.
:param provider_package_id: id of the provider package
- :param version_suffix: version suffix to be added to the release version (for example rc1)
- :param package_dependencies: dependencies of the package
- :param extras: extras of the package
- :param backport_packages: whether to prepare regular (False) or backport (True) packages
+ :param backport_packages: whether we are preparing backport packages
+ :return: install requirements of the package
"""
- setup.write_version()
- provider_package_name = get_pip_package_name(provider_package_id, backport_packages=backport_packages)
- package_name = f'{provider_package_name}'
- package_prefix = f'airflow.providers.{provider_package_id}'
- found_packages = find_packages()
- found_packages = [package for package in found_packages if package.startswith(package_prefix)]
+ dependencies = PROVIDERS_REQUIREMENTS[provider_package_id]
if backport_packages:
airflow_dependency = 'apache-airflow~=1.10' if provider_package_id != 'cncf.kubernetes' \
else 'apache-airflow>=1.10.12, <2.0.0'
else:
airflow_dependency = 'apache-airflow>=2.0.0a0'
+ install_requires = [airflow_dependency]
+ install_requires.extend(dependencies)
+ return install_requires
- install_requires = [
- airflow_dependency
+
+def get_setup_requirements() -> List[str]:
+ """
+ Returns setup requirements (common for all package for now).
+ :return: setup requirements
+ """
+ return [
+ 'bowler',
Review comment:
Good catch. True! Copy&paste victim.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510152279
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,101 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{RELEASE_NO_LEADING_ZEROS}}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
+ sys.exit(1)
+ if sys.argv[1] == "--version-suffix":
+ if len(sys.argv) < 3:
+ print("ERROR! --version-suffix needs parameter!", file=sys.stderr)
+ sys.exit(1)
+ suffix = sys.argv[2]
+ sys.argv = [sys.argv[0]] + sys.argv[3:]
+ do_setup(version_suffix=suffix)
Review comment:
Looks like, while I was INTENDING to do this, I did not. It was a good call @ashb :).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-715268487
All green and corrected @ashb!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510171189
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
This line is the problem - it stops setup.py being used normally. Please make it possible to ruin without passing any args
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
I still don't understand why we need this - we don't commit the files, so when releasing rc to pypi why not just regenerate this file with the rc already in the version?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-713520279
Hey @ashb => just waiting for your OK here. I think I addressed everything :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #11630:
URL: https://github.com/apache/airflow/pull/11630
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510814859
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) > 0 and sys.argv[1] == "--version-suffix-for-pypi":
Review comment:
My bad, sorry. I reset, regenerated from scratch and it looks good now:
```
[jarek:~/code/airflow/airflow/providers/google] fix-setup-py-in-provider-packages(+42/-4)+* ± ls
ads BACKPORT_PROVIDERS_CHANGES_2020.10.05.md cloud config_templates __init__.py PROVIDERS_CHANGES_0.0.1.md setup.cfg suite
BACKPORT_PROVIDERS_CHANGES_2020.06.24.md BACKPORT_README.md common firebase marketing_platform README.md setup.py
[jarek:~/code/airflow/airflow/providers/google] fix-setup-py-in-provider-packages(+42/-4)+* ± python setup.py
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: no commands supplied
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510217644
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
Oh yeah. Only the ones from "provider_packages" :D. There whole `airflow` tree is ignored as well as all those is dynamically overwritten during packaging.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510214712
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
Nope, happy to commit them - just assumed cos we have a template file and you ignored them they we wouldn't be commiting them
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-714488295
I've added a bit more comprehensive tests to verify if suffixes is properly specified and also do this at readme/setup generation as well. so I think it's going to be good :). I iwll verify some of the pacakges after they get generated in the CI build as well :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510060616
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,101 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{RELEASE_NO_LEADING_ZEROS}}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
+ sys.exit(1)
+ if sys.argv[1] == "--version-suffix":
+ if len(sys.argv) < 3:
+ print("ERROR! --version-suffix needs parameter!", file=sys.stderr)
+ sys.exit(1)
+ suffix = sys.argv[2]
+ sys.argv = [sys.argv[0]] + sys.argv[3:]
+ do_setup(version_suffix=suffix)
Review comment:
And Just to clarify - for A1/B1 the version is already part of the sources, when readmes/setup files are generated. This parameter from the beginning was only supposed to be used for RC candidates that need different RC in sources and different in PyPI. I just simply clarify it by changing its name and adding extra comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r507643932
##########
File path: provider_packages/prepare_provider_packages.py
##########
@@ -253,91 +228,59 @@ def get_package_release_version(provider_package_id: str,
backport_packages=backport_packages).release_version + version_suffix
-def do_setup_package_providers(provider_package_id: str,
- version_suffix: str,
- package_dependencies: Iterable[str],
- extras: Dict[str, List[str]],
- backport_packages: bool) -> None:
+def get_install_requirements(
+ provider_package_id: str,
+ backport_packages: bool) -> List[str]:
"""
- The main setup method for package.
+ Returns install requirements for the package.
:param provider_package_id: id of the provider package
- :param version_suffix: version suffix to be added to the release version (for example rc1)
- :param package_dependencies: dependencies of the package
- :param extras: extras of the package
- :param backport_packages: whether to prepare regular (False) or backport (True) packages
+ :param backport_packages: whether we are preparing backport packages
+ :return: install requirements of the package
"""
- setup.write_version()
- provider_package_name = get_pip_package_name(provider_package_id, backport_packages=backport_packages)
- package_name = f'{provider_package_name}'
- package_prefix = f'airflow.providers.{provider_package_id}'
- found_packages = find_packages()
- found_packages = [package for package in found_packages if package.startswith(package_prefix)]
+ dependencies = PROVIDERS_REQUIREMENTS[provider_package_id]
if backport_packages:
airflow_dependency = 'apache-airflow~=1.10' if provider_package_id != 'cncf.kubernetes' \
else 'apache-airflow>=1.10.12, <2.0.0'
else:
airflow_dependency = 'apache-airflow>=2.0.0a0'
+ install_requires = [airflow_dependency]
+ install_requires.extend(dependencies)
+ return install_requires
- install_requires = [
- airflow_dependency
+
+def get_setup_requirements() -> List[str]:
+ """
+ Returns setup requirements (common for all package for now).
+ :return: setup requirements
+ """
+ return [
+ 'bowler',
+ 'docutils',
+ 'gitpython',
+ 'setuptools',
+ 'semver',
Review comment:
Same here. We only need semver now in the prepare* script. The setup.py we generate now from the jinja2 template is pretty simple and generic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#issuecomment-712303995
All right - this is the only blocker to release backport providers for "cncf.kubernetes" and alpha2 for airflow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510216454
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,105 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if len(sys.argv) == 1:
+ print("ERROR! Missing first param", file=sys.stderr)
Review comment:
I hope I only ignored the "provider_package" ones :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r507618470
##########
File path: provider_packages/prepare_provider_packages.py
##########
@@ -253,91 +228,59 @@ def get_package_release_version(provider_package_id: str,
backport_packages=backport_packages).release_version + version_suffix
-def do_setup_package_providers(provider_package_id: str,
- version_suffix: str,
- package_dependencies: Iterable[str],
- extras: Dict[str, List[str]],
- backport_packages: bool) -> None:
+def get_install_requirements(
+ provider_package_id: str,
+ backport_packages: bool) -> List[str]:
"""
- The main setup method for package.
+ Returns install requirements for the package.
:param provider_package_id: id of the provider package
- :param version_suffix: version suffix to be added to the release version (for example rc1)
- :param package_dependencies: dependencies of the package
- :param extras: extras of the package
- :param backport_packages: whether to prepare regular (False) or backport (True) packages
+ :param backport_packages: whether we are preparing backport packages
+ :return: install requirements of the package
"""
- setup.write_version()
- provider_package_name = get_pip_package_name(provider_package_id, backport_packages=backport_packages)
- package_name = f'{provider_package_name}'
- package_prefix = f'airflow.providers.{provider_package_id}'
- found_packages = find_packages()
- found_packages = [package for package in found_packages if package.startswith(package_prefix)]
+ dependencies = PROVIDERS_REQUIREMENTS[provider_package_id]
if backport_packages:
airflow_dependency = 'apache-airflow~=1.10' if provider_package_id != 'cncf.kubernetes' \
else 'apache-airflow>=1.10.12, <2.0.0'
else:
airflow_dependency = 'apache-airflow>=2.0.0a0'
+ install_requires = [airflow_dependency]
+ install_requires.extend(dependencies)
+ return install_requires
- install_requires = [
- airflow_dependency
+
+def get_setup_requirements() -> List[str]:
+ """
+ Returns setup requirements (common for all package for now).
+ :return: setup requirements
+ """
+ return [
+ 'bowler',
+ 'docutils',
+ 'gitpython',
+ 'setuptools',
+ 'semver',
Review comment:
```suggestion
```
Nor semver?
##########
File path: provider_packages/prepare_provider_packages.py
##########
@@ -253,91 +228,59 @@ def get_package_release_version(provider_package_id: str,
backport_packages=backport_packages).release_version + version_suffix
-def do_setup_package_providers(provider_package_id: str,
- version_suffix: str,
- package_dependencies: Iterable[str],
- extras: Dict[str, List[str]],
- backport_packages: bool) -> None:
+def get_install_requirements(
+ provider_package_id: str,
+ backport_packages: bool) -> List[str]:
"""
- The main setup method for package.
+ Returns install requirements for the package.
:param provider_package_id: id of the provider package
- :param version_suffix: version suffix to be added to the release version (for example rc1)
- :param package_dependencies: dependencies of the package
- :param extras: extras of the package
- :param backport_packages: whether to prepare regular (False) or backport (True) packages
+ :param backport_packages: whether we are preparing backport packages
+ :return: install requirements of the package
"""
- setup.write_version()
- provider_package_name = get_pip_package_name(provider_package_id, backport_packages=backport_packages)
- package_name = f'{provider_package_name}'
- package_prefix = f'airflow.providers.{provider_package_id}'
- found_packages = find_packages()
- found_packages = [package for package in found_packages if package.startswith(package_prefix)]
+ dependencies = PROVIDERS_REQUIREMENTS[provider_package_id]
if backport_packages:
airflow_dependency = 'apache-airflow~=1.10' if provider_package_id != 'cncf.kubernetes' \
else 'apache-airflow>=1.10.12, <2.0.0'
else:
airflow_dependency = 'apache-airflow>=2.0.0a0'
+ install_requires = [airflow_dependency]
+ install_requires.extend(dependencies)
+ return install_requires
- install_requires = [
- airflow_dependency
+
+def get_setup_requirements() -> List[str]:
+ """
+ Returns setup requirements (common for all package for now).
+ :return: setup requirements
+ """
+ return [
+ 'bowler',
Review comment:
We shouldn't need bowler in the build sdists
```suggestion
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #11630: The .tar.gz provider packages are installable now.
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #11630:
URL: https://github.com/apache/airflow/pull/11630#discussion_r510302296
##########
File path: provider_packages/SETUP_TEMPLATE.py.jinja2
##########
@@ -0,0 +1,102 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE\
+# OVERWRITTEN WHEN RUNNING
+#
+# ./breeze prepare-provider-readme
+#
+# IF YOU WANT TO MODIFY IT, YOU SHOULD MODIFY THE TEMPLATE
+# `SETUP_TEMPLATE.py.jinja2` IN the `provider_packages` DIRECTORY
+
+"""Setup.py for the {{ PACKAGE_PIP_NAME }} package."""
+
+import io
+import logging
+import os
+import sys
+
+from os.path import dirname
+from setuptools import find_packages, setup
+
+logger = logging.getLogger(__name__)
+
+version = '{{ RELEASE_NO_LEADING_ZEROS }}{{VERSION_SUFFIX }}'
+
+my_dir = dirname(__file__)
+
+try:
+ with io.open(os.path.join(my_dir,
+ '{{ PROVIDER_PATH }}/{{ README_FILE }}'),
+ encoding='utf-8') as f:
+ long_description = f.read()
+except FileNotFoundError:
+ long_description = ''
+
+
+def do_setup(version_suffix_for_pypi=''):
+ """
+ Perform the package setup.
+ """
+ setup(
+ name='{{ PACKAGE_PIP_NAME }}',
+ description='{{ PROVIDER_TYPE }} package {{ PACKAGE_PIP_NAME }} for Apache Airflow',
+ long_description=long_description,
+ long_description_content_type='text/markdown',
+ license='Apache License 2.0',
+ version=version + version_suffix_for_pypi,
+ packages=find_packages(
+ include=['airflow.providers.{{ PROVIDER_PACKAGE_ID }}*']),
+ zip_safe=False,
+ install_requires={{ INSTALL_REQUIREMENTS }},
+ setup_requires={{ SETUP_REQUIREMENTS }},
+ extras_require={{ EXTRAS_REQUIREMENTS }},
+ classifiers=[
+ 'Development Status :: 5 - Production/Stable',
+ 'Environment :: Console',
+ 'Environment :: Web Environment',
+ 'Intended Audience :: Developers',
+ 'Intended Audience :: System Administrators',
+ 'License :: OSI Approved :: Apache Software License',
+ 'Programming Language :: Python :: 3.6',
+ 'Programming Language :: Python :: 3.7',
+ 'Programming Language :: Python :: 3.8',
+ 'Topic :: System :: Monitoring',
+ ],
+ author='Apache Software Foundation',
+ author_email='dev@airflow.apache.org',
+ url='http://airflow.apache.org/',
+ download_url=(
+ 'https://dist.apache.org/repos/dist/release/airflow/{{ PROVIDERS_FOLDER }}' + version),
+ python_requires='~=3.6',
+ )
+
+#
+# Note that --version-suffix-for-pypi should only be used in case we generate RC packages for PyPI
+# Those packages should have actual RC version in order to be published even if source version
+# should be the final one.
+#
+if __name__ == "__main__":
+ suffix=''
+ if sys.argv[1] == "--version-suffix-for-pypi":
Review comment:
My bad. Working with templates :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org