You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/25 12:22:27 UTC
[GitHub] [airflow] potiuk opened a new pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
potiuk opened a new pull request #13312:
URL: https://github.com/apache/airflow/pull/13312
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548913271
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,19 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are built from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``
* The ``-ci`` suffix is added for CI images
-* The ``-manifest`` is added for manifest images (see below for explanation of manifest images)
We also store (to increase speed of local build/pulls) python images that were used to build
the CI images. Each CI image, when built uses current python version of the base images. Those
python images are regularly updated (with bugfixes/security fixes), so for example python3.8 from
last week might be a different image than python3.8 today. Therefore whenever we push CI image
to airflow repository, we also push the python image that was used to build it this image is stored
-as ``apache/airflow:python-3.8-<BRANCH_OR_TAG>``.
+as ``apache/airflow:python<PYTHON_MAJOR_MINOR_VERSION>-<BRANCH_OR_TAG>``.
Review comment:
Yep. It is a fix :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#issuecomment-751336416
> May you please double-check on your latest force-push? Seems something is wrong: most fixes applied to address my earlier comments are gone now.
Silly me
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG edited a comment on pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
XD-DENG edited a comment on pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#issuecomment-751333871
> all should be fixed @XD-DENG :)
May you please double-check on your latest force-push? Seems something is wrong: most fixes applied to address my earlier comments are gone now.
UPDATE:
For example, the two highlighted parts below. They were pointed out and addressed earlier. But they are "back" now ;-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #13312:
URL: https://github.com/apache/airflow/pull/13312
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548962101
##########
File path: IMAGES.rst
##########
@@ -566,15 +570,24 @@ Image manifests
---------------
Together with the main CI images we also build and push image manifests. Those manifests are very small images
-that contain only results of the docker inspect for the image. This is in order to be able to
-determine very quickly if the image in the docker registry has changed a lot since the last time.
-Unfortunately docker registry (specifically DockerHub registry) has no anonymous way of querying image
-details via API, you need to download the image to inspect it. We overcame it in the way that
-always when we build the image we build a very small image manifest and push it to registry together
-with the main CI image. The tag for the manifest image is the same as for the image it refers
-to with added ``-manifest`` suffix. The manifest image for ``apache/airflow:master-python3.6-ci`` is named
+that contain only content of randomly generated file at the 'crucial' part of the CI image building
Review comment:
```suggestion
that contain only content of randomly generated file at the 'crucial' part of the CI image building.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548911988
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,19 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are built from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``
* The ``-ci`` suffix is added for CI images
-* The ``-manifest`` is added for manifest images (see below for explanation of manifest images)
Review comment:
IF it's removed because it's no longer applicable, section `Image manifests` should be removed as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#issuecomment-751304730
all should be fixed @XD-DENG :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548910392
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,20 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are build from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``,
+ ``3.8``
Review comment:
```suggestion
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#issuecomment-751333871
> all should be fixed @XD-DENG :)
May you please double-check on your latest force-push? Seems something is wrong: most fixes applied to address my earlier comments are gone now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548910439
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,20 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are build from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``,
Review comment:
```suggestion
* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548911518
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,19 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are built from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``
* The ``-ci`` suffix is added for CI images
-* The ``-manifest`` is added for manifest images (see below for explanation of manifest images)
Review comment:
Why this line is removed?
##########
File path: IMAGES.rst
##########
@@ -302,7 +303,7 @@ in the `<#ci-image-build-arguments>`_ chapter below.
Here just a few examples are presented which should give you general understanding of what you can customize.
-This builds the production image in version 3.7 with additional airflow extras from 1.10.10 Pypi package and
+This builds the production image in version 3.7 with additional airflow extras from 2.0.0 Pypi package and
Review comment:
```suggestion
This builds the production image in version 3.7 with additional airflow extras from 2.0.0 PyPI package and
```
An extremely minor change.
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,19 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are built from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``
* The ``-ci`` suffix is added for CI images
-* The ``-manifest`` is added for manifest images (see below for explanation of manifest images)
We also store (to increase speed of local build/pulls) python images that were used to build
the CI images. Each CI image, when built uses current python version of the base images. Those
python images are regularly updated (with bugfixes/security fixes), so for example python3.8 from
last week might be a different image than python3.8 today. Therefore whenever we push CI image
to airflow repository, we also push the python image that was used to build it this image is stored
-as ``apache/airflow:python-3.8-<BRANCH_OR_TAG>``.
+as ``apache/airflow:python<PYTHON_MAJOR_MINOR_VERSION>-<BRANCH_OR_TAG>``.
Review comment:
Just double-check: it was written as `python-3.8-<BRANCH_OR_TAG>` but now as `python<PYTHON_MAJOR_MINOR_VERSION>-<BRANCH_OR_TAG>`, i.e., a dash is removed. I assume this is a fix? (because among the tags on DockerHub and the samples below, I don't see dash between `python` and python version number)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#issuecomment-751337091
The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest master or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#issuecomment-751336588
It should be fixed now :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548914094
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,19 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are built from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``
* The ``-ci`` suffix is added for CI images
-* The ``-manifest`` is added for manifest images (see below for explanation of manifest images)
Review comment:
Good point. Mistake. We still have the manifest image but it contains a bit different data (randomly generated UUID at the point of building image that is a decision point on whether it is (way) faster to pull + build or just to build the image. I updated the description of the manifest image and restored the deleted line here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on a change in pull request #13312: Updates IMAGES documentation to reflect Airflow 2.0 changes
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on a change in pull request #13312:
URL: https://github.com/apache/airflow/pull/13312#discussion_r548889666
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,20 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are build from git tags
+ and they are "fixed" once built.
+* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.6``, ``3.7``,
+ ``3.8``
Review comment:
This seems not rendered correctly
##########
File path: IMAGES.rst
##########
@@ -39,19 +39,20 @@ The images are named as follows:
where:
-* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``, ``v1-10-test``, ``1.10.14``
- The ``master`` and ``v1-10-test`` labels are built from branches so they change over time. The ``1.10.*`` and in
- the future ``2.*`` labels are build from git tags and they are "fixed" once built.
-* ``PYTHON_MAJOR_MINOR_VERSION`` - version of python used to build the image. Examples: ``3.5``, ``3.7``
+* ``BRANCH_OR_TAG`` - branch or tag used when creating the image. Examples: ``master``,
+ ``v2-0-test``, ``v1-10-test``, ``2.0.0``. The ``master``, ``v1-10-test`` ``v2-0-test`` labels are
+ built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are build from git tags
Review comment:
```suggestion
built from branches so they change over time. The ``1.10.*`` and ``2.*`` labels are built from git tags
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org