You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/08 22:14:40 UTC
[GitHub] [airflow] ashb opened a new pull request #12931: Apply labels to Docker images in a single instruction
ashb opened a new pull request #12931:
URL: https://github.com/apache/airflow/pull/12931
While looking at the build logs for something else I noticed this
oddity at the end of the CI logs:
```
Tue, 08 Dec 2020 21:20:19 GMT Step 125/135 : LABEL org.apache.airflow.distro="debian"
...
Tue, 08 Dec 2020 21:21:14 GMT Step 133/135 : LABEL org.apache.airflow.commitSha=${COMMIT_SHA}
Tue, 08 Dec 2020 21:21:14 GMT ---> Running in 1241a5f6cdb7
Tue, 08 Dec 2020 21:21:21 GMT Removing intermediate container 1241a5f6cdb7
```
Applying all the labels took 1m2s! Hopefully applying these in a single
layer/command should speed things up.
A less extreme example still took 43s
```
Tue, 08 Dec 2020 20:44:40 GMT Step 125/135 : LABEL org.apache.airflow.distro="debian"
...
Tue, 08 Dec 2020 20:45:18 GMT Step 133/135 : LABEL org.apache.airflow.commitSha=${COMMIT_SHA}
Tue, 08 Dec 2020 20:45:18 GMT ---> Running in dc601207dbcb
Tue, 08 Dec 2020 20:45:23 GMT Removing intermediate container dc601207dbcb
Tue, 08 Dec 2020 20:45:23 GMT ---> 5aae5dd0f702
```
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on a change in pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on a change in pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#discussion_r538860543
##########
File path: Dockerfile
##########
@@ -260,19 +260,19 @@ RUN AIRFLOW_SITE_PACKAGE="/root/.local/lib/python${PYTHON_MAJOR_MINOR_VERSION}/s
RUN find /root/.local -executable -print0 | xargs --null chmod g+x && \
find /root/.local -print0 | xargs --null chmod g+rw
-LABEL org.apache.airflow.distro="debian"
-LABEL org.apache.airflow.distro.version="buster"
-LABEL org.apache.airflow.module="airflow"
-LABEL org.apache.airflow.component="airflow"
-LABEL org.apache.airflow.image="airflow-build-image"
ARG BUILD_ID
ENV BUILD_ID=${BUILD_ID}
ARG COMMIT_SHA
ENV COMMIT_SHA=${COMMIT_SHA}
-LABEL org.apache.airflow.buildImage.buildId=${BUILD_ID}
-LABEL org.apache.airflow.buildImage.commitSha=${COMMIT_SHA}
+
+LABEL org.apache.airflow.distro.version="buster" \
Review comment:
Looks like you're not longer adding the "debian" label. Is that intentional?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on a change in pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on a change in pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#discussion_r538860543
##########
File path: Dockerfile
##########
@@ -260,19 +260,19 @@ RUN AIRFLOW_SITE_PACKAGE="/root/.local/lib/python${PYTHON_MAJOR_MINOR_VERSION}/s
RUN find /root/.local -executable -print0 | xargs --null chmod g+x && \
find /root/.local -print0 | xargs --null chmod g+rw
-LABEL org.apache.airflow.distro="debian"
-LABEL org.apache.airflow.distro.version="buster"
-LABEL org.apache.airflow.module="airflow"
-LABEL org.apache.airflow.component="airflow"
-LABEL org.apache.airflow.image="airflow-build-image"
ARG BUILD_ID
ENV BUILD_ID=${BUILD_ID}
ARG COMMIT_SHA
ENV COMMIT_SHA=${COMMIT_SHA}
-LABEL org.apache.airflow.buildImage.buildId=${BUILD_ID}
-LABEL org.apache.airflow.buildImage.commitSha=${COMMIT_SHA}
+
+LABEL org.apache.airflow.distro.version="buster" \
Review comment:
Looks like you're no longer adding the "debian" label. Is that intentional?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#issuecomment-741162902
With this change:
```
Tue, 08 Dec 2020 22:22:18 GMT Step 125/127 : LABEL org.apache.airflow.distro="debian" org.apache.airflow.distro.version="buster" org.apache.airflow.module="airflow" org.apache.airflow.component="airflow" org.apache.airflow.image="airflow-ci" org.apache.airflow.uid="0" org.apache.airflow.gid="0" org.apache.airflow.buildId=${BUILD_ID} org.apache.airflow.commitSha=${COMMIT_SHA}
Tue, 08 Dec 2020 22:22:18 GMT ---> Running in 8d7218d319bc
Tue, 08 Dec 2020 22:22:25 GMT Removing intermediate container 8d7218d319bc
Tue, 08 Dec 2020 22:22:25 GMT ---> 8fd2bc64723b
```
Just seven seconds. (Still slow for what it's doing, but much much quicker than it was)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #12931:
URL: https://github.com/apache/airflow/pull/12931
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#issuecomment-741856253
It makes a difference when the docker engine is somewhat remote or has slow filesystem. I thnk we can do the same with env-variables and ARGS. it's nicer to keep them separate, but they would suffer the same delays. I will give it a shot after 2.0rc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#discussion_r538864117
##########
File path: Dockerfile
##########
@@ -260,19 +260,19 @@ RUN AIRFLOW_SITE_PACKAGE="/root/.local/lib/python${PYTHON_MAJOR_MINOR_VERSION}/s
RUN find /root/.local -executable -print0 | xargs --null chmod g+x && \
find /root/.local -print0 | xargs --null chmod g+rw
-LABEL org.apache.airflow.distro="debian"
-LABEL org.apache.airflow.distro.version="buster"
-LABEL org.apache.airflow.module="airflow"
-LABEL org.apache.airflow.component="airflow"
-LABEL org.apache.airflow.image="airflow-build-image"
ARG BUILD_ID
ENV BUILD_ID=${BUILD_ID}
ARG COMMIT_SHA
ENV COMMIT_SHA=${COMMIT_SHA}
-LABEL org.apache.airflow.buildImage.buildId=${BUILD_ID}
-LABEL org.apache.airflow.buildImage.commitSha=${COMMIT_SHA}
+
+LABEL org.apache.airflow.distro.version="buster" \
Review comment:
```suggestion
LABEL org.apache.airflow.distro="debian" \
org.apache.airflow.distro.version="buster" \
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#issuecomment-741681817
> Good spot!
This definitely feels like "this shouldn't matter", but I'll take it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12931: Apply labels to Docker images in a single instruction
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12931:
URL: https://github.com/apache/airflow/pull/12931#discussion_r538863372
##########
File path: Dockerfile
##########
@@ -260,19 +260,19 @@ RUN AIRFLOW_SITE_PACKAGE="/root/.local/lib/python${PYTHON_MAJOR_MINOR_VERSION}/s
RUN find /root/.local -executable -print0 | xargs --null chmod g+x && \
find /root/.local -print0 | xargs --null chmod g+rw
-LABEL org.apache.airflow.distro="debian"
-LABEL org.apache.airflow.distro.version="buster"
-LABEL org.apache.airflow.module="airflow"
-LABEL org.apache.airflow.component="airflow"
-LABEL org.apache.airflow.image="airflow-build-image"
ARG BUILD_ID
ENV BUILD_ID=${BUILD_ID}
ARG COMMIT_SHA
ENV COMMIT_SHA=${COMMIT_SHA}
-LABEL org.apache.airflow.buildImage.buildId=${BUILD_ID}
-LABEL org.apache.airflow.buildImage.commitSha=${COMMIT_SHA}
+
+LABEL org.apache.airflow.distro.version="buster" \
Review comment:
Nope, good spot.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org