[GitHub] [airflow] potiuk commented on a change in pull request #6373: Depends on [AIRFLOW-5680] [AIRFLOW-5704] Optimise kubernetes image

[GitBox <gi...@apache.org>]

potiuk commented on a change in pull request #6373: Depends on [AIRFLOW-5680] [AIRFLOW-5704] Optimise kubernetes image
URL: https://github.com/apache/airflow/pull/6373#discussion_r336760858
 
 

 ##########
 File path: Dockerfile
 ##########
 @@ -174,39 +170,8 @@ RUN if [[ "${APT_DEPS_IMAGE}" == "airflow-apt-deps-ci" ]]; then \
         ;\
     fi
 
-# TODO: We should think about removing those and moving them into docker-compose dependencies.
 COPY scripts/ci/docker_build/ci_build_install_deps.sh /tmp/ci_build_install_deps.sh
 
-# Kubernetes dependencies
-RUN \
-if [[ "${APT_DEPS_IMAGE}" == "airflow-apt-deps-ci" ]]; then \
 
 Review comment:
   It's not at runtime - it's just grouping all the CI-related installs in single RUN command.
   
   This is mostly to decrease the build time for slim (in the future production) image. The problem is that currently in multi-staging images, even if you have a stage that is not used, it will not be skipped (this will be improved when BUILDKIT goes out of beta and we will be able to use it including caching - in BUILDKIT, unused stages are skipped while building). The "if" there is a hack to skip building CI dependencies if we are only building slim image.
   
   Separating it to single script makes it slightly faster to rebuild the slim image- every RUN command takes couple of seconds when executed so if we have a separate RUN and if in every RUN, we got 20-30 seconds longer builds always (because this stage is not part of the image, so we do not have cache for it, it is also not cached so it is always built - even if we locally built the slim image. Unfortunately, we cannot use both --cache-from and local cache at the same time (https://github.com/moby/moby/issues/32612) - this could also solve the problem.
   
   This will also be useful in the PROD image change I am working on.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services