[GitHub] [airflow] potiuk edited a comment on pull request #12188: Upgrade azure blob to v12

[GitBox <gi...@apache.org>]

potiuk edited a comment on pull request #12188:
URL: https://github.com/apache/airflow/pull/12188#issuecomment-735276782


   That should be rather easy I think. In this case we need to make a little more changes because we do not handle well "uninstalling of a previously installed packages from cache"  but it can be done rather easily:
   
   In the Dockerfile.ci please change those two lines - > change the AIRFLOW_CI_BUILD_EPOCH to 4, and AIRFLOW_PRE_CACHED_PIP_PACKAGES to "false" :
   
   ```
   # By changing the CI build epoch we can force reinstalling Airflow from the current master
   # It can also be overwritten manually by setting the AIRFLOW_CI_BUILD_EPOCH environment variable.
   ARG AIRFLOW_CI_BUILD_EPOCH="3"
   ENV AIRFLOW_CI_BUILD_EPOCH=${AIRFLOW_CI_BUILD_EPOCH}
   
   ARG AIRFLOW_PRE_CACHED_PIP_PACKAGES="true"
   ENV AIRFLOW_PRE_CACHED_PIP_PACKAGES=${AIRFLOW_PRE_CACHED_PIP_PACKAGES}
   ```
   
   What those two changes mean:
   * increasing the EPOCH causes invalidation of Docker Cache from this place. This means that for your PR the dockerfile will be rebuilt starting from that line.
   * setting AIRFLOW_PRE_CACHED_PIP_PACKAGES to "false" disables the optimisation where we use the previously build images as cache for new builds - thanks to that we save a lot of time for image builds, but whatever was originally installed in the image remains there (including 'azure-storage' package).
   
   If you change both  values, your build will run longer, but it will not have the 'azure-storage'  package installed at all (since you removed it from the setup.py).
   
   When we see that all that works, We will just leave the EPOCH change. Once your change will be merge to master, all new builds will use the new master version of setup.py and will not install 'azure-storage'  once the rebase to master and get your EPOCH change.
   
   I hope it helps , ping me on slaack when you re-rerun it and still have some problems.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org