You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/29 16:22:32 UTC
[GitHub] [airflow] potiuk opened a new pull request #21211: Avoid changing executable bits in docker scripts
potiuk opened a new pull request #21211:
URL: https://github.com/apache/airflow/pull/21211
There are various problems with executable bits in scripts used
in different environments in docker builds:
* depending on the umask of the Linux host system, group bits
might be set or not - this might lead to Docker cache
invalidation
* on Windows host systems, when file is copied to Docker, the
executable bits might be lost when the file is copied to
docker context
* when AUFS is used as backing storage changing executable bit
might lead to crashes if there is no extra sync
* changing executable bit of the script leads to actual change
of the cache while building, which also might produce
cache invalidation
As the result we cannot rely on the executable bits of the scripts
and cannot change them in the image either.
This change removes executable bits from the scripts for group and
other, and executes all the docker scripts via `bash` command.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #21211: Avoid changing executable bits in docker scripts
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #21211:
URL: https://github.com/apache/airflow/pull/21211#issuecomment-1024941590
This is an interesting one. I found it while testing codespaces. We should really avoid changing executable bits in docker steps in-the middle of docker, as it can cause some unexpected cache invalidations depending on umask in the host or in the docker engine :scream:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21211: Avoid changing executable bits in docker scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21211:
URL: https://github.com/apache/airflow/pull/21211#issuecomment-1024941747
Cache invalidation IS HARD
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #21211: Avoid changing executable bits in docker scripts
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #21211:
URL: https://github.com/apache/airflow/pull/21211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #21211: Avoid changing executable bits in docker scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #21211:
URL: https://github.com/apache/airflow/pull/21211#issuecomment-1024941590
This is an interesting one. I found it while testing codespaces. We should really avoid changing executable bits in docker steps in-the middle of docker, as it can cause some unexpected cache invalidations depending on umask :scream:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21211: Avoid changing executable bits in docker scripts
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21211:
URL: https://github.com/apache/airflow/pull/21211#issuecomment-1024959612
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org