You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/20 13:44:37 UTC
[GitHub] [airflow] potiuk opened a new pull request #20987: Add extra sync when adding executable flag to installation scripts
potiuk opened a new pull request #20987:
URL: https://github.com/apache/airflow/pull/20987
Seems that when AUFS is a backing storage for Docker, changing
the script to executable and executing it right after during the
build phase might cause an error: 'text file busy'
https://github.com/moby/moby/issues/13594
Workaround for that is to add extra `sync` command after changing
the executable flag to make sure that the filesystem change has
propageted to the underlying AUFS storage.
This PR adds the sync and also makes sure that both CI And PROD
image use same formatting, executable bits and `&&` between
commands rather than `;`. The `&&` is better to separate the
commands because it will not continue with execution steps in the
same bash command after previous command fails. This caused
confusion as to what is the reason for docker build failure.
The problem was raised in the #20971 discussion.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #20987: Add extra sync when adding executable flag to installation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #20987:
URL: https://github.com/apache/airflow/pull/20987#discussion_r788877958
##########
File path: Dockerfile
##########
@@ -438,21 +438,23 @@ RUN apt-get update \
COPY scripts/docker/install_mysql.sh /scripts/docker/install_mssql.sh /scripts/docker/
# fix permission issue in Azure DevOps when running the scripts
-RUN chmod a+x /scripts/docker/install_mysql.sh && \
- /scripts/docker/install_mysql.sh prod && \
- chmod a+x /scripts/docker/install_mssql.sh && \
- /scripts/docker/install_mssql.sh && \
- adduser --gecos "First Last,RoomNumber,WorkPhone,HomePhone" --disabled-password \
- --quiet "airflow" --uid "${AIRFLOW_UID}" --gid "0" --home "${AIRFLOW_USER_HOME_DIR}" && \
+RUN chmod a+x /scripts/docker/install_mysql.sh \
+ && sync \
+ && /scripts/docker/install_mysql.sh prod \
+ && chmod a+x /scripts/docker/install_mssql.sh \
+ && sync \
Review comment:
Definitely worth it!
I fixed it and also synchronized and updated comments explaining why we are doing it at all (running chmod+x is for Azure where +x is not carried via filesystem - which is lame but we have to do it for Azure) as this is far from obvious why we are doing it at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #20987: Add extra sync when adding executable flag to installation scripts
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #20987:
URL: https://github.com/apache/airflow/pull/20987#discussion_r788814428
##########
File path: Dockerfile
##########
@@ -438,21 +438,23 @@ RUN apt-get update \
COPY scripts/docker/install_mysql.sh /scripts/docker/install_mssql.sh /scripts/docker/
# fix permission issue in Azure DevOps when running the scripts
-RUN chmod a+x /scripts/docker/install_mysql.sh && \
- /scripts/docker/install_mysql.sh prod && \
- chmod a+x /scripts/docker/install_mssql.sh && \
- /scripts/docker/install_mssql.sh && \
- adduser --gecos "First Last,RoomNumber,WorkPhone,HomePhone" --disabled-password \
- --quiet "airflow" --uid "${AIRFLOW_UID}" --gid "0" --home "${AIRFLOW_USER_HOME_DIR}" && \
+RUN chmod a+x /scripts/docker/install_mysql.sh \
+ && sync \
+ && /scripts/docker/install_mysql.sh prod \
+ && chmod a+x /scripts/docker/install_mssql.sh \
+ && sync \
Review comment:
Perhaps?
```suggestion
RUN chmod a+x /scripts/docker/install_mysql.sh /scripts/docker/install_mssql.sh \
&& sync \
&& /scripts/docker/install_mysql.sh prod \
```
(Either way avoiding two `sync`s probably worth it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20987: Add extra sync when adding executable flag to installation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20987:
URL: https://github.com/apache/airflow/pull/20987#issuecomment-1017625001
Even better now!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #20987: Add extra sync when adding executable flag to installation scripts
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #20987:
URL: https://github.com/apache/airflow/pull/20987#issuecomment-1017673590
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #20987: Add extra sync when adding executable flag to installation scripts
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #20987:
URL: https://github.com/apache/airflow/pull/20987
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20987: Add extra sync when adding executable flag to installation scripts
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20987:
URL: https://github.com/apache/airflow/pull/20987#issuecomment-1017701904
Images built and verified - looks good. Merging :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org