You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/02 12:39:19 UTC
[GitHub] [airflow] ashb opened a new pull request #9103: Produce less verbose output when building up docker mounts
ashb opened a new pull request #9103:
URL: https://github.com/apache/airflow/pull/9103
The previous method of generating this list had two "problems"/niggles
that this PR solves, when running with VERBOSE=true
- Firstly, LOCAL_MOUNTS was set at the top level, so running with
`set -x` produced 30 extra lines of output when ever the \_util.sh was sourced
- Because of the `while read` used, it created 4 or 5 lines _per_ mount,
resulting in a lot verbose output.
Nothing I've changed here is "critical", it's just making it a bit
easier to see with the debug output what is going on, by running fewer
commands.
I have also expanded the BATS test a little bit to check each pair (`-v`
and its following option)
---
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Target Github ISSUE in description if exists
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb edited a comment on pull request #9103: Produce less verbose output when building up docker mounts
Posted by GitBox <gi...@apache.org>.
ashb edited a comment on pull request #9103:
URL: https://github.com/apache/airflow/pull/9103#issuecomment-637515915
Just double checking this _works_ right when fed to docker :) Yup, was just missing a `/`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #9103: Produce less verbose output when building up docker mounts
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #9103:
URL: https://github.com/apache/airflow/pull/9103#issuecomment-637515915
Just double checking this _works_ right when fed to docker :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #9103: Produce less verbose output when building up docker mounts
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #9103:
URL: https://github.com/apache/airflow/pull/9103
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #9103: Produce less verbose output when building up docker mounts
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #9103:
URL: https://github.com/apache/airflow/pull/9103#issuecomment-637519995
Example verbose output with this change:
```
++ read -r -a EXTRA_DOCKER_FLAGS
+++ convert_local_mounts_to_docker_params
+++ generate_local_mounts_list /home/ash/code/airflow/airflow/
+++ local prefix=/home/ash/code/airflow/airflow/
+++ LOCAL_MOUNTS=("$prefix".bash_aliases:/root/.bash_aliases:cached "$prefix".bash_history:/root/.bash_history:cached "$prefix".coveragerc:/opt/airflow/.coveragerc:cached "$prefix".dockerignore:/opt/airflow/.dockerignore:cached "$prefix".flake8:/opt/airflow/.flake8:cached "$prefix".github:/opt/airflow/.github:cached "$prefix".inputrc:/root/.inputrc:cached "$prefix".kube:/root/.kube:cached "$prefix".rat-excludes:/opt/airflow/.rat-excludes:cached "$prefix"CHANGELOG.txt:/opt/airflow/CHANGELOG.txt:cached "$prefix"Dockerfile.ci:/opt/airflow/Dockerfile.ci:cached "$prefix"LICENSE:/opt/airflow/LICENSE:cached "$prefix"MANIFEST.in:/opt/airflow/MANIFEST.in:cached "$prefix"NOTICE:/opt/airflow/NOTICE:cached "$prefix"airflow:/opt/airflow/airflow:cached "$prefix"backport_packages:/opt/airflow/backport_packages:cached "$prefix"common:/opt/airflow/common:cached "$prefix"dags:/opt/airflow/dags:cached "$prefix"dev:/opt/airflow/dev:cached "$prefix"docs:/opt/airflow/docs:cached "$prefix"files:/files:cached "$prefix"dist:/dist:cached "$prefix"hooks:/opt/airflow/hooks:cached "$prefix"logs:/root/airflow/logs:cached "$prefix"pylintrc:/opt/airflow/pylintrc:cached "$prefix"pytest.ini:/opt/airflow/pytest.ini:cached "$prefix"requirements:/opt/airflow/requirements:cached "$prefix"scripts:/opt/airflow/scripts:cached "$prefix"scripts/ci/in_container/entrypoint_ci.sh:/entrypoint_ci.sh:cached "$prefix"setup.cfg:/opt/airflow/setup.cfg:cached "$prefix"setup.py:/opt/airflow/setup.py:cached "$prefix"tests:/opt/airflow/tests:cached "$prefix"tmp:/opt/airflow/tmp:cached)
+++ printf -- '-v %q ' /home/ash/code/airflow/airflow/.bash_aliases:/root/.bash_aliases:cached /home/ash/code/airflow/airflow/.bash_history:/root/.bash_history:cached /home/ash/code/airflow/airflow/.coveragerc:/opt/airflow/.coveragerc:cached /home/ash/code/airflow/airflow/.dockerignore:/opt/airflow/.dockerignore:cached /home/ash/code/airflow/airflow/.flake8:/opt/airflow/.flake8:cached /home/ash/code/airflow/airflow/.github:/opt/airflow/.github:cached /home/ash/code/airflow/airflow/.inputrc:/root/.inputrc:cached /home/ash/code/airflow/airflow/.kube:/root/.kube:cached /home/ash/code/airflow/airflow/.rat-excludes:/opt/airflow/.rat-excludes:cached /home/ash/code/airflow/airflow/CHANGELOG.txt:/opt/airflow/CHANGELOG.txt:cached /home/ash/code/airflow/airflow/Dockerfile.ci:/opt/airflow/Dockerfile.ci:cached /home/ash/code/airflow/airflow/LICENSE:/opt/airflow/LICENSE:cached /home/ash/code/airflow/airflow/MANIFEST.in:/opt/airflow/MANIFEST.in:cached /home/ash/code/airflow/airflow/NOTICE:/opt/airflow/NOTICE:cached /home/ash/code/airflow/airflow/airflow:/opt/airflow/airflow:cached /home/ash/code/airflow/airflow/backport_packages:/opt/airflow/backport_packages:cached /home/ash/code/airflow/airflow/common:/opt/airflow/common:cached /home/ash/code/airflow/airflow/dags:/opt/airflow/dags:cached /home/ash/code/airflow/airflow/dev:/opt/airflow/dev:cached /home/ash/code/airflow/airflow/docs:/opt/airflow/docs:cached /home/ash/code/airflow/airflow/files:/files:cached /home/ash/code/airflow/airflow/dist:/dist:cached /home/ash/code/airflow/airflow/hooks:/opt/airflow/hooks:cached /home/ash/code/airflow/airflow/logs:/root/airflow/logs:cached /home/ash/code/airflow/airflow/pylintrc:/opt/airflow/pylintrc:cached /home/ash/code/airflow/airflow/pytest.ini:/opt/airflow/pytest.ini:cached /home/ash/code/airflow/airflow/requirements:/opt/airflow/requirements:cached /home/ash/code/airflow/airflow/scripts:/opt/airflow/scripts:cached /home/ash/code/airflow/airflow/scripts/ci/in_container/entrypoint_ci.sh:/entrypoint_ci.sh:cached /home/ash/code/airflow/airflow/setup.cfg:/opt/airflow/setup.cfg:cached /home/ash/code/airflow/airflow/setup.py:/opt/airflow/setup.py:cached /home/ash/code/airflow/airflow/tests:/opt/airflow/tests:cached /home/ash/code/airflow/airflow/tmp:/opt/airflow/tmp:cached
```
Before
```
'
.bash_aliases /root/
.bash_history /root/
.coveragerc /opt/airflow/
.dockerignore /opt/airflow/
.flake8 /opt/airflow/
.github /opt/airflow/
.inputrc /root/
.kube /root/
.rat-excludes /opt/airflow/
CHANGELOG.txt /opt/airflow/
Dockerfile.ci /opt/airflow/
LICENSE /opt/airflow/
MANIFEST.in /opt/airflow/
NOTICE /opt/airflow/
airflow /opt/airflow/
backport_packages /opt/airflow/
common /opt/airflow/
dags /opt/airflow/
dev /opt/airflow/
docs /opt/airflow/
files /
dist /
hooks /opt/airflow/
logs /root/airflow/
pylintrc /opt/airflow/
pytest.ini /opt/airflow/
requirements /opt/airflow/
scripts /opt/airflow/
scripts/ci/in_container/entrypoint_ci.sh /
setup.cfg /opt/airflow/
setup.py /opt/airflow/
tests /opt/airflow/
tmp /opt/airflow/
+++ sed '/^$/d'
'
+++ awk -v AIRFLOW_SOURCES=/home/ash/code/airflow/airflow '
function basename(file) {
sub(".*/", "", file)
return file
}
{ print "-v"; print AIRFLOW_SOURCES "/" $1 ":" $2 basename($1) ":cached" }'
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
++ EXTRA_DOCKER_FLAGS+=("${LINE}")
++ IFS=
++ read -r LINE
```
(and that's not even all of them)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org