You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/07 16:02:48 UTC
[GitHub] [arrow] raulcd opened a new pull request, #14345: ARROW-17953: [Archery] Add archery docker info command
raulcd opened a new pull request, #14345:
URL: https://github.com/apache/arrow/pull/14345
Some usage examples:
```
$ archery docker info debian-go-cgo --only command
Service debian-go-cgo docker compose config:
- command: /bin/bash -c "
/arrow/ci/scripts/go_build.sh /arrow &&
/arrow/ci/scripts/go_test.sh /arrow"
```
or
```
$ archery docker info debian-go-cgo
Service debian-go-cgo docker compose config:
- image: ${REPO}:${ARCH}-debian-${DEBIAN}-go-${GO}-cgo
- build
- context: .
- dockerfile: ci/docker/debian-go-cgo.dockerfile
- cache_from: ['${REPO}:${ARCH}-debian-${DEBIAN}-go-${GO}-cgo']
- args
- base: ${REPO}:${ARCH}-debian-${DEBIAN}-go-${GO}
- shm_size: 2G
- volumes: ['.:/arrow:delegated', '${DOCKER_VOLUME_PREFIX}debian-ccache:/ccache:delegated']
- environment
- ARROW_GO_TESTCGO: 1
- command: /bin/bash -c "
/arrow/ci/scripts/go_build.sh /arrow &&
/arrow/ci/scripts/go_test.sh /arrow"
```
or
```
$ archery docker info ubuntu-cuda-cpp --only dockerfile
Service ubuntu-cuda-cpp docker compose config:
- dockerfile: ci/docker/ubuntu-${UBUNTU}-cpp.dockerfile
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14345:
URL: https://github.com/apache/arrow/pull/14345#discussion_r991314402
##########
dev/archery/archery/docker/cli.py:
##########
@@ -289,3 +289,26 @@ def docker_compose_images(obj):
click.echo('Available images:')
for image in compose.images():
click.echo(f' - {image}')
+
+
+@docker.command('info')
+@click.argument('service_name')
+@click.option('--only', '-o', required=False,
+ help="Show only specific docker-compose key. Examples of keys:"
+ " command, environment, build, dockerfile")
+@click.pass_obj
+def docker_compose_info(obj, service_name, only):
+ """Show docker-compose definition info for service_name.
+
+ SERVICE_NAME is the name of the docker service defined on
+ the docker-compose. Look at `archery docker images` output for names.
+ """
+ compose = obj['compose']
+ try:
+ service = compose.config.raw_config["services"][service_name]
+ except KeyError:
+ click.echo(f'Service name {service_name} could not be found')
Review Comment:
This should behave more as an error, so for example:
```suggestion
click.echo(f'Service name {service_name} could not be found', err=True)
sys.exit(1)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] assignUser commented on pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
assignUser commented on PR #14345:
URL: https://github.com/apache/arrow/pull/14345#issuecomment-1282491296
+1
```
Service conda-cpp docker-compose config:
image: ${REPO}:${ARCH}-conda-cpp
build
context: .
dockerfile: ci/docker/conda-cpp.dockerfile
cache_from: ['${REPO}:${ARCH}-conda-cpp']
args
repo: ${REPO}
arch: ${ARCH}
shm_size: 2G
ulimits
core: ${ULIMIT_CORE}
environment
ARROW_BUILD_BENCHMARKS: ON
ARROW_BUILD_EXAMPLES: ON
ARROW_ENABLE_TIMING_TESTS: <inherited>
ARROW_EXTRA_ERROR_CONTEXT: ON
ARROW_MIMALLOC: ON
ARROW_USE_LD_GOLD: ON
ARROW_USE_PRECOMPILED_HEADERS: ON
BUILD_DOCS_PYTHON: ON
CCACHE_COMPILERCHECK: content
CCACHE_COMPRESS: 1
CCACHE_COMPRESSLEVEL: 6
CCACHE_MAXSIZE: 1G
CCACHE_DIR: /ccache
AWS_ACCESS_KEY_ID: <inherited>
AWS_SECRET_ACCESS_KEY: <inherited>
SCCACHE_BUCKET: <inherited>
SCCACHE_S3_KEY_PREFIX: ${SCCACHE_S3_KEY_PREFIX:-sccache}
volumes: ['.:/arrow:delegated', '${DOCKER_VOLUME_PREFIX}conda-ccache:/ccache:delegated']
command: [' /arrow/ci/scripts/cpp_build.sh /arrow /build && /arrow/ci/scripts/cpp_test.sh /arrow /build']
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14345:
URL: https://github.com/apache/arrow/pull/14345#discussion_r991315398
##########
dev/archery/archery/docker/core.py:
##########
@@ -417,3 +417,19 @@ def _push(service):
def images(self):
return sorted(self.config.hierarchy.keys())
+
+ def info(self, key_name, filters=None, prefix=' '):
+ output = []
+ for key, value in key_name.items():
+ if hasattr(value, 'items'):
+ temp_filters = filters
+ if key == filters or filters is None:
+ output.append(f'{prefix}- {key}')
+ # Keep showing this specific key
+ # as parent matched filter
+ temp_filters = None
+ output.extend(self.info(value, temp_filters, prefix + " "))
Review Comment:
Should probably use two spaces for better and more visible alignment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14345:
URL: https://github.com/apache/arrow/pull/14345#discussion_r991314936
##########
dev/archery/archery/docker/cli.py:
##########
@@ -289,3 +289,26 @@ def docker_compose_images(obj):
click.echo('Available images:')
for image in compose.images():
click.echo(f' - {image}')
+
+
+@docker.command('info')
+@click.argument('service_name')
+@click.option('--only', '-o', required=False,
+ help="Show only specific docker-compose key. Examples of keys:"
+ " command, environment, build, dockerfile")
+@click.pass_obj
+def docker_compose_info(obj, service_name, only):
+ """Show docker-compose definition info for service_name.
+
+ SERVICE_NAME is the name of the docker service defined on
+ the docker-compose. Look at `archery docker images` output for names.
+ """
+ compose = obj['compose']
+ try:
+ service = compose.config.raw_config["services"][service_name]
+ except KeyError:
+ click.echo(f'Service name {service_name} could not be found')
+ else:
+ click.echo(f'Service {service_name} docker compose config:')
Review Comment:
Nit
```suggestion
click.echo(f'Service {service_name} docker-compose config:')
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #14345:
URL: https://github.com/apache/arrow/pull/14345#discussion_r991316321
##########
dev/archery/archery/docker/cli.py:
##########
@@ -289,3 +289,26 @@ def docker_compose_images(obj):
click.echo('Available images:')
for image in compose.images():
click.echo(f' - {image}')
+
+
+@docker.command('info')
+@click.argument('service_name')
+@click.option('--only', '-o', required=False,
Review Comment:
Nit: I would keep the `-o` shortcut for an output file or something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #14345:
URL: https://github.com/apache/arrow/pull/14345#discussion_r994511754
##########
dev/archery/archery/docker/cli.py:
##########
@@ -289,3 +289,26 @@ def docker_compose_images(obj):
click.echo('Available images:')
for image in compose.images():
click.echo(f' - {image}')
+
+
+@docker.command('info')
+@click.argument('service_name')
+@click.option('--only', '-o', required=False,
Review Comment:
I've updated to `-s` or `--show`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #14345:
URL: https://github.com/apache/arrow/pull/14345#issuecomment-1271778983
https://issues.apache.org/jira/browse/ARROW-17953
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #14345:
URL: https://github.com/apache/arrow/pull/14345#issuecomment-1277877682
@pitrou @assignUser this should be ready to review again. Thanks for the feedback!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #14345:
URL: https://github.com/apache/arrow/pull/14345#discussion_r994512491
##########
dev/archery/archery/docker/core.py:
##########
@@ -417,3 +417,19 @@ def _push(service):
def images(self):
return sorted(self.config.hierarchy.keys())
+
+ def info(self, key_name, filters=None, prefix=' '):
+ output = []
+ for key, value in key_name.items():
+ if hasattr(value, 'items'):
+ temp_filters = filters
+ if key == filters or filters is None:
+ output.append(f'{prefix}- {key}')
+ # Keep showing this specific key
+ # as parent matched filter
+ temp_filters = None
+ output.extend(self.info(value, temp_filters, prefix + " "))
Review Comment:
good idea, with that I can remove the `-` and have a more visible alignment:
```
$ archery docker info debian-go-cgo
Service debian-go-cgo docker-compose config:
image: ${REPO}:${ARCH}-debian-${DEBIAN}-go-${GO}-cgo
build
context: .
dockerfile: ci/docker/debian-go-cgo.dockerfile
cache_from: ['${REPO}:${ARCH}-debian-${DEBIAN}-go-${GO}-cgo']
args
base: ${REPO}:${ARCH}-debian-${DEBIAN}-go-${GO}
shm_size: 2G
volumes: ['.:/arrow:delegated', '${DOCKER_VOLUME_PREFIX}debian-ccache:/ccache:delegated']
environment
ARROW_GO_TESTCGO: 1
command: /bin/bash -c "
/arrow/ci/scripts/go_build.sh /arrow &&
/arrow/ci/scripts/go_test.sh /arrow"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou merged pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
pitrou merged PR #14345:
URL: https://github.com/apache/arrow/pull/14345
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #14345: ARROW-17953: [Archery] Add archery docker info command
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #14345:
URL: https://github.com/apache/arrow/pull/14345#issuecomment-1281059610
I've noticed something:
```
$ archery docker info conda-cpp
Service conda-cpp docker-compose config:
image: ${REPO}:${ARCH}-conda-cpp
build
context: .
dockerfile: ci/docker/conda-cpp.dockerfile
cache_from: ['${REPO}:${ARCH}-conda-cpp']
args
repo: ${REPO}
arch: ${ARCH}
shm_size: 2G
ulimits
core: ${ULIMIT_CORE}
environment
ARROW_BUILD_BENCHMARKS: ON
ARROW_BUILD_EXAMPLES: ON
ARROW_ENABLE_TIMING_TESTS: None
ARROW_EXTRA_ERROR_CONTEXT: ON
ARROW_MIMALLOC: ON
ARROW_USE_LD_GOLD: ON
ARROW_USE_PRECOMPILED_HEADERS: ON
BUILD_DOCS_PYTHON: ON
CCACHE_COMPILERCHECK: content
CCACHE_COMPRESS: 1
CCACHE_COMPRESSLEVEL: 6
CCACHE_MAXSIZE: 1G
CCACHE_DIR: /ccache
AWS_ACCESS_KEY_ID: None
AWS_SECRET_ACCESS_KEY: None
SCCACHE_BUCKET: None
SCCACHE_S3_KEY_PREFIX: ${SCCACHE_S3_KEY_PREFIX:-sccache}
volumes: ['.:/arrow:delegated', '${DOCKER_VOLUME_PREFIX}conda-ccache:/ccache:delegated']
command: [' /arrow/ci/scripts/cpp_build.sh /arrow /build && /arrow/ci/scripts/cpp_test.sh /arrow /build']
```
Printing `ARROW_ENABLE_TIMING_TESTS: None` is a bit unfortunate. What happens actually is that the value is inherited from the calling shell's environment. So perhaps print out `<inherited>` or something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org