You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "Alex-PLACET (via GitHub)" <gi...@apache.org> on 2025/05/21 09:58:28 UTC
[PR] Add external library support in Archery [arrow]
Alex-PLACET opened a new pull request, #46530:
URL: https://github.com/apache/arrow/pull/46530
### Rationale for this change
Address #46513:
Add the possibility to use a library which is not part of the "official" implementations, in the integration tests
### What changes are included in this PR?
Add new options in the CLI to be able to define the path to a library to test, and it's compatibility with the tests (IPC producer/consumer, c data array/schema impoter/exporter).
Add tester_external_library.py
Modify runner.py to use ExternalLibraryTester
### Are these changes tested?
Yes localy
### Are there any user-facing changes?
Yes, new options in the CLI and environment variable supported.
there is no breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "Alex-PLACET (via GitHub)" <gi...@apache.org>.
Alex-PLACET commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2100124823
##########
.gitignore:
##########
@@ -108,3 +108,4 @@ java/.mvn/.develocity/
# rat
filtered_rat.txt
rat.txt
+build
Review Comment:
to remove
##########
dev/archery/archery/cli.py:
##########
@@ -17,6 +17,7 @@
from collections import namedtuple
from io import StringIO
+from typing import Optional
Review Comment:
To remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-2897363476
:warning: GitHub issue #46513 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-3223236656
> If we use the YAML approach, I want to replace the all existing testers with YAML configurations.
This sounds an order of magnitude more difficult than simply supporting YAML for third-party (external) implementations. For example, the Java tester uses JPype to load and access the JVM in-process from Python.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "Alex-PLACET (via GitHub)" <gi...@apache.org>.
Alex-PLACET commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2111241542
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
for lang in implementations:
param = f'with_{lang}'
if with_all:
args[param] = with_all
- enabled_implementations += args[param]
+ enabled_implementations = True
- enabled_formats = 0
+ enabled_formats : bool = False
for fmt in formats:
- param = f'run_{fmt}'
- enabled_formats += args[param]
+ enabled_formats = True
if gen_path:
# XXX See GH-37575: this option is only used by the JS test suite
# and might not be useful anymore.
os.makedirs(gen_path, exist_ok=True)
write_js_test_json(gen_path)
else:
- if enabled_formats == 0:
+ if enabled_formats == False:
Review Comment:
I finally simplified the code to raise an error if no format is selected
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
for lang in implementations:
param = f'with_{lang}'
if with_all:
args[param] = with_all
- enabled_implementations += args[param]
+ enabled_implementations = True
- enabled_formats = 0
+ enabled_formats : bool = False
for fmt in formats:
- param = f'run_{fmt}'
- enabled_formats += args[param]
+ enabled_formats = True
if gen_path:
# XXX See GH-37575: this option is only used by the JS test suite
# and might not be useful anymore.
os.makedirs(gen_path, exist_ok=True)
write_js_test_json(gen_path)
else:
- if enabled_formats == 0:
+ if enabled_formats == False:
raise click.UsageError(
"Need to enable at least one format to test "
"(IPC, Flight, C Data Interface); try --help")
- if enabled_implementations == 0:
+ if enabled_implementations == False:
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-3220782502
> If you want, we can close this pull request and create a new one for the YAML later.
What do you think @kou @raulcd ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-2940407895
A more general comment. The approach in this PR is a bit inflexible, as any implementation-specific setting is exposed as a dedicated CLI options. This may proliferate CLI options as we need more settings to be exposed.
For example, some per-implementation skips are currently hardcoded in the data generation ([example](https://github.com/apache/arrow/blob/832bfa1ec67c3a6d43dab5abe7d84271680fee9c/dev/archery/archery/integration/datagen.py#L1901-L1916)). Perhaps we would like an external implementation to tell which tests it needs to skip. That will complicate the CLI quite a bit.
Another possibility is to allow passing a custom YAML file that records all the implementation-specific settings. The CLI would only have one additional argument and the YAML would carry the rest of the information in a structured way.
But of course, Sparrow might remain the only user of this functionality, so it's also up to you. What do you think @Alex-PLACET ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "Alex-PLACET (via GitHub)" <gi...@apache.org>.
Alex-PLACET commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2111239178
##########
dev/archery/archery/cli.py:
##########
@@ -745,6 +745,30 @@ def _set_default(opt, default):
@click.option('--with-rust', type=bool, default=False,
help='Include Rust in integration tests',
envvar="ARCHERY_INTEGRATION_WITH_RUST")
+@click.option('--with-external-library', type=click.Path(exists=True, file_okay=False, dir_okay=True, executable=False),
+ help='Include external library in integration tests',
+ envvar="ARCHERY_INTEGRATION_WITH_EXTERNAL_LIBRARY")
+@click.option('--external-library-IPC-producer', type=bool, default=False,
Review Comment:
Ok fixed
##########
dev/archery/archery/cli.py:
##########
@@ -745,6 +745,30 @@ def _set_default(opt, default):
@click.option('--with-rust', type=bool, default=False,
help='Include Rust in integration tests',
envvar="ARCHERY_INTEGRATION_WITH_RUST")
+@click.option('--with-external-library', type=click.Path(exists=True, file_okay=False, dir_okay=True, executable=False),
+ help='Include external library in integration tests',
+ envvar="ARCHERY_INTEGRATION_WITH_EXTERNAL_LIBRARY")
+@click.option('--external-library-IPC-producer', type=bool, default=False,
+ help='Set external library as supporting producing IPC in integration tests',
+ envvar="ARCHERY_INTEGRATION_EXTERNAL_LIBRARY_IPC_PRODUCER")
+@click.option('--external-library-IPC-consumer', type=bool, default=False,
Review Comment:
Ok fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-3220577711
@Alex-PLACET Would you like to comment on https://github.com/apache/arrow/pull/46530#issuecomment-2940407895 ? I think we'll need something more flexible anyway, because at some points some skips will be required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "Alex-PLACET (via GitHub)" <gi...@apache.org>.
Alex-PLACET commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2111241868
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2110634441
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
for lang in implementations:
param = f'with_{lang}'
if with_all:
args[param] = with_all
- enabled_implementations += args[param]
+ enabled_implementations = True
- enabled_formats = 0
+ enabled_formats : bool = False
for fmt in formats:
- param = f'run_{fmt}'
- enabled_formats += args[param]
+ enabled_formats = True
if gen_path:
# XXX See GH-37575: this option is only used by the JS test suite
# and might not be useful anymore.
os.makedirs(gen_path, exist_ok=True)
write_js_test_json(gen_path)
else:
- if enabled_formats == 0:
+ if enabled_formats == False:
raise click.UsageError(
"Need to enable at least one format to test "
"(IPC, Flight, C Data Interface); try --help")
- if enabled_implementations == 0:
+ if enabled_implementations == False:
Review Comment:
```suggestion
if not enabled_implementations:
```
##########
dev/archery/archery/cli.py:
##########
@@ -745,6 +745,30 @@ def _set_default(opt, default):
@click.option('--with-rust', type=bool, default=False,
help='Include Rust in integration tests',
envvar="ARCHERY_INTEGRATION_WITH_RUST")
+@click.option('--with-external-library', type=click.Path(exists=True, file_okay=False, dir_okay=True, executable=False),
+ help='Include external library in integration tests',
+ envvar="ARCHERY_INTEGRATION_WITH_EXTERNAL_LIBRARY")
+@click.option('--external-library-IPC-producer', type=bool, default=False,
+ help='Set external library as supporting producing IPC in integration tests',
+ envvar="ARCHERY_INTEGRATION_EXTERNAL_LIBRARY_IPC_PRODUCER")
+@click.option('--external-library-IPC-consumer', type=bool, default=False,
Review Comment:
```suggestion
@click.option('--external-library-ipc-consumer', type=bool, default=False,
```
##########
dev/archery/archery/cli.py:
##########
@@ -745,6 +745,30 @@ def _set_default(opt, default):
@click.option('--with-rust', type=bool, default=False,
help='Include Rust in integration tests',
envvar="ARCHERY_INTEGRATION_WITH_RUST")
+@click.option('--with-external-library', type=click.Path(exists=True, file_okay=False, dir_okay=True, executable=False),
+ help='Include external library in integration tests',
+ envvar="ARCHERY_INTEGRATION_WITH_EXTERNAL_LIBRARY")
+@click.option('--external-library-IPC-producer', type=bool, default=False,
Review Comment:
```suggestion
@click.option('--external-library-ipc-producer', type=bool, default=False,
```
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
for lang in implementations:
param = f'with_{lang}'
if with_all:
args[param] = with_all
- enabled_implementations += args[param]
+ enabled_implementations = True
Review Comment:
Do we need this...?
```suggestion
if args[param]:
enabled_implementations = True
```
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
Review Comment:
If we want to use `bool` for this, we may need to rename this something like `have_enabled_implementation`.
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
for lang in implementations:
param = f'with_{lang}'
if with_all:
args[param] = with_all
- enabled_implementations += args[param]
+ enabled_implementations = True
- enabled_formats = 0
+ enabled_formats : bool = False
for fmt in formats:
- param = f'run_{fmt}'
- enabled_formats += args[param]
+ enabled_formats = True
if gen_path:
# XXX See GH-37575: this option is only used by the JS test suite
# and might not be useful anymore.
os.makedirs(gen_path, exist_ok=True)
write_js_test_json(gen_path)
else:
- if enabled_formats == 0:
+ if enabled_formats == False:
Review Comment:
```suggestion
if not enabled_formats:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add external library support in Archery [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-2897360700
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - Contributing Overview](https://arrow.apache.org/docs/developers/overview.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "Alex-PLACET (via GitHub)" <gi...@apache.org>.
Alex-PLACET commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-3220669684
Apologies for the delayed response.
You’re right, the skipping feature is indeed missing. This wasn’t an issue for Sparrow, as its primary goal was full compatibility.
The main purpose of these changes is to enable non-Apache developers to test their libraries in their own CI environments. We’re already using this branch in our CI to run Sparrow against the Apache Arrow C++ implementation.
Using a YAML file could be a good solution, but I don’t have the bandwidth to implement it at the moment.
If you want, we can close this pull request and create a new one for the YAML later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "Alex-PLACET (via GitHub)" <gi...@apache.org>.
Alex-PLACET commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2111241125
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +868,28 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
+ enabled_implementations : bool = False
for lang in implementations:
param = f'with_{lang}'
if with_all:
args[param] = with_all
- enabled_implementations += args[param]
+ enabled_implementations = True
Review Comment:
I finally simplified the code to raise an error if no implementation is selected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-2937936418
Could you add a document how to use this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #46530:
URL: https://github.com/apache/arrow/pull/46530#discussion_r2126837861
##########
dev/archery/archery/cli.py:
##########
@@ -745,6 +766,30 @@ def _set_default(opt, default):
@click.option('--with-rust', type=bool, default=False,
help='Include Rust in integration tests',
envvar="ARCHERY_INTEGRATION_WITH_RUST")
+@click.option('--with-external-library', type=click.Path(exists=True, file_okay=False, dir_okay=True, executable=False),
+ help='Path to the external library to include in integration tests',
+ envvar="ARCHERY_INTEGRATION_WITH_EXTERNAL_LIBRARY")
+@click.option('--external-library-ipc-producer', is_flag=True, default=False, callback=validate_conditional_options,
Review Comment:
Can you ensure you wrap long lines to max 90 ~chars? This makes the code more readable, especially in diff views.
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +889,21 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
- for lang in implementations:
- param = f'with_{lang}'
- if with_all:
- args[param] = with_all
- enabled_implementations += args[param]
-
- enabled_formats = 0
- for fmt in formats:
- param = f'run_{fmt}'
- enabled_formats += args[param]
-
if gen_path:
# XXX See GH-37575: this option is only used by the JS test suite
# and might not be useful anymore.
os.makedirs(gen_path, exist_ok=True)
write_js_test_json(gen_path)
else:
- if enabled_formats == 0:
+ have_enabled_formats: Final[bool] = any(args[f"run_{fmt}"] for fmt in formats)
Review Comment:
The `Final` annotation seems pedantic to me. Besides, `any` returns a boolean, so no annotation should be required.
##########
dev/archery/README.md:
##########
@@ -48,4 +48,90 @@ Additionally, if you would prefer to install everything at once,
`pip install -e "arrow/dev/archery[all]"` is an alias for all of
the above subpackages.
-For some prior art on benchmarking in Arrow, see [this prototype](https://github.com/apache/arrow/tree/0409498819332fc479f8df38babe3426d707fb9e/dev/benchmarking).
\ No newline at end of file
+For some prior art on benchmarking in Arrow, see [this prototype](https://github.com/apache/arrow/tree/0409498819332fc479f8df38babe3426d707fb9e/dev/benchmarking).
+
+# Usage
+## Integration tests
Review Comment:
We already have docs for the Integration tests in https://github.com/apache/arrow/blob/main/docs/source/format/Integration.rst, I suggest augmenting this document rather than the README.
##########
dev/archery/archery/cli.py:
##########
@@ -844,29 +889,21 @@ def integration(with_all=False, random_seed=12345, **args):
implementations = ['cpp', 'csharp', 'java', 'js', 'go', 'nanoarrow', 'rust']
formats = ['ipc', 'flight', 'c_data']
- enabled_implementations = 0
- for lang in implementations:
- param = f'with_{lang}'
- if with_all:
- args[param] = with_all
- enabled_implementations += args[param]
-
- enabled_formats = 0
- for fmt in formats:
- param = f'run_{fmt}'
- enabled_formats += args[param]
-
if gen_path:
# XXX See GH-37575: this option is only used by the JS test suite
# and might not be useful anymore.
os.makedirs(gen_path, exist_ok=True)
write_js_test_json(gen_path)
else:
- if enabled_formats == 0:
+ have_enabled_formats: Final[bool] = any(args[f"run_{fmt}"] for fmt in formats)
+ if not have_enabled_formats:
raise click.UsageError(
"Need to enable at least one format to test "
"(IPC, Flight, C Data Interface); try --help")
- if enabled_implementations == 0:
+ have_enabled_implementation: Final[bool] = with_all or any(
Review Comment:
Same here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-46513: [Archery] Add external library support in Archery [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #46530:
URL: https://github.com/apache/arrow/pull/46530#issuecomment-3222108308
If we use the YAML approach, I want to replace the all existing testers with YAML configurations.
In the case, we may want to specify multiple YAML files. For example:
apache/arrow provides one YAML file for all known implementations:
```bash
archery integration ... --config /.../apache/arrow/dev/archery/data/builtins.yaml ...
```
apache/arrow provides one YAML file per known implementation:
```bash
archery integration ... --config=/.../apache/arrow/dev/archery/data/{cpp,go,js}.yaml ...
```
Each repository provides a YAML file for their implementation:
```bash
archery integration ... \
--config=/.../apache/arrow/dev/archery/data/cpp.yaml
--config=/.../apache/arrow-go/integration/archery.yaml
--config=/.../apache/arrow-js/integration/archery.yaml
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org