You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "bkietz (via GitHub)" <gi...@apache.org> on 2024/08/15 20:03:39 UTC
[PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
bkietz opened a new pull request, #43715:
URL: https://github.com/apache/arrow/pull/43715
<!--
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how
to contribute here:
* [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
-->
### Rationale for this change
Nanoarrow can now read and write IPC files as of https://github.com/apache/arrow-nanoarrow/pull/585 so it should no longer be skipped as a producer/consumer
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
### What changes are included in this PR?
Nanoarrow's tester is updated to point to the new integration executable and to report nanoarrow as a consumer/producer of IPC files.
Notably the `null_trivial` case is skipped even though nanoarrow nominally supports it since it represents a corner case in which nanoarrow's flatbuffers library will not accept some vectors produced by other flatbuffers libraries https://github.com/dvidelabs/flatcc/issues/287
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
### Are these changes tested?
Yes
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
### Are there any user-facing changes?
No
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please uncomment the line below and explain which changes are breaking.
-->
<!-- **This PR includes breaking changes to public APIs.** -->
<!--
Please uncomment the line below (and provide explanation) if the changes fix either (a) a security vulnerability, (b) a bug that caused incorrect or invalid data to be produced, or (c) a bug that causes a crash (even when the API contract is upheld). We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
-->
<!-- **This PR contains a "Critical Fix".** -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2372154321
I believe it's https://github.com/apache/arrow-rs/issues/5052 !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2364083755
After https://github.com/apache/arrow-nanoarrow/pull/627 and https://github.com/apache/arrow-nanoarrow/pull/626, I believe all the integration tests targeting the monorepo implementations will pass.
There are still failures for nanoarrow producing/nanoarrow consuming, but I think those are better solved there once this PR is merged since we have the CI job set up to pull from arrow main and build a fresh nanoarrow for each commit on a PR.
```bash
# From a checkout of this PR
export gold_dir=$(pwd)/../arrow-testing/data/arrow-ipc-stream/integration
export ARROW_NANOARROW_PATH=$(pwd)/../arrow-nanoarrow/build
archery integration --with-nanoarrow 1 --run-ipc \
--gold-dirs=$gold_dir/0.14.1 \
--gold-dirs=$gold_dir/0.17.1 \
--gold-dirs=$gold_dir/1.0.0-bigendian \
--gold-dirs=$gold_dir/1.0.0-littleendian \
--gold-dirs=$gold_dir/2.0.0-compression \
--gold-dirs=$gold_dir/4.0.0-shareddict
```
<details>
```
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_primitive_no_batches.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_dictionary.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_null_trivial.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_primitive_large_offsets.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_primitive_zerolength.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_run_end_encoded.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_binary_view.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_nested_dictionary.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_extension.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_map_non_canonical.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_interval.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_dictionary_unsigned.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_null.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_datetime.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_map.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_nested.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_interval_mdn.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_decimal.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_duration.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_list_view.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_recursive_nested.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_union.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_custom_metadata.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_duplicate_fieldnames.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_nested_large_offsets.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_primitive.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/arrow-integration-wbk6m9b5/generated_decimal256.json
-- Creating binary inputs
-- Validating file
-- Validating stream
======================================================================
******************************************************
Tests against golden files in /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1
******************************************************
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_map.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_map.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_map.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000011C
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_dictionary.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_datetime.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_datetime.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_datetime.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000032C
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_decimal.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_decimal.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_decimal.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000008C
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_interval.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_interval.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_interval.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000017C
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_nested.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_nested.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_nested.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000019C
--------------
======================================================================
Command STREAM_TO_FILE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000087C
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_primitive_no_batches.gold.json
-- Validating file
-- Validating stream
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 385, in _run_gold
consumer.stream_to_file(consumer_stream_path, consumer_file_path)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 76, in stream_to_file
self.run_shell_command([_INTEGRATION_EXE, '<', stream_path], env={
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester.py", line 233, in run_shell_command
subprocess.check_call(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration < /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_primitive_no_batches.stream' returned non-zero exit status 22.
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_primitive_zerolength.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_primitive_zerolength.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_primitive_zerolength.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x00000624
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/0.14.1_primitive.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_primitive.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_primitive.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000063C
--------------
======================================================================
******************************************************
Tests against golden files in /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.17.1
******************************************************
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/tmpej5fzvgi/0.17.1_union.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.17.1/generated_union.arrow_file reads identical to /tmp/tmpej5fzvgi/0.17.1_union.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 24 buffers in message but found 28
--------------
======================================================================
******************************************************
Tests against golden files in /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian
******************************************************
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_map.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_map.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_map.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of map array to have length >= 100663296 but found array with length 6
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_datetime.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_datetime.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_datetime.gold.json
Command VALIDATE failed (22=Invalid argument): Found 30 differences between batches:
Path: Batch 0.children[0]
- {"name": "f0", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": [-972622337, -1598018560, -75956737, -1128527616, 1577586688, -1926554880, 0]}
+ {"name": "f0", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": [-719162, 2932896, -165637, 638140, 1050718, 1780621, 0]}
Path: Batch 0.children[1]
- {"name": "f1", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 1, 1], "DATA": ["0", "36303453599170560", "-153002174253498368", "133853394807750656", "-73948351124668416", "-4802460786159452160", "6882072922687537152"]}
+ {"name": "f1", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 1, 1], "DATA": ["0", "253402214400000", "213620221665533", "139715093912321", "138329576765950", "62712364882621", "35475826442847"]}
Path: Batch 0.children[2]
- {"name": "f2", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 0, 1], "DATA": [0, 0, 525926400, -811335680, -391184384, 0, -251854848]}
+ {"name": "f2", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 0, 1], "DATA": [0, 0, 22815, 421
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_nested_dictionary.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_interval.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_interval.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_interval.gold.json
Command VALIDATE failed (22=Invalid argument): Found 12 differences between batches:
Path: Batch 0.children[0]
- {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": ["128", "-129", "-5758134827132877538", "-5984203302733857979", "7152141364016051091", "1946707673701025977", "0"]}
+ {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": ["-9223372036854775808", "9223372036854775807", "2199428923299600048", "4988814404219237292", "-7785723983210331805", "-5113702496382483429", "0"]}
Path: Batch 0.children[1]
- {"name": "f2", "count": 7, "VALIDITY": [1, 1, 0, 1, 1, 0, 1], "DATA": ["128", "-129", "0", "-8588564788114424009", "4714234549317537763", "0", "-915230144468162553"]}
+ {"name": "f2", "count": 7, "VALIDITY": [1, 1, 0, 1, 1, 0, 1], "DATA": ["-9223372036854775808", "9223372036854775807", "0", "4034946345919434632", "-2085181525232227263", "0", "569773930746957043"]}
Path: Batch 0.children[2]
- {"name": "f3", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 1, 1], "DATA": ["0", "0", "-91386228020645345", "8297967703166746363",
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_custom_metadata.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_dictionary_unsigned.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_primitive.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive.gold.json
Command VALIDATE failed (22=Invalid argument): Expected binary array buffer 2 to have size >= 369098752 bytes but found buffer with 22 bytes
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive_large_offsets.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_primitive_large_offsets.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive_large_offsets.gold.json
Command VALIDATE failed (22=Invalid argument): Expected large_binary array buffer 2 to have size >= 1369094286720630784 bytes but found buffer with 19 bytes
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_duplicate_fieldnames.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_duplicate_fieldnames.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_duplicate_fieldnames.gold.json
Command VALIDATE failed (22=Invalid argument): Expected string array buffer 2 to have size >= 134217728 bytes but found buffer with 8 bytes
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_union.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_union.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_union.gold.json
Command VALIDATE failed (22=Invalid argument): Expected string array buffer 2 to have size >= 822083584 bytes but found buffer with 49 bytes
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_null.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_null.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_null.gold.json
Command VALIDATE failed (22=Invalid argument): Found 2 differences between batches:
Path: Batch 0.children[1]
- {"name": "f1", "count": 10, "VALIDITY": [0, 1, 0, 1, 1, 0, 1, 1, 1, 0], "DATA": [0, -129, 0, 1473345331, -397256497, 0, -1669323424, -628195398, -1620465554, 0]}
+ {"name": "f1", "count": 10, "VALIDITY": [0, 1, 0, 1, 1, 0, 1, 1, 1, 0], "DATA": [0, 2147483647, 0, 863490391, -816295192, 0, 1613070492, -1166045478, 1856530847, 0]}
Path: Batch 0.children[3]
- {"name": "f3", "count": 10, "VALIDITY": [0, 0, 1, 1, 1, 0, 1, 0, 0, 0], "DATA": [0.0, 0.0, 1.9002189372376783e-109, 9.829027743251749e-178, -2.1175806223511997e+56, 0.0, 1.1920392509715109e-14, 0.0, 0.0, 0.0]}
+ {"name": "f3", "count": 10, "VALIDITY": [0, 0, 1, 1, 1, 0, 1, 0, 0, 0], "DATA": [0.0, 0.0, -587.995, 603.128, 1959.738, 0.0, -438.465, 0.0, 0.0, 0.0]}
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_dictionary.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_map_non_canonical.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_map_non_canonical.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_map_non_canonical.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of map array to have length >= 184549376 but found array with length 11
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_extension.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_null_trivial.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_recursive_nested.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_recursive_nested.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_recursive_nested.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of list array to have length >= 134217728 but found array with length 8
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_nested.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_nested.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_nested.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of list array to have length >= 251658240 but found array with length 15
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_nested_large_offsets.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_nested_large_offsets.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_nested_large_offsets.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of large list array to have length >= 1224979098644774912 but found array with length 17
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive_no_batches.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive_zerolength.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_decimal256.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_decimal256.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_decimal256.gold.json
Command VALIDATE failed (22=Invalid argument): Found 1089 differences between batches:
Path: Batch 0.children[0]
- {"name": "f0", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 0, 1], "DATA": ["0", "-37078337769718894962251108817035780212840189605462990638900398314494535663617", "44012908020322448267609753248962852605928833126326122667726357114593162035199", "-54009362295389357650022329187996731063181254861555557649287163238231163535361", "-12982109917889636080410433322026198813372849684931462221804907743398852558849", "0", "37865473646572296991777977983078515742704918403723945086956043389824762118143"]}
+ {"name": "f0", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 0, 1], "DATA": ["0", "-41942369422925886428931794240078936402", "-114321348277747672141566107601624871327", "-7473422529517336432944396399317313656", "-31083044846781926940152882137004880669", "0", "-140830396400176073233117491079825213613"]}
Path: Batch 0.children[1]
- {"name": "f1", "count": 7, "VALIDITY": [0, 0, 0, 0, 1, 0, 0], "DATA": ["0", "0", "0", "0", "-1620272279296278412426461366559779130214349
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-bigendian_decimal.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 22.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_decimal.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_decimal.gold.json
Command VALIDATE failed (22=Invalid argument): Found 1295 differences between batches:
Path: Batch 0.children[0]
- {"name": "f0", "count": 7, "VALIDITY": [1, 0, 0, 0, 0, 1, 0], "DATA": ["13551894800775900110464595419264450560", "0", "0", "0", "0", "156199866395303219549082921071928147967", "0"]}
+ {"name": "f0", "count": 7, "VALIDITY": [1, 0, 0, 0, 0, 1, 0], "DATA": ["12810", "0", "0", "0", "0", "-32139", "0"]}
Path: Batch 0.children[1]
- {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 1], "DATA": ["-93715765999695103867346928246093512704", "98669217202737329424965021744169484288", "126832235363430234482114433833859284992", "17581117162998926350204260570739245056", "70423122292307867125759121713212162047", "90517311134837650048172142507293933568", "-130139728462316919681488359995572486144"]}
+ {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 1], "DATA": ["32697", "15178", "27487", "14861", "-1484", "6468", "6302"]}
Path: Batch 0.children[2]
- {"name": "f2", "count": 7, "VALIDITY": [0, 1, 0, 0, 0, 1, 1], "DATA": ["0", "154855304897857
--------------
======================================================================
******************************************************
Tests against golden files in /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-littleendian
******************************************************
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_map.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_nested_dictionary.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_datetime.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_dictionary_unsigned.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_custom_metadata.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_interval.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_null_trivial.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_primitive_large_offsets.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_null.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_primitive_no_batches.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_duplicate_fieldnames.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_dictionary.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_extension.gold.json
-- Skipping test because producer nanoarrow does not support IPC
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_union.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_map_non_canonical.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_recursive_nested.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_nested.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_nested_large_offsets.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_primitive_zerolength.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_primitive.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_decimal256.gold.json
-- Validating file
-- Validating stream
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/1.0.0-littleendian_decimal.gold.json
-- Validating file
-- Validating stream
======================================================================
******************************************************
Tests against golden files in /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression
******************************************************
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/tmpej5fzvgi/2.0.0-compression_lz4.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 95.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_lz4.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_lz4.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/2.0.0-compression_uncompressible_lz4.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 95.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_uncompressible_lz4.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_uncompressible_lz4.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/2.0.0-compression_zstd.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 95.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_zstd.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_zstd.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
======================================================================
======================================================================
Testing file /tmp/tmpej5fzvgi/2.0.0-compression_uncompressible_zstd.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 95.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_uncompressible_zstd.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_uncompressible_zstd.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
======================================================================
******************************************************
Tests against golden files in /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/4.0.0-shareddict
******************************************************
##########################################################
IPC: nanoarrow producing, nanoarrow consuming
##########################################################
======================================================================
Testing file /tmp/tmpej5fzvgi/4.0.0-shareddict_shared_dict.gold.json
-- Validating file
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 136, in run_cmd
output = subprocess.check_output(cmd, **kwargs)
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration']' returned non-zero exit status 95.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 307, in _run_ipc_test_case
run_binaries(producer, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 129, in run_gold
return self._run_gold(gold_dir, consumer, test_case)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/runner.py", line 373, in _run_gold
consumer.validate(json_path, producer_file_path,
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 70, in validate
return self._run(arrow_path, json_path, 'VALIDATE', quirks)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/tester_nanoarrow.py", line 67, in _run
run_cmd([_INTEGRATION_EXE], env=env)
File "/home/dewey/gh/arrow/dev/archery/archery/integration/util.py", line 145, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/4.0.0-shareddict/generated_shared_dict.arrow_file reads identical to /tmp/tmpej5fzvgi/4.0.0-shareddict_shared_dict.gold.json
Command VALIDATE failed (95=Operation not supported): Schema message field with DictionaryEncoding not supported
--------------
======================================================================
################# FAILURES #################
FAILED TEST: datetime nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_datetime.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_datetime.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000032C
--------------
FAILED TEST: map nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_map.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_map.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000011C
--------------
FAILED TEST: primitive nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_primitive.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_primitive.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000063C
--------------
FAILED TEST: interval nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_interval.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_interval.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000017C
--------------
FAILED TEST: decimal nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_decimal.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_decimal.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000008C
--------------
FAILED TEST: primitive_no_batches nanoarrow producing, nanoarrow consuming
<class 'subprocess.CalledProcessError'>: Command '/home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration < /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_primitive_no_batches.stream' returned non-zero exit status 22.
FAILED TEST: nested nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_nested.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_nested.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000019C
--------------
FAILED TEST: primitive_zerolength nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_primitive_zerolength.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_primitive_zerolength.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x00000624
--------------
FAILED TEST: union nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.17.1/generated_union.arrow_file reads identical to /tmp/tmpej5fzvgi/0.17.1_union.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 24 buffers in message but found 28
--------------
FAILED TEST: datetime nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_datetime.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_datetime.gold.json
Command VALIDATE failed (22=Invalid argument): Found 30 differences between batches:
Path: Batch 0.children[0]
- {"name": "f0", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": [-972622337, -1598018560, -75956737, -1128527616, 1577586688, -1926554880, 0]}
+ {"name": "f0", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": [-719162, 2932896, -165637, 638140, 1050718, 1780621, 0]}
Path: Batch 0.children[1]
- {"name": "f1", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 1, 1], "DATA": ["0", "36303453599170560", "-153002174253498368", "133853394807750656", "-73948351124668416", "-4802460786159452160", "6882072922687537152"]}
+ {"name": "f1", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 1, 1], "DATA": ["0", "253402214400000", "213620221665533", "139715093912321", "138329576765950", "62712364882621", "35475826442847"]}
Path: Batch 0.children[2]
- {"name": "f2", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 0, 1], "DATA": [0, 0, 525926400, -811335680, -391184384, 0, -251854848]}
+ {"name": "f2", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 0, 1], "DATA": [0, 0, 22815, 421
--------------
FAILED TEST: map nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_map.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_map.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of map array to have length >= 100663296 but found array with length 6
--------------
FAILED TEST: decimal256 nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_decimal256.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_decimal256.gold.json
Command VALIDATE failed (22=Invalid argument): Found 1089 differences between batches:
Path: Batch 0.children[0]
- {"name": "f0", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 0, 1], "DATA": ["0", "-37078337769718894962251108817035780212840189605462990638900398314494535663617", "44012908020322448267609753248962852605928833126326122667726357114593162035199", "-54009362295389357650022329187996731063181254861555557649287163238231163535361", "-12982109917889636080410433322026198813372849684931462221804907743398852558849", "0", "37865473646572296991777977983078515742704918403723945086956043389824762118143"]}
+ {"name": "f0", "count": 7, "VALIDITY": [0, 1, 1, 1, 1, 0, 1], "DATA": ["0", "-41942369422925886428931794240078936402", "-114321348277747672141566107601624871327", "-7473422529517336432944396399317313656", "-31083044846781926940152882137004880669", "0", "-140830396400176073233117491079825213613"]}
Path: Batch 0.children[1]
- {"name": "f1", "count": 7, "VALIDITY": [0, 0, 0, 0, 1, 0, 0], "DATA": ["0", "0", "0", "0", "-1620272279296278412426461366559779130214349
--------------
FAILED TEST: primitive nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_primitive.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive.gold.json
Command VALIDATE failed (22=Invalid argument): Expected binary array buffer 2 to have size >= 369098752 bytes but found buffer with 22 bytes
--------------
FAILED TEST: interval nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_interval.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_interval.gold.json
Command VALIDATE failed (22=Invalid argument): Found 12 differences between batches:
Path: Batch 0.children[0]
- {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": ["128", "-129", "-5758134827132877538", "-5984203302733857979", "7152141364016051091", "1946707673701025977", "0"]}
+ {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 0], "DATA": ["-9223372036854775808", "9223372036854775807", "2199428923299600048", "4988814404219237292", "-7785723983210331805", "-5113702496382483429", "0"]}
Path: Batch 0.children[1]
- {"name": "f2", "count": 7, "VALIDITY": [1, 1, 0, 1, 1, 0, 1], "DATA": ["128", "-129", "0", "-8588564788114424009", "4714234549317537763", "0", "-915230144468162553"]}
+ {"name": "f2", "count": 7, "VALIDITY": [1, 1, 0, 1, 1, 0, 1], "DATA": ["-9223372036854775808", "9223372036854775807", "0", "4034946345919434632", "-2085181525232227263", "0", "569773930746957043"]}
Path: Batch 0.children[2]
- {"name": "f3", "count": 7, "VALIDITY": [0, 0, 1, 1, 1, 1, 1], "DATA": ["0", "0", "-91386228020645345", "8297967703166746363",
--------------
FAILED TEST: decimal nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_decimal.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_decimal.gold.json
Command VALIDATE failed (22=Invalid argument): Found 1295 differences between batches:
Path: Batch 0.children[0]
- {"name": "f0", "count": 7, "VALIDITY": [1, 0, 0, 0, 0, 1, 0], "DATA": ["13551894800775900110464595419264450560", "0", "0", "0", "0", "156199866395303219549082921071928147967", "0"]}
+ {"name": "f0", "count": 7, "VALIDITY": [1, 0, 0, 0, 0, 1, 0], "DATA": ["12810", "0", "0", "0", "0", "-32139", "0"]}
Path: Batch 0.children[1]
- {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 1], "DATA": ["-93715765999695103867346928246093512704", "98669217202737329424965021744169484288", "126832235363430234482114433833859284992", "17581117162998926350204260570739245056", "70423122292307867125759121713212162047", "90517311134837650048172142507293933568", "-130139728462316919681488359995572486144"]}
+ {"name": "f1", "count": 7, "VALIDITY": [1, 1, 1, 1, 1, 1, 1], "DATA": ["32697", "15178", "27487", "14861", "-1484", "6468", "6302"]}
Path: Batch 0.children[2]
- {"name": "f2", "count": 7, "VALIDITY": [0, 1, 0, 0, 0, 1, 1], "DATA": ["0", "154855304897857
--------------
FAILED TEST: primitive_large_offsets nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_primitive_large_offsets.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_primitive_large_offsets.gold.json
Command VALIDATE failed (22=Invalid argument): Expected large_binary array buffer 2 to have size >= 1369094286720630784 bytes but found buffer with 19 bytes
--------------
FAILED TEST: duplicate_fieldnames nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_duplicate_fieldnames.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_duplicate_fieldnames.gold.json
Command VALIDATE failed (22=Invalid argument): Expected string array buffer 2 to have size >= 134217728 bytes but found buffer with 8 bytes
--------------
FAILED TEST: union nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_union.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_union.gold.json
Command VALIDATE failed (22=Invalid argument): Expected string array buffer 2 to have size >= 822083584 bytes but found buffer with 49 bytes
--------------
FAILED TEST: null nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_null.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_null.gold.json
Command VALIDATE failed (22=Invalid argument): Found 2 differences between batches:
Path: Batch 0.children[1]
- {"name": "f1", "count": 10, "VALIDITY": [0, 1, 0, 1, 1, 0, 1, 1, 1, 0], "DATA": [0, -129, 0, 1473345331, -397256497, 0, -1669323424, -628195398, -1620465554, 0]}
+ {"name": "f1", "count": 10, "VALIDITY": [0, 1, 0, 1, 1, 0, 1, 1, 1, 0], "DATA": [0, 2147483647, 0, 863490391, -816295192, 0, 1613070492, -1166045478, 1856530847, 0]}
Path: Batch 0.children[3]
- {"name": "f3", "count": 10, "VALIDITY": [0, 0, 1, 1, 1, 0, 1, 0, 0, 0], "DATA": [0.0, 0.0, 1.9002189372376783e-109, 9.829027743251749e-178, -2.1175806223511997e+56, 0.0, 1.1920392509715109e-14, 0.0, 0.0, 0.0]}
+ {"name": "f3", "count": 10, "VALIDITY": [0, 0, 1, 1, 1, 0, 1, 0, 0, 0], "DATA": [0.0, 0.0, -587.995, 603.128, 1959.738, 0.0, -438.465, 0.0, 0.0, 0.0]}
--------------
FAILED TEST: map_non_canonical nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_map_non_canonical.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_map_non_canonical.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of map array to have length >= 184549376 but found array with length 11
--------------
FAILED TEST: recursive_nested nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_recursive_nested.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_recursive_nested.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of list array to have length >= 134217728 but found array with length 8
--------------
FAILED TEST: nested nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_nested.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_nested.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of list array to have length >= 251658240 but found array with length 15
--------------
FAILED TEST: nested_large_offsets nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/1.0.0-bigendian/generated_nested_large_offsets.arrow_file reads identical to /tmp/tmpej5fzvgi/1.0.0-bigendian_nested_large_offsets.gold.json
Command VALIDATE failed (22=Invalid argument): Expected child of large list array to have length >= 1224979098644774912 but found array with length 17
--------------
FAILED TEST: lz4 nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_lz4.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_lz4.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
FAILED TEST: zstd nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_zstd.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_zstd.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
FAILED TEST: uncompressible_lz4 nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_uncompressible_lz4.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_uncompressible_lz4.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
FAILED TEST: uncompressible_zstd nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/2.0.0-compression/generated_uncompressible_zstd.arrow_file reads identical to /tmp/tmpej5fzvgi/2.0.0-compression_uncompressible_zstd.gold.json
Command VALIDATE failed (95=Operation not supported): The nanoarrow_ipc extension does not support compression
--------------
FAILED TEST: shared_dict nanoarrow producing, nanoarrow consuming
<class 'RuntimeError'>: Command failed: /home/dewey/gh/arrow/../arrow-nanoarrow/build/nanoarrow_ipc_integration
With output:
--------------
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/4.0.0-shareddict/generated_shared_dict.arrow_file reads identical to /tmp/tmpej5fzvgi/4.0.0-shareddict_shared_dict.gold.json
Command VALIDATE failed (95=Operation not supported): Schema message field with DictionaryEncoding not supported
--------------
28 failures, 16 skips
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2364144951
This failure is due to nanoarrow's reliance on [post-0.15 stream continuations](https://github.com/bkietz/arrow/blob/de12a7af86261f82d799c4a9aef840d41ec0886b/docs/source/format/Columnar.rst?plain=1#L1224-L1226)
```
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.14.1/generated_map.arrow_file reads identical to /tmp/tmpej5fzvgi/0.14.1_map.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 0xFFFFFFFF at start of message but found 0x0000011C
```
and this one is because [pre 1.0 unions had null bufffers](https://github.com/bkietz/arrow/blob/de12a7af86261f82d799c4a9aef840d41ec0886b/format/Schema.fbs#L47-L50), which adds one more buffer per union field than we expect
```
Validating that /home/dewey/gh/arrow/../arrow-testing/data/arrow-ipc-stream/integration/0.17.1/generated_union.arrow_file reads identical to /tmp/tmpej5fzvgi/0.17.1_union.gold.json
Command VALIDATE failed (22=Invalid argument): Expected 24 buffers in message but found 28
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2361951080
> nanoarrow's reader seems to choke on certain types of offset buffers exported by JS and csharp
https://github.com/apache/arrow-nanoarrow/pull/626
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2371821543
CI failure unrelated. Merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2372134518
nanoarrow / rust integration tests are failing now it seems: https://github.com/apache/arrow-rs/issues/6448
Do we know if the underlying issue is tracked anywhere?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz merged PR #43715:
URL: https://github.com/apache/arrow/pull/43715
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2361864134
Just a few notes from some investigating:
- I reran the workflow since we've fixed the Go nullability issue and at least one issue with big endian footers since the last run
- The Python errors for nanoarrow produce/consume are in archery and can be fixed with
```diff
--- a/dev/archery/archery/integration/tester_nanoarrow.py
+++ b/dev/archery/archery/integration/tester_nanoarrow.py
@@ -56,8 +56,8 @@ class NanoarrowTester(Tester):
'JSON_PATH': json_path,
'COMMAND': command,
**{
- f'QUIRK_{q}': 1
- for q in quirks or []
+ f'QUIRK_{q}': "1"
+ for q in quirks or ""
},
```
- After that, nanoarrow produce/consume with itself passes golden file tests for 1.0.0-littleendian, but not any of the others.
- nanoarrow's reader seems to choke on certain types of offset buffers exported by JS and csharp
- All other failures seem related to nanoarrow's stream-to-file conversion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #43715:
URL: https://github.com/apache/arrow/pull/43715#issuecomment-2364086361
...and there are still Rust failures due to https://github.com/google/flatbuffers/issues/8150 (originally reported as https://github.com/apache/arrow-rs/issues/5052 ).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43715:
URL: https://github.com/apache/arrow/pull/43715#discussion_r1773418536
##########
dev/archery/archery/integration/tester_nanoarrow.py:
##########
@@ -45,17 +50,39 @@ class NanoarrowTester(Tester):
name = "nanoarrow"
+ def _run(self, arrow_path, json_path, command, quirks):
+ env = {
+ 'ARROW_PATH': arrow_path,
+ 'JSON_PATH': json_path,
+ 'COMMAND': command,
+ **{
+ f'QUIRK_{q}': "1"
+ for q in quirks or ""
Review Comment:
Nit (Python strings are sequences, but still)
```suggestion
for q in quirks or ()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43715:
URL: https://github.com/apache/arrow/pull/43715#discussion_r1773462586
##########
dev/archery/archery/integration/tester_nanoarrow.py:
##########
@@ -45,17 +50,39 @@ class NanoarrowTester(Tester):
name = "nanoarrow"
+ def _run(self, arrow_path, json_path, command, quirks):
+ env = {
+ 'ARROW_PATH': arrow_path,
+ 'JSON_PATH': json_path,
+ 'COMMAND': command,
+ **{
+ f'QUIRK_{q}': "1"
+ for q in quirks or ""
Review Comment:
:+1: will merge after CI is green again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43715:
URL: https://github.com/apache/arrow/pull/43715#discussion_r1721555961
##########
dev/archery/archery/integration/datagen.py:
##########
@@ -1876,7 +1876,8 @@ def _temp_path():
generate_null_case([10, 0]),
- generate_null_trivial_case([0, 0]),
+ generate_null_trivial_case([0, 0])
+ .skip_tester('nanoarrow'),
Review Comment:
Can we add issue references for these skips?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43680: [Integration] Unskip nanoarrow in IPC integration tests [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on code in PR #43715:
URL: https://github.com/apache/arrow/pull/43715#discussion_r1769282248
##########
dev/archery/archery/integration/datagen.py:
##########
@@ -1914,12 +1914,15 @@ def _temp_path():
generate_duplicate_fieldnames_case()
.skip_tester('JS'),
- generate_dictionary_case(),
+ generate_dictionary_case()
+ .skip_tester('nanoarrow'),
Review Comment:
The issue reference for all of these skips is https://github.com/apache/arrow-nanoarrow/issues/622 (all of these files contain dictionaries)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org