You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2023/07/05 10:03:49 UTC
[GitHub] [arrow] raulcd opened a new issue, #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
raulcd opened a new issue, #36476:
URL: https://github.com/apache/arrow/issues/36476
### Describe the bug, including details regarding any error messages, version, and platform.
Some of our nightly verification tasks for integration tests are currently failing:
* [verify-rc-source-integration-linux-almalinux-8-amd64](https://github.com/ursacomputing/crossbow/actions/runs/5454949796/jobs/9925735698)
* [verify-rc-source-integration-linux-ubuntu-22.04-amd64](https://github.com/ursacomputing/crossbow/actions/runs/5454953723/jobs/9925743768)
* [verify-rc-source-integration-macos-amd64](https://github.com/ursacomputing/crossbow/actions/runs/5454948201/jobs/9925731645)
On ` Testing file flight_sql:extension`.
Part of the extract of the log:
```
Testing file flight_sql:extension
==========================================================
Traceback (most recent call last):
File "/arrow/dev/archery/archery/integration/util.py", line 133, in run_cmd
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client', '-host', 'localhost', '-port=46229', '-path', '/tmp/arrow-integration-4wqpenif/generated_primitive.json']' died with <Signals.SIGABRT: 6>.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/arrow/dev/archery/archery/integration/runner.py", line 376, in _run_flight_test_case
consumer.flight_request(port, **client_args)
File "/arrow/dev/archery/archery/integration/tester_cpp.py", line 135, in flight_request
run_cmd(cmd)
File "/arrow/dev/archery/archery/integration/util.py", line 142, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client -host localhost -port=46229 -path /tmp/arrow-integration-4wqpenif/generated_primitive.json
With output:
--------------
Opening JSON file '/tmp/arrow-integration-4wqpenif/generated_primitive.json'
/arrow/cpp/src/arrow/result.cc:28: ValueOrDie called on an error: NotImplemented: Flight returned unimplemented error, with message: No message returned for unary request
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x5f860b)[0x55cb8540060b]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x5f872d)[0x55cb8540072d]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x51b776)[0x55cb85323776]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x51b8e0)[0x55cb853238e0]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f855b)[0x55cb8500055b]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f4d46)[0x55cb84ffcd46]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1d5248)[0x55cb84fdd248]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f1952385d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f1952385e40]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f24b5)[0x55cb84ffa4b5]
--------------
Traceback (most recent call last):
File "/arrow/dev/archery/archery/integration/util.py", line 133, in run_cmd
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client', '-host', 'localhost', '-port=39267', '-path', '/tmp/arrow-integration-4wqpenif/generated_primitive_zerolength.json']' died with <Signals.SIGABRT: 6>.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/arrow/dev/archery/archery/integration/runner.py", line 376, in _run_flight_test_case
consumer.flight_request(port, **client_args)
File "/arrow/dev/archery/archery/integration/tester_cpp.py", line 135, in flight_request
run_cmd(cmd)
File "/arrow/dev/archery/archery/integration/util.py", line 142, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client -host localhost -port=39267 -path /tmp/arrow-integration-4wqpenif/generated_primitive_zerolength.json
With output:
--------------
Opening JSON file '/tmp/arrow-integration-4wqpenif/generated_primitive_zerolength.json'
/arrow/cpp/src/arrow/result.cc:28: ValueOrDie called on an error: NotImplemented: Flight returned unimplemented error, with message: No message returned for unary request
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x5f860b)[0x5639bcbcd60b]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x5f872d)[0x5639bcbcd72d]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x51b776)[0x5639bcaf0776]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x51b8e0)[0x5639bcaf08e0]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f855b)[0x5639bc7cd55b]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f4d46)[0x5639bc7c9d46]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1d5248)[0x5639bc7aa248]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f9d23309d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f9d23309e40]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f24b5)[0x5639bc7c74b5]
--------------
Traceback (most recent call last):
File "/arrow/dev/archery/archery/integration/util.py", line 133, in run_cmd
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client', '-host', 'localhost', '-port=40325', '-path', '/tmp/arrow-integration-4wqpenif/generated_primitive_large_offsets.json']' died with <Signals.SIGABRT: 6>.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/arrow/dev/archery/archery/integration/runner.py", line 376, in _run_flight_test_case
consumer.flight_request(port, **client_args)
File "/arrow/dev/archery/archery/integration/tester_cpp.py", line 135, in flight_request
run_cmd(cmd)
File "/arrow/dev/archery/archery/integration/util.py", line 142, in run_cmd
raise RuntimeError(sio.getvalue())
RuntimeError: Command failed: /tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client -host localhost -port=40325 -path /tmp/arrow-integration-4wqpenif/generated_primitive_large_offsets.json
With output:
--------------
Opening JSON file '/tmp/arrow-integration-4wqpenif/generated_primitive_large_offsets.json'
/arrow/cpp/src/arrow/result.cc:28: ValueOrDie called on an error: NotImplemented: Flight returned unimplemented error, with message: No message returned for unary request
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x5f860b)[0x55d94629360b]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x5f872d)[0x55d94629372d]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x51b776)[0x55d9461b6776]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x51b8e0)[0x55d9461b68e0]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f855b)[0x55d945e9355b]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f4d46)[0x55d945e8fd46]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1d5248)[0x55d945e70248]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f01cdad5d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f01cdad5e40]
/tmp/arrow-HEAD.yQTKg/cpp-build/release/flight-test-integration-client(+0x1f24b5)[0x55d945e8d4b5]
```
### Component(s)
Continuous Integration, FlightRPC
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm closed issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm closed issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
URL: https://github.com/apache/arrow/issues/36476
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621871359
Hmm, I can reproduce it now, but frustratingly it goes away if I try to debug things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621663122
I couldn't reproduce on a manual build. Running under Docker now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621443855
CC @lidavidm @kou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621944476
> which is encoded "in zero bytes" in Protobuf via the field tag
Scratch that, I misremembered (maybe Thrift does that?)
Protobuf bools are integers, so they should always be 0 or 1. And the serialization code is all in headers, and are inlinable. So I suppose what happens is this code:
```cpp
inline uint8_t* WireFormatLite::WriteBoolNoTagToArray(bool value,
uint8_t* target) {
return io::CodedOutputStream::WriteVarint32ToArray(value ? 1 : 0, target);
}
```
gets optimized to something like
```cpp
inline uint8_t* WireFormatLite::WriteBoolNoTagToArray(bool value,
uint8_t* target) {
return io::CodedOutputStream::WriteVarint32ToArray(static_cast<int32_t>(value), target);
}
```
and then an uninitialized boolean value will get dumped to the wire as an invalid Protobuf.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621737108
I couldn't reproduce this with Docker either, though I just realized I need Ubuntu 22.04 to match CI so I'm trying that next.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621919225
Ok, a packet capture in Wireshark shows that in the crashing case, the server is somehow sending an invalid Protobuf; what should be a boolean field (which is encoded "in zero bytes" in Protobuf via the field tag) is instead sent as a varint. But it's not clear why this happens, and why only sometimes; and we use the Protobuf libraries to implement this, so it shouldn't be under our control I would think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on issue #36476: [CI][Flight] Integration tests for FlightSQL fail on MacOS, almalinux and Ubuntu
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #36476:
URL: https://github.com/apache/arrow/issues/36476#issuecomment-1621925008
ok, it appears to be a read of an uninitialized value...if the new boolean flag is uninitialized, I suppose Protobuf's serializer doesn't always treat it properly. (I'm going to dig further to see what happens...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org