You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Andrew Lamb (Jira)" <ji...@apache.org> on 2020/12/24 13:25:00 UTC
[jira] [Commented] (ARROW-11027) [Rust] [Parquet] Intermittent test
failure thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid
byte when reading bool'
[ https://issues.apache.org/jira/browse/ARROW-11027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17254541#comment-17254541 ]
Andrew Lamb commented on ARROW-11027:
-------------------------------------
FYI [~gbowyer@fastmail.co.uk]
When tried running with some of the code from https://github.com/apache/arrow/pull/8948 (a previous attempt to figure out what is going on here) the problem still happens on my machine:
{code}
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ git cherry-pick 16051b53cd3a1eebdba7a50356ee49585b69e152
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ bash ~/Documents/loop.sh 2>& 1 | tee /tmp/loop.log | grep panic
thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18
0: std::panicking::begin_panic
at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:505
{code}
When I picked the second commit from that PR the problem still happens
{code}
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ git cherry-pick 04f680e121eed9fe524f60b91faa5e7b64fa9921
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ git cherry-pick 04f680e121eed9fe524f60b91faa5e7b64fa9921
[master 870ff05f9] [Rust][Hack] Replace murmur2-64a with wyhash
Author: Greg Bowyer <gb...@fastmail.co.uk>
Date: Wed Dec 16 17:13:24 2020 -0800
2 files changed, 3 insertions(+), 2 deletions(-)
(arrow_dev) alamb@MacBook-Pro:~/Software/arrow/rust$ bash ~/Documents/loop.sh 2>& 1 | tee /tmp/loop.log | grep panic
thread 'encodings::encoding::tests::test_bool' panicked at 'assertion failed: `(left == right)`
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
1: std::panicking::begin_panic_fmt
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
thread 'encodings::encoding::tests::test_bool' panicked at 'assertion failed: `(left == right)`
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
1: std::panicking::begin_panic_fmt
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
^C
{code}
> [Rust] [Parquet] Intermittent test failure thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool'
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-11027
> URL: https://issues.apache.org/jira/browse/ARROW-11027
> Project: Apache Arrow
> Issue Type: Bug
> Reporter: Andrew Lamb
> Priority: Major
>
> The following parquet tests intermittently fail
> {code}
> test encodings::encoding::tests::test_bool ... FAILED
> thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18
> {code}
> ARROW-10943 tracks an intermittent error in the parquet tests. We think we fixed one error but there is still something wrong.
> I can reproduce this locally at ca685a0c08bb41f43a80e5605e4cc8f9efb77cca by running the following shell script:
> {code}
> #!/bin/bash
> # run a command in a loop
> cd /Users/alamb/Software/arrow2/rust
> export PARQUET_TEST_DATA=`pwd`/../cpp/submodules/parquet-testing/data
> export ARROW_TEST_DATA=`pwd`/../testing/data
> export RUST_BACKTRACE=1
> i=0
> while true; do
> echo "iteration $i"
> cargo test -p parquet -- test_bool
> i=$((i+1))
> done
> {code}
> Here is an example of the full stack trace / errors:
> {code}
> test encodings::encoding::tests::test_bool ... FAILED
> failures:
> ---- encodings::encoding::tests::test_bool stdout ----
> thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18
> stack backtrace:
> 0: std::panicking::begin_panic
> at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:505
> 1: <bool as parquet::util::bit_util::FromBytes>::from_ne_bytes
> at ./src/util/bit_util.rs:73
> 2: parquet::util::bit_util::from_ne_slice
> at ./src/util/bit_util.rs:32
> 3: parquet::encodings::rle::RleDecoder::get_batch
> at ./src/encodings/rle.rs:402
> 4: <parquet::encodings::decoding::RleValueDecoder<T> as parquet::encodings::decoding::Decoder<T>>::get
> at ./src/encodings/decoding.rs:322
> 5: parquet::encodings::encoding::tests::put_and_get
> at ./src/encodings/encoding.rs:1271
> 6: <T as parquet::encodings::encoding::tests::EncodingTester<T>>::test_internal
> at ./src/encodings/encoding.rs:1214
> 7: parquet::encodings::encoding::tests::EncodingTester::test
> at ./src/encodings/encoding.rs:1159
> 8: parquet::encodings::encoding::tests::test_bool
> at ./src/encodings/encoding.rs:969
> 9: parquet::encodings::encoding::tests::test_bool::{{closure}}
> at ./src/encodings/encoding.rs:966
> 10: core::ops::function::FnOnce::call_once
> at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
> 11: core::ops::function::FnOnce::call_once
> at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227
> note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)