You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Qingyou Meng (Jira)" <ji...@apache.org> on 2021/01/15 05:45:00 UTC
[jira] [Comment Edited] (ARROW-11239) [Rust]
array::transform::tests::test_struct failed
[ https://issues.apache.org/jira/browse/ARROW-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17265711#comment-17265711 ]
Qingyou Meng edited comment on ARROW-11239 at 1/15/21, 5:44 AM:
----------------------------------------------------------------
Test passed if construct the `expect` directly (not from slice()).
look like the problem is caused by ArrayRef::slice(), which contains cloned buffers and null bitmap.
1. Add new `array_data_slice_problem` to verify failure case 1:
{noformat}
#[test]
fn array_data_slice_problem() {
let strings: ArrayRef = Arc::new(StringArray::from(vec![
None,
Some("b"),
]));
let strings_slice = strings.slice(1, 1);
let strings2: ArrayRef = Arc::new(StringArray::from(vec![
Some("b"),
]));
assert_eq!(&strings2, &strings_slice, "\ns##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
}
Failed with error:
thread 'array::transform::tests::array_data_slice_problem' panicked at 'assertion failed: `(left == right)`
left: `StringArray
[
"b",
]`,
right: `StringArray
[
"b",
]`:
##1: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 0, buffers: [Buffer { data: Bytes { ptr: 0x7fb5ad504600, len: 8, data: [0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fb5ad504680, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fb5ad504700, len: 1, data: [1] }, offset: 0 } }) }
##2: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 1, buffers: [Buffer { data: Bytes { ptr: 0x7fb5ad504300, len: 12, data: [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fb5ad504480, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fb5ad504380, len: 1, data: [2] }, offset: 0 } }) }'{noformat}
2. Adjust `array_data_slice_problem`, add element `Some("a")`:
{noformat}
#[test]
fn array_data_slice_problem() {
let strings: ArrayRef = Arc::new(StringArray::from(vec![
Some("a"),
None,
Some("b"),
]));
let strings_slice = strings.slice(2, 1);
let strings2: ArrayRef = Arc::new(StringArray::from(vec![
Some("b"),
]));
assert_eq!(&strings2, &strings_slice, "\ns##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
}
Panic with error:
thread 'array::transform::tests::array_data_slice_problem' panicked at 'range end index 2 out of range for slice of length 0',
{noformat}
was (Author: mqy):
Test passed if construct the expect directly.
look like the problem is caused by ArrayRef::slice(), which contains cloned buffers and null bitmap.
1. Add new `array_data_slice_problem` to verify failure case 1:
{noformat}
#[test]
fn array_data_slice_problem() {
let strings: ArrayRef = Arc::new(StringArray::from(vec![
None,
Some("b"),
]));
let strings_slice = strings.slice(1, 1);
let strings2: ArrayRef = Arc::new(StringArray::from(vec![
Some("b"),
]));
assert_eq!(&strings2, &strings_slice, "\ns##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
}
Failed with error:
thread 'array::transform::tests::array_data_slice_problem' panicked at 'assertion failed: `(left == right)`
left: `StringArray
[
"b",
]`,
right: `StringArray
[
"b",
]`:
##1: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 0, buffers: [Buffer { data: Bytes { ptr: 0x7fb5ad504600, len: 8, data: [0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fb5ad504680, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fb5ad504700, len: 1, data: [1] }, offset: 0 } }) }
##2: ArrayData { data_type: Utf8, len: 1, null_count: 0, offset: 1, buffers: [Buffer { data: Bytes { ptr: 0x7fb5ad504300, len: 12, data: [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0] }, offset: 0 }, Buffer { data: Bytes { ptr: 0x7fb5ad504480, len: 1, data: [98] }, offset: 0 }], child_data: [], null_bitmap: Some(Bitmap { bits: Buffer { data: Bytes { ptr: 0x7fb5ad504380, len: 1, data: [2] }, offset: 0 } }) }'{noformat}
2. Adjust `array_data_slice_problem`, add element `Some("a")`:
{noformat}
#[test]
fn array_data_slice_problem() {
let strings: ArrayRef = Arc::new(StringArray::from(vec![
Some("a"),
None,
Some("b"),
]));
let strings_slice = strings.slice(2, 1);
let strings2: ArrayRef = Arc::new(StringArray::from(vec![
Some("b"),
]));
assert_eq!(&strings2, &strings_slice, "\ns##1: {:?}\n##2: {:?}", strings2.data(), strings_slice.data());
}
Panic with error:
thread 'array::transform::tests::array_data_slice_problem' panicked at 'range end index 2 out of range for slice of length 0',
{noformat}
> [Rust] array::transform::tests::test_struct failed
> --------------------------------------------------
>
> Key: ARROW-11239
> URL: https://issues.apache.org/jira/browse/ARROW-11239
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust
> Affects Versions: 3.0.0
> Reporter: Qingyou Meng
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Test *array::transform::tests::test_struct* in *arrow/src/array/equal/mod.rs* failed when swap the first two elements:
> change from
> {code:java}
> // code placeholder
> let strings: ArrayRef = Arc::new(StringArray::from(vec![
> Some("joe"),
> None,{code}
> to
> {code:java}
> // code placeholder
> let strings: ArrayRef = Arc::new(StringArray::from(vec![
> None,
> Some("joe"),{code}
> The failure was first found when I report https://issues.apache.org/jira/browse/ARROW-11160
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)