[I] `make_array` of a sliced sparse union `ArrayData` reads unsliced children [arrow-rs]

["mbutrovich (via GitHub)" <gi...@apache.org>]

mbutrovich opened a new issue, #11582:
URL: https://github.com/apache/arrow-rs/issues/11582

   ### Describe the bug
   
   Converting sliced `ArrayData` of a sparse union back to an array returns values from the wrong rows. `UnionArray::slice` is correct, and so are dense unions.
   
   ### To Reproduce
   
   ```rust
   use std::sync::Arc;
   use arrow_array::{make_array, Array, ArrayRef, Int32Array, UnionArray};
   use arrow_schema::{DataType, Field, UnionFields};
   
   fn main() {
       let fields = UnionFields::try_new([0], [Field::new("a", DataType::Int32, true)]).unwrap();
       let child: ArrayRef = Arc::new(Int32Array::from(vec![10, 20, 30]));
       let union = UnionArray::try_new(fields, vec![0i8, 0, 0].into(), None, vec![child]).unwrap();
   
       let sliced = make_array(union.to_data().slice(1, 1));
       let sliced = sliced.as_any().downcast_ref::<UnionArray>().unwrap();
       println!("{:?}", sliced.value(0)); // [10], expected [20]
   }
   ```
   
   ### Expected behavior
   
   `make_array(union.to_data().slice(1, 1))` holds the union's second row, `[20]`, as `union.slice(1, 1)` does.
   
   ### Additional context
   
   `From<ArrayData> for UnionArray` slices the type ids by the data offset but builds each child with `make_array(cd)`, unsliced ([union_array.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-array/src/array/union_array.rs#L689-L709)). For a dense union the offsets buffer locates the child rows, so the offset is accounted for. A sparse union's children are indexed by row ([Columnar.rst](https://github.com/apache/arrow/blob/fcf8f3014227cafe24deeb044c2d51f5fc9902e5/docs/source/format/Columnar.rst?plain=1#L942-L944)), so they need the same slice.
   
   Sliced `ArrayData` of a sparse union also arrives this way from `ArrayData::slice` callers and from FFI producers that export an offset ([CDataInterface.rst](https://github.com/apache/arrow/blob/6b4c5248127881c506d4e21e6c95a5c4ced5d0d4/docs/source/format/CDataInterface.rst?plain=1#L456-L459)). An IPC round trip of a sliced `List` of a sparse union fails, which may have the same cause (not verified; see the IPC issue).
   
   The unsliced children also reach `logical_nulls`. If the child above is `[Some(10), None, Some(30)]`, `is_null` on the sliced array returns 3 rows for a 1-row array, because the one-field path returns the child's nulls as they are ([union_array.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-array/src/array/union_array.rs#L797-L809)).
   
   Found with property-based tests that build arrays in every physical layout the Arrow format allows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] `make_array` of a sliced sparse union `ArrayData` reads unsliced children [arrow-rs]

["Jefffrey (via GitHub)" <gi...@apache.org>]

Jefffrey closed issue #11582: `make_array` of a sliced sparse union `ArrayData` reads unsliced children
URL: https://github.com/apache/arrow-rs/issues/11582


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org