[I] `MutableArrayData::try_extend_nulls` fails on a struct or fixed-size list whose union child has no nullable field [arrow-rs]

["mbutrovich (via GitHub)" <gi...@apache.org>]

mbutrovich opened a new issue, #11836:
URL: https://github.com/apache/arrow-rs/issues/11836

   ### Describe the bug
   
   Since #11209, extending a union with nulls returns an error when no field is nullable ([union.rs](https://github.com/apache/arrow-rs/blob/01a9917658fecd639991e92a5a5810be710b539c/arrow-data/src/transform/union.rs#L22-L34)). A struct or fixed-size list extends every child with nulls ([structure.rs](https://github.com/apache/arrow-rs/blob/01a9917658fecd639991e92a5a5810be710b539c/arrow-data/src/transform/structure.rs#L46-L53), [fixed_size_list.rs](https://github.com/apache/arrow-rs/blob/01a9917658fecd639991e92a5a5810be710b539c/arrow-data/src/transform/fixed_size_list.rs#L58-L68)), so it now fails on such a child, although its own validity can hold the null. `merge_n` with a `None` index reaches it.
   
   ### To Reproduce
   
   ```rust
   use std::sync::Arc;
   
   use arrow_array::{Array, ArrayRef, Int32Array, StructArray, UnionArray};
   use arrow_data::transform::MutableArrayData;
   use arrow_schema::{DataType, Field, UnionFields};
   use arrow_select::merge::merge_n;
   
   fn main() {
       // A struct whose child is a union with no nullable field
       let fields = UnionFields::try_new([0], [Field::new("a", DataType::Int32, false)]).unwrap();
       let union = UnionArray::try_new(fields, vec![0].into(), None, vec![Arc::new(Int32Array::from(vec![1])) as ArrayRef])
           .unwrap();
       let s = StructArray::from(vec![(
           Arc::new(Field::new("u", union.data_type().clone(), false)),
           Arc::new(union) as ArrayRef,
       )]);
   
       println!("{:?}", merge_n(&[&s as &dyn Array], &[Some(0), None]));
       // Err(InvalidArgumentError("struct column 0 (\"u\") failed: Invalid argument error: Cannot extend a union with no nullable fields with nulls"))
   
       let data = s.to_data();
       let mut m = MutableArrayData::new(vec![&data], true, 2);
       m.extend(0, 0, 1);
       println!("{:?}", m.try_extend_nulls(1));
       // the same error
   }
   ```
   
   ### Expected behavior
   
   `[{u: 1}, null]` from `merge_n`, and `Ok(())` from `try_extend_nulls`.
   
   ### Additional context
   
   This is the `MutableArrayData` form of #11608, which #11731 fixes for `take`.
   
   Found with property-based tests that build arrays in every physical layout the Arrow format allows, on `main` at `01a991765`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] `MutableArrayData::try_extend_nulls` fails on a struct or fixed-size list whose union child has no nullable field [arrow-rs]

["ahlag (via GitHub)" <gi...@apache.org>]

ahlag commented on issue #11836:
URL: https://github.com/apache/arrow-rs/issues/11836#issuecomment-6488750675

   I'd like to take this. My plan is to mirror #11731 on the `MutableArrayData` side: when a struct or fixed-size list extends its children with nulls, a union child with no nullable field gets copies of an existing value instead, since the parent's validity already masks those slots. Other children keep using `try_extend_nulls`.
   
   Two questions before I open a PR:
   1. If every source array is empty there's nothing to copy, so I'd keep returning the error. Is that OK?
   2. Should run-end encoded children whose values field isn't nullable get the same treatment here, or leave that to #11835?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org