[I] [C++] SparseUnionArray::Make cannot build a valid array from a sliced type_ids array [arrow]

["oversk7 (via GitHub)" <gi...@apache.org>]

oversk7 opened a new issue, #54907:
URL: https://github.com/apache/arrow/issues/54907

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   `SparseUnionArray::Make` uses `type_ids.offset()` as the offset of the new union array, but it checks the children's length against `type_ids.length()` only. As a result, there is no way to build a valid sparse union from a sliced `type_ids` array:
   
   - Children of length `type_ids.length()` are accepted. However, the resulting array is invalid: union slot `i` refers to child position `type_ids.offset() + i`, which these children do not have.
   - Children of length `type_ids.offset() + type_ids.length()`, which the resulting array would need, are rejected.
   
   Reproducer (pyarrow 26.0.0 from PyPI, Python 3.12, Windows 11). `UnionArray.from_sparse` calls `SparseUnionArray::Make` and then `Validate()`:
   
   ```python
   import pyarrow as pa
   
   type_ids = pa.array([0, 1, 0], pa.int8()).slice(1)   # offset 1, length 2
   for n in (2, 3):
       children = [pa.array(range(n), pa.int64()), pa.array(["s"] * n)]
       try:
           pa.UnionArray.from_sparse(type_ids, children)
           print(n, "accepted")
       except Exception as e:
           print(n, e)
   # 2 Sparse union child array #0 has length smaller than expected for union array (2 < 3)
   # 3 Sparse UnionArray must have len(child) == len(type_ids) for all children
   ```
   
   Without the `Validate()` call that pyarrow adds, the first case silently produces an array that maps its slots to the wrong child positions.
   
   Code: [array_nested.cc](https://github.com/apache/arrow/blob/373d64c02cb23663b63c5853099fca43a46c39fc/cpp/src/arrow/array/array_nested.cc#L1396-L1428).
   
   One possible fix is to slice the type ids buffer and build the array with offset 0. Children would then keep corresponding to the logical positions of `type_ids`, as the existing error message ("len(child) == len(type_ids)") suggests. Another option is to require `len(child) == type_ids.offset() + type_ids.length()`.
   
   This was found with an AI-assisted analysis of the C++ code against the columnar format specification. The reproducer above was run against pyarrow 26.0.0.
   
   ### Component(s)
   
   C++
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [C++] SparseUnionArray::Make cannot build a valid array from a sliced type_ids array [arrow]

["JonSnow1807 (via GitHub)" <gi...@apache.org>]

JonSnow1807 commented on issue #54907:
URL: https://github.com/apache/arrow/issues/54907#issuecomment-6344134915

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org