You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2026/10/04 06:11:57 UTC

[PR] fix(arrow-data): account for parent offset in FixedSizeList null validation [arrow-rs]

viirya opened a new pull request, #11362:
URL: https://github.com/apache/arrow-rs/pull/11362

   # Which issue does this PR close?
   
   No linked issue.
   
   # Rationale for this change
   
   FixedSizeList nullability validation compares expanded parent validity with the unsliced child validity, ignoring the parent offset. This can both reject valid arrays and accept nulls in non-nullable items.
   
   For example, with list size 2, parent offset 1 and length 2, and child values `[0, 1, null, null, 4, 5]`, parent validity `[false, true]` should be accepted but is rejected; `[true, false]` should be rejected but is accepted. When the parent has no null buffer, child nulls outside the visible range can also cause an incorrect rejection.
   
   # What changes are included in this PR?
   
   - Slice child validity to `offset * list_size .. (offset + len) * list_size` before checking non-nullable items.
   - Include the parent offset in the required child length check so validation rejects undersized children before slicing.
   - Add regression coverage for masked and visible child nulls, child offsets, absent parent null buffers, nulls outside the visible range, empty slices, zero-sized lists, and undersized children.
   
   # Are these changes tested?
   
   All three new regression tests failed before the fix and pass afterward.
   
   - `cargo test -p arrow-buffer -p arrow-data -p arrow-array -p arrow-select -p arrow-ipc --lib`: 1,690 passed, 1 ignored.
   - `cargo clippy -p arrow-data --all-targets -- -D warnings`: passed.
   - `cargo fmt --all -- --check`: passed.
   - `git diff --check`: passed.
   
   # Are there any user-facing changes?
   
   Validation now correctly accepts child nulls masked by a sliced FixedSizeList parent or outside its visible range, and rejects visible nulls in non-nullable items. Undersized child arrays are rejected when the parent offset requires additional elements. No public API changes.
   
   # AI assistance
   
   AI assistance was used to investigate the validation semantics, implement the fix and regression tests, review the diff, run the checks above, and prepare this PR description.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(arrow-data): account for parent offset in FixedSizeList null validation [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #11362:
URL: https://github.com/apache/arrow-rs/pull/11362#issuecomment-5977867841

   thanks @viirya 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(arrow-data): account for parent offset in FixedSizeList null validation [arrow-rs]

Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #11362:
URL: https://github.com/apache/arrow-rs/pull/11362#issuecomment-5981963386

   Thank you @Jefffrey 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(arrow-data): account for parent offset in FixedSizeList null validation [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey merged PR #11362:
URL: https://github.com/apache/arrow-rs/pull/11362


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org