You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "1932819331hy-debug (via GitHub)" <gi...@apache.org> on 2026/10/16 01:42:47 UTC

[PR] GH-43146: [C++][Python] Validate list value nullability when casting [arrow]

1932819331hy-debug opened a new pull request, #53948:
URL: https://github.com/apache/arrow/pull/53948

   ### Rationale for this change
   
   Casting a list value field to non-nullable currently accepts null values, producing an array whose data contradicts its nested field nullability. For example, `[[1, null, 3], [], [4, 5]]` can be cast to `list<item: int64 not null>`.
   
   Fixes #43146.
   
   ### What changes are included in this PR?
   
   - Validate the converted values in list, large-list, and fixed-size-list casts. Checking the output also catches nulls introduced by dictionary decoding.
   - Check only values referenced by valid parent lists, respecting both parent and child offsets.
   - Propagate ancestor validity before recursive nested casts, including mixed struct/list nesting. Compact hidden map entries with the existing `Flatten` utility so nested keys retain the map's non-null key constraint.
   
   ### Are these changes tested?
   
   - Built `arrow-compute-scalar-cast-test` from the current Arrow 27 source with Apple Clang 17 in Debug mode; all **113 tests passed**.
   - Five new C++ regression tests cover safe/unsafe casts, all list/fixed-size-list conversion directions, dictionary decoding, parent/child slices, unknown null counts, null and empty lists, NullType, and masked descendants through lists, structs, and maps. List-view inputs also have rejection coverage.
   - Added four PyArrow `Table.cast` regression cases for list/large-list and safe/unsafe modes. All four reproduce the missing exception against released PyArrow 26.0.0. The Python extension was not rebuilt locally; the fixed behavior was validated through the native C++ suite.
   - Applied clang-format and checked the diff for whitespace errors.
   
   ### Are there any user-facing changes?
   
   List casts that would produce null values in a non-nullable value field now raise `ArrowInvalid`, including with `safe=False`. The added list-value check respects null ancestors and slice boundaries.
   
   **This PR contains a "Critical Fix"** for casts that currently produce data inconsistent with the declared nested nullability.
   
   ### Was AI used for this PR?
   
   Code, regression tests, and this description were generated with AI assistance. The implementation was independently reviewed by AI agents, exercised against the reported failure, and validated with the native test suite above.
   
   **PR code and description written by:**
   
   - [ ] Human
   - [x] AI
   
   **Reviewed before submission by:**
   
   - [ ] Human
   - [x] AI
   - [ ] Not reviewed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-43146: [C++][Python] Validate list value nullability when casting [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd closed pull request #53948: GH-43146: [C++][Python] Validate list value nullability when casting
URL: https://github.com/apache/arrow/pull/53948


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-43146: [C++][Python] Validate list value nullability when casting [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #53948:
URL: https://github.com/apache/arrow/pull/53948#issuecomment-6206113506

   :warning: GitHub issue #43146 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-43146: [C++][Python] Validate list value nullability when casting [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on PR #53948:
URL: https://github.com/apache/arrow/pull/53948#issuecomment-6216035403

   Fully automated PRs are not welcome, see https://arrow.apache.org/docs/developers/overview.html#ai-generated-code


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org