You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "lccnl (via GitHub)" <gi...@apache.org> on 2023/03/23 11:06:43 UTC

[GitHub] [arrow] lccnl opened a new issue, #34704: [Python] When selecting child array in StructArray nullability is lost

lccnl opened a new issue, #34704:
URL: https://github.com/apache/arrow/issues/34704

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   Hello,
   when a struct array has null values, accessing a child via `array.field` returns an array where all values are valid. 
   This is not the case if the struct is flattened and we take the child in the list returned.
   I tested and observed it with pyarrow 8.0 and 11.0 (did not try other versions).
   
   ```
   import pyarrow as pa
   
   struct=pa.StructArray.from_arrays([pa.array([1])],names=['ex1'])
   struct=pa.concat_arrays([struct,pa.array([None],type=struct.type)])
   
   assert not all(struct.is_valid().to_pylist())
   assert all(struct.field('ex1').is_valid().to_pylist())
   assert not all(struct.flatten()[0].is_valid().to_pylist())```
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorisvandenbossche commented on issue #34704: [Python] When selecting child array in StructArray nullability is lost

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #34704:
URL: https://github.com/apache/arrow/issues/34704#issuecomment-1482996551

   Yes, this is essentially https://github.com/apache/arrow/issues/14970. IMO, we should just change `field(..)` to give the expected result, and add another method to give you the "raw" child.
   
   We can probably keep the discussion at https://github.com/apache/arrow/issues/14970 and close this as a duplicate


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] westonpace commented on issue #34704: [Python] When selecting child array in StructArray nullability is lost

Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on issue #34704:
URL: https://github.com/apache/arrow/issues/34704#issuecomment-1482981311

   >  when a struct array has null values, accessing a child via array.field returns an array where all values are valid.
   
   I believe this is intentional, though perhaps could be better documented.  CC @jorisvandenbossche for a second opinion.
   
   Perhaps we need a `flattened_field` method?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org