You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "0x26res (via GitHub)" <gi...@apache.org> on 2023/09/15 08:48:14 UTC

[GitHub] [arrow] 0x26res opened a new issue, #37733: ListArray.values don't take offset into consideration

0x26res opened a new issue, #37733:
URL: https://github.com/apache/arrow/issues/37733

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   According to the doc for [Array.offset](https://arrow.apache.org/docs/python/generated/pyarrow.Array.html#pyarrow.Array.offset):
   
   > A relative position into another array’s data.
   >
   > The purpose is to enable zero-copy slicing. This value defaults to zero but must be applied on all operations with the physical storage buffers.
   
   So in particular "must be applied on all operations with the physical storage buffers."
   
   I'm wondering if it should be applied to the `ListArray.values`.
   
   Here's an example:
   
   ```
   import pyarrow as pa
   
   values = [[1], [1, 2], [1, 2, 3]]
   array = pa.array(values)
   assert array.to_pylist() == values
   assert array.values.to_pylist() == [1, 1, 2, 1, 2, 3]
   
   slice = array[1:]
   assert slice.to_pylist() == [[1, 2], [1, 2, 3]]
   assert slice.values == array.values  # Wrong Should skip the first value
   ```
   
   The work around is to calculate the values offset my self, by looking at  `ListArray.offsets` at position `ListArray.offset`, but it's not straightforward.
   
   Alternatively if `ListArray.values` isn't going to respect `ListArray.offset` it should be documented [here](https://arrow.apache.org/docs/python/generated/pyarrow.ListArray.html#pyarrow.ListArray.values) 
   
   
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] 0x26res closed issue #37733: ListArray.values don't take offset into consideration

Posted by "0x26res (via GitHub)" <gi...@apache.org>.
0x26res closed issue #37733: ListArray.values don't take offset into consideration
URL: https://github.com/apache/arrow/issues/37733


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] 0x26res commented on issue #37733: ListArray.values don't take offset into consideration

Posted by "0x26res (via GitHub)" <gi...@apache.org>.
0x26res commented on issue #37733:
URL: https://github.com/apache/arrow/issues/37733#issuecomment-1721165121

   @js8544 thanks, that's helpful.
   
   I actually now realize why the underlying `ListArray.values` ignores the offset. It makes sense as long as `ListArray.offsets` takes into consideration the `ListArray.offset`. 
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] js8544 commented on issue #37733: ListArray.values don't take offset into consideration

Posted by "js8544 (via GitHub)" <gi...@apache.org>.
js8544 commented on issue #37733:
URL: https://github.com/apache/arrow/issues/37733#issuecomment-1721039488

   The doc for ListArray.values was added recently in #35865. You can access it via the dev version of doc: https://arrow.apache.org/docs/dev/python/generated/pyarrow.FixedSizeListArray.html#pyarrow.FixedSizeListArray.values


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org