You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "AlenkaF (via GitHub)" <gi...@apache.org> on 2023/06/19 11:44:22 UTC
[GitHub] [arrow] AlenkaF opened a new pull request, #36164: GH-35573: [Python]pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
AlenkaF opened a new pull request, #36164:
URL: https://github.com/apache/arrow/pull/36164
### Rationale for this change
`pa.FixedShapeTensorArray.to_numpy_ndarray` fails if called on a sliced `FixedShapeTensorArray`.
### What changes are included in this PR?
The use of `pyarrow.FixedSizeListArray.values` is replaced with `pyarrow.FixedSizeListArray.flatten()` in `FixedShapeTensorArray.to_numpy_ndarray`.
### Are these changes tested?
Yes, test is added to _python/pyarrow/tests/test_extension_type.py_
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] danepitkin commented on pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "danepitkin (via GitHub)" <gi...@apache.org>.
danepitkin commented on PR #36164:
URL: https://github.com/apache/arrow/pull/36164#issuecomment-1607693515
LGTM!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #36164:
URL: https://github.com/apache/arrow/pull/36164#discussion_r1241605059
##########
python/pyarrow/array.pxi:
##########
@@ -3153,7 +3153,7 @@ class FixedShapeTensorArray(ExtensionArray):
Note: ``permutation`` should be trivial (``None`` or ``[0, 1, ..., len(shape)-1]``).
"""
if self.type.permutation is None or self.type.permutation == list(range(len(self.type.shape))):
- np_flat = np.asarray(self.storage.values)
+ np_flat = np.asarray(self.storage.flatten())
Review Comment:
The underlying type (storage type) of the `FixedShapeTensor` is a `FixedSizeList` which always has the `flatten()` API:
https://arrow.apache.org/docs/python/generated/pyarrow.FixedSizeListArray.html#pyarrow.FixedSizeListArray.flatten
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] AlenkaF commented on a diff in pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #36164:
URL: https://github.com/apache/arrow/pull/36164#discussion_r1241605059
##########
python/pyarrow/array.pxi:
##########
@@ -3153,7 +3153,7 @@ class FixedShapeTensorArray(ExtensionArray):
Note: ``permutation`` should be trivial (``None`` or ``[0, 1, ..., len(shape)-1]``).
"""
if self.type.permutation is None or self.type.permutation == list(range(len(self.type.shape))):
- np_flat = np.asarray(self.storage.values)
+ np_flat = np.asarray(self.storage.flatten())
Review Comment:
The underlying type (storage type) of the `FixedShapeTensor` is `FixedSizeList` which always has the `flatten()` API:
https://arrow.apache.org/docs/python/generated/pyarrow.FixedSizeListArray.html#pyarrow.FixedSizeListArray.flatten
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche merged pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche merged PR #36164:
URL: https://github.com/apache/arrow/pull/36164
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #36164:
URL: https://github.com/apache/arrow/pull/36164#issuecomment-1619053006
Conbench analyzed the 6 benchmark runs on commit `14e4d7bc`.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/14748329815) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] danepitkin commented on a diff in pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "danepitkin (via GitHub)" <gi...@apache.org>.
danepitkin commented on code in PR #36164:
URL: https://github.com/apache/arrow/pull/36164#discussion_r1240120921
##########
python/pyarrow/array.pxi:
##########
@@ -3153,7 +3153,7 @@ class FixedShapeTensorArray(ExtensionArray):
Note: ``permutation`` should be trivial (``None`` or ``[0, 1, ..., len(shape)-1]``).
"""
if self.type.permutation is None or self.type.permutation == list(range(len(self.type.shape))):
- np_flat = np.asarray(self.storage.values)
+ np_flat = np.asarray(self.storage.flatten())
Review Comment:
I'm curious, is the underlying `Array` object guaranteed to be chunked and have a `flatten()` API? I tried digging through the code a bit, but had trouble finding the answer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] danepitkin commented on a diff in pull request #36164: GH-35573: [Python] pa.FixedShapeTensorArray.to_numpy_ndarray fails on sliced arrays
Posted by "danepitkin (via GitHub)" <gi...@apache.org>.
danepitkin commented on code in PR #36164:
URL: https://github.com/apache/arrow/pull/36164#discussion_r1242356186
##########
python/pyarrow/array.pxi:
##########
@@ -3153,7 +3153,7 @@ class FixedShapeTensorArray(ExtensionArray):
Note: ``permutation`` should be trivial (``None`` or ``[0, 1, ..., len(shape)-1]``).
"""
if self.type.permutation is None or self.type.permutation == list(range(len(self.type.shape))):
- np_flat = np.asarray(self.storage.values)
+ np_flat = np.asarray(self.storage.flatten())
Review Comment:
Awesome, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org