You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/07/16 08:43:42 UTC
[GitHub] [arrow] kszucs opened a new pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
kszucs opened a new pull request #7783:
URL: https://github.com/apache/arrow/pull/7783
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455723830
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
Review comment:
Second example?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455681439
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
Review comment:
What's the point of a second example?
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
Review comment:
Please, can you write a test with non-repeated arrays instead?
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
+ assert result.equals(expected)
+
+ indices = pa.chunked_array([[1], [1, 2]])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ [
+ "J"
+ ],
+ [
+ "J",
+ "q"
+ ]
+ ])
+ assert result.equals(expected)
Review comment:
Similarly, what's the point of this test?
##########
File path: python/pyarrow/_compute.pyx
##########
@@ -189,7 +191,7 @@ num_kernels: {}
with nogil:
result = GetResultValue(self.base_func.Execute(c_args,
c_options,
- c_exec_ctx))
+ &c_exec_ctx))
Review comment:
So that was the error? Nice :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455743862
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
+ assert result.equals(expected)
+
+ indices = pa.chunked_array([[1], [1, 2]])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ [
+ "J"
+ ],
+ [
+ "J",
+ "q"
+ ]
+ ])
+ assert result.equals(expected)
Review comment:
Ah, thanks, I see.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455723033
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
Review comment:
To use a chunked array as indices because the number of output chunks depends on it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#issuecomment-659257753
https://issues.apache.org/jira/browse/ARROW-9504
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#issuecomment-659406064
AppVeyor: https://ci.appveyor.com/project/kszucs/arrow/builds/34131280
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455741659
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
+ assert result.equals(expected)
+
+ indices = pa.chunked_array([[1], [1, 2]])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ [
+ "J"
+ ],
+ [
+ "J",
+ "q"
+ ]
+ ])
+ assert result.equals(expected)
Review comment:
The chunking of the output array depends on the indices.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455741020
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
Review comment:
Updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455723338
##########
File path: python/pyarrow/_compute.pyx
##########
@@ -189,7 +191,7 @@ num_kernels: {}
with nogil:
result = GetResultValue(self.base_func.Execute(c_args,
c_options,
- c_exec_ctx))
+ &c_exec_ctx))
Review comment:
It wasn't initialized.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #7783:
URL: https://github.com/apache/arrow/pull/7783
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #7783: ARROW-9504: [C++/Python] Segmentation fault on ChunkedArray.take
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #7783:
URL: https://github.com/apache/arrow/pull/7783#discussion_r455723033
##########
File path: python/pyarrow/tests/test_compute.py
##########
@@ -284,6 +284,66 @@ def test_take_indices_types():
arr.take(indices)
+def test_take_on_chunked_array():
+ # ARROW-9504
+ arr = pa.chunked_array([
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ],
+ [
+ "m",
+ "J",
+ "q",
+ "k",
+ "t"
+ ]
+ ])
+
+ indices = np.array([0, 5, 1, 6, 2, 7, 3, 8, 4, 9])
+ result = arr.take(indices)
+ expected = pa.chunked_array([
+ "m",
+ "m",
+ "J",
+ "J",
+ "q",
+ "q",
+ "k",
+ "k",
+ "t"
+ "t"
+ ])
Review comment:
To use a chunked array as indices because the number of output chunks depends on it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org