You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "AlenkaF (via GitHub)" <gi...@apache.org> on 2023/06/06 09:47:57 UTC

[GitHub] [arrow] AlenkaF opened a new pull request, #35933: GH-35599: Implement __reduce__ method for FixedShapeTensorType

AlenkaF opened a new pull request, #35933:
URL: https://github.com/apache/arrow/pull/35933

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] AlenkaF commented on a diff in pull request #35933: GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable.

Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #35933:
URL: https://github.com/apache/arrow/pull/35933#discussion_r1226671566


##########
python/pyarrow/types.pxi:
##########
@@ -1586,6 +1586,10 @@ cdef class FixedShapeTensorType(BaseExtensionType):
     def __arrow_ext_class__(self):
         return FixedShapeTensorArray
 
+    def __reduce__(self):

Review Comment:
   As the logic for the `__arrow_ext_deserialize__` of the `FixedShapeTensorType` is happening in C++ we can not use `__arrow_ext_deserialize__` and `__arrow_ext_serialize__` to define `__reduce__` here.
   
   But we can do that for `ExtensionType class`. I have created a separate issue for it: https://github.com/apache/arrow/issues/36038



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable. [arrow]

Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on PR #35933:
URL: https://github.com/apache/arrow/pull/35933#issuecomment-1842219082

   Opened a new issue: https://github.com/apache/arrow/issues/39094


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable. [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on PR #35933:
URL: https://github.com/apache/arrow/pull/35933#issuecomment-1841288493

   We actually did that somewhat as a follow-up in https://github.com/apache/arrow/pull/36170, defining `__reduce__` on `ExtensionType` (which can be moved to `BaseExtensionType`), but with using the Python APIs instead of the C++ APIs (so using `self.__arrow_ext_deserialize__` instead of `self.ext_type.Deserialize`), which then requires those dunders to be implemented (even when it is wrapping one defined in C++). This can indeed be cleaned up.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable. [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35933:
URL: https://github.com/apache/arrow/pull/35933#issuecomment-1841014923

   Instead of defining `__reduce__` for each concrete extension type, why not define it on `BaseExtensionType` by making of use of C++-level (de)serialization?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] AlenkaF commented on a diff in pull request #35933: GH-35599: Implement __reduce__ method for FixedShapeTensorType

Posted by "AlenkaF (via GitHub)" <gi...@apache.org>.
AlenkaF commented on code in PR #35933:
URL: https://github.com/apache/arrow/pull/35933#discussion_r1219317177


##########
python/pyarrow/types.pxi:
##########
@@ -1586,6 +1586,10 @@ cdef class FixedShapeTensorType(BaseExtensionType):
     def __arrow_ext_class__(self):
         return FixedShapeTensorArray
 
+    def __reduce__(self):

Review Comment:
   I have started with implementing the `__reduce__` method for `FixedShapeTensorType`. I have to think about how to/if it is possible to use the extension type parameters when defining the method on the `BaseExtensionType`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable. [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #35933:
URL: https://github.com/apache/arrow/pull/35933#issuecomment-1841292655

   A cleanup would nice indeed, to avoid such [spurious additions](https://github.com/apache/arrow/pull/38008/files#diff-9f6b1034bb57bace094700a1b4748c88e40c9f98aec94a959ca466b5cbce0401).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] ursabot commented on pull request #35933: GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable.

Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #35933:
URL: https://github.com/apache/arrow/pull/35933#issuecomment-1589293116

   Benchmark runs are scheduled for baseline = 98559fe057c75fae235c32799a0aac24b587aa52 and contender = 059d8b669aeeda0ab8ac1ae9eab336a93daa3613. 059d8b669aeeda0ab8ac1ae9eab336a93daa3613 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/d7c7790ac64f45e5b604edc146e2ff10...b4eb28cd735942e1a3029f44f938bbab/)
   [Finished :arrow_down:0.62% :arrow_up:0.09%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/a255944112ac436aac7da6a4ea2a548c...f7e07e8df4724c4588320d58a44b4b61/)
   [Finished :arrow_down:0.33% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/83b7fbc9dcd64c6ab52145c2432ac48b...d836c137e6ab4283b140b0106c03f92a/)
   [Finished :arrow_down:0.39% :arrow_up:0.03%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/168e4b9b04a64f3bb0b96073bfd14804...d923a450e4ec46bba381aaa8ade63597/)
   Buildkite builds:
   [Finished] [`059d8b66` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/3030)
   [Finished] [`059d8b66` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/3066)
   [Finished] [`059d8b66` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/3031)
   [Finished] [`059d8b66` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/3056)
   [Finished] [`98559fe0` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/3029)
   [Finished] [`98559fe0` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/3065)
   [Finished] [`98559fe0` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/3030)
   [Finished] [`98559fe0` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/3055)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorisvandenbossche merged pull request #35933: GH-35599: [Python] Canonical fixed-shape tensor extension array/type is not picklable.

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche merged PR #35933:
URL: https://github.com/apache/arrow/pull/35933


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org