[GitHub] [arrow] jorisvandenbossche commented on issue #26685: [Python] Pickling a sliced array serializes all the buffers

["jorisvandenbossche (via GitHub)" <gi...@apache.org>]

jorisvandenbossche commented on issue #26685:
URL: https://github.com/apache/arrow/issues/26685#issuecomment-1604429524

   I am not sure this issue prevents Ray from updating. First, this issue is not causing a "broken" package, but AFAIK it's only a performance issue (that can be worked around with a copy). Further, looking at the Ray PRs linked above, it seems they actually implemented a workaround on their side by using IPC to support Arrow 7+ (https://github.com/ray-project/ray/pull/29055). And the comment in Ray's setup.py mentions _"Serialization workaround for pyarrow 7.0.0+ doesn't work for Windows."_, so mentioning Windows, while this issue is not Windows-specific.
   
   Of course we should still fix this issue, because it is an annoying an unexpected behaviour of pickling arrow data, but so not sure this is the blocker for updating Ray's pyarrow dependency.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org