You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/24 11:13:43 UTC

[GitHub] [arrow] NightMachinary commented on issue #4802: pyarrow / pandas support for tensors (multi-dimensional arrays)

NightMachinary commented on issue #4802:
URL: https://github.com/apache/arrow/issues/4802#issuecomment-1019985875


   I have a `(10**6, 10**4)` numpy array. I want to store this in an efficient format and be able to load batches from this file to use in an out-of-core algorithm. Is Arrow suitable for this purpose? How do I do it?
   
   PS: I actually generate the big numpy array as 10 parts of `(10**5, 10**4)`; so I need to be able to append to the storage file.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org