[GitHub] [arrow] westonpace commented on issue #35126: [Python][C++] Slow table slice when the table has many chunks

["westonpace (via GitHub)" <gi...@apache.org>]

westonpace commented on issue #35126:
URL: https://github.com/apache/arrow/issues/35126#issuecomment-1512520873

   Is the goal to iterate the table batch-by-batch?  Could you use `Table.to_batches` instead?
   
   > Is there a way to tell pyarrow.concat_tables to return a table with a single chunk so I can avoid an extra copy by calling combine_chunks()?
   
   There is not an "extra copy" (at least, not an extra copy of the actual buffers of data).  `concat_tables` is a zero-copy operation in this case.  `combine_chunks` is not.  So these two operations combined will form a single copy.  I don't see much advantage in fusing them into a single method.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org