You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "js8544 (via GitHub)" <gi...@apache.org> on 2023/08/23 07:51:48 UTC

[GitHub] [arrow] js8544 commented on issue #37318: [C++][Python] pyarrow.ChunkedArray.combine_chunks is slow

js8544 commented on issue #37318:
URL: https://github.com/apache/arrow/issues/37318#issuecomment-1689462920

   > Hmm, interesting. Ideally on the C++ side we would change concat_arrays to not allocate a new array if there's only one chunk. But that would actually be a breaking change, since that behavior has been used for years as the only way to copy an array (sometimes you explicitly want a copy for memory usage reasons). So we're in a bit of a bind on the C++ side.
   
   I agree that we shouldn't change `Concatenate` because that would be breaking. But I think this particular optimization can happen within `combine_chunks` without changing the behavior of `Concatenate`. C++'s `table` has [the same optimization](https://github.com/apache/arrow/blob/main/cpp/src/arrow/table.cc#L546) as the one OP suggested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org