You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/29 20:51:49 UTC

[GitHub] [arrow-rs] jorgecarleitao edited a comment on pull request #348: Reduce memory usage of concat (large)utf8

jorgecarleitao edited a comment on pull request #348:
URL: https://github.com/apache/arrow-rs/pull/348#issuecomment-850896662


   I did not have the time to review this, but I think that this will over-allocate in all cases where `MutableArrayData`'s API is used other than concat.
   
   In particular, this will overallocate in:
   
   * `filter` arrays, as it now pre-allocates the maximum size of the string array, even if we only keep a single slot.
   * `zip` kernel: we now allocate `falsy.len() + truthy.len()`, even though we will only pick either from one or the other.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org