You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/06/18 21:06:34 UTC

[GitHub] [arrow] lidavidm edited a comment on pull request #10470: ARROW-12983: [C++][Python][R] Properly overflow to chunked array in Python-to-Arrow conversion

lidavidm edited a comment on pull request #10470:
URL: https://github.com/apache/arrow/pull/10470#issuecomment-863618250


   Hmm, I'm not sure if ExtendAsMuchAsPossible should replace Extend everywhere. For one, if you use Extend, you'll then have to check that the number of items added is equal to the number of items you gave it, and in that case, you'll also lose the actual error (or we'll have to declare `Status ExtendAsMuchAsPossible(..., int64_t* appended)`). 
   
   Maybe what we could do instead is expose a `int64_t Converter::MaxItemsPerChunk(int64_t)` so that the chunked converter can handle all the logic internally, and the converters themselves could assume everything fits in a chunk. Though that's much less flexible, it's basically the current situation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org